00:00:00 [---] Coprocessor specified in cc_config.xml. Type NVIDIA (GPU); count 1 00:00:00 [GPUGRID] Found app_config.xml 00:00:00 [GPUGRID] PythonGPU: Max 1 concurrent jobs 00:00:00 [---] Config: GUI RPC allowed from any host 00:00:00 [---] log flags: file_xfer, sched_ops, task, cpu_sched_debug, rr_simulation, work_fetch_debug 00:00:00 [World Community Grid] General prefs: from World Community Grid (last modified 01-Oct-2022 11:21:02) 00:00:00 [World Community Grid] Host location: none 00:00:00 [World Community Grid] General prefs: using your defaults 00:00:00 [---] Reading preferences override file 00:00:00 [---] Preferences: 00:00:00 [---] max memory usage when active: 14678.62 MB 00:00:00 [---] max memory usage when idle: 14678.62 MB 00:00:00 [---] (to change preferences, visit a project web site or select Preferences in the Manager) 00:00:00 [---] [cpu_sched_debug] Request CPU reschedule: Prefs update 00:00:00 [---] [work_fetch] Request work fetch: Prefs update 00:00:00 [Albert@Home] URL https://albert.phys.uwm.edu/; Computer ID 25642; resource share 100 00:00:00 [Amicable Numbers] URL https://sech.me/boinc/Amicable/; Computer ID 176627; resource share 100 00:00:00 [Amicable Numbers] Not using NVIDIA GPU: project preferences 00:00:00 [Asteroids@home] URL http://asteroidsathome.net/boinc/; Computer ID 707148; resource share 100 00:00:00 [Asteroids@home] Not using CPU: project preferences 00:00:00 [BOINC@TACC] URL https://boinc.tacc.utexas.edu/; Computer ID 6478; resource share 100 00:00:00 [climateprediction.net] URL https://climateprediction.net/; Computer ID 1515124; resource share 100 00:00:00 [collatz] URL https://boinc.thesonntags.com/collatz/; Computer ID 873394; resource share 100 00:00:00 [collatz] Not using CPU: project preferences 00:00:00 [Cosmology@Home] URL http://www.cosmologyathome.org/; Computer ID 435337; resource share 100 00:00:00 [DENIS@home] URL https://denis.usj.es/denisathome/; Computer ID 214437; resource share 100 00:00:00 [Einstein@Home] URL http://einstein.phys.uwm.edu/; Computer ID 12116855; resource share 100 00:00:00 [Einstein@Home] Not using CPU: project preferences 00:00:00 [Gaia@home] URL http://gaiaathome.eu/gaiaathome/; Computer ID 3602; resource share 100 00:00:00 [Gerasim@home] URL http://gerasim.boinc.ru/; Computer ID 54788; resource share 100 00:00:00 [GPUGRID] URL https://www.gpugrid.net/; Computer ID 352477; resource share 100 00:00:00 [iThena.Computational] URL https://comp.ithena.net/usr/; Computer ID 233; resource share 100 00:00:00 [iThena.Measurements] URL https://root.ithena.net/usr/; Computer ID 12067; resource share 100 00:00:00 [latinsquares] URL https://boinc.multi-pool.info/latinsquares/; Computer ID 100762; resource share 100 00:00:00 [LHC@home] URL https://lhcathome.cern.ch/lhcathome/; Computer ID 10627860; resource share 100 00:00:00 [lhcathome-dev] URL https://lhcathomedev.cern.ch/lhcathome-dev/; Computer ID 4699; resource share 100 00:00:00 [LODA] URL https://boinc.loda-lang.org/loda/; Computer ID 432; resource share 100 00:00:00 [Milkyway@Home] URL http://milkyway.cs.rpi.edu/milkyway/; Computer ID 880024; resource share 100 00:00:00 [MindModeling] URL http://mindmodeling.org/; Computer ID 145638; resource share 100 00:00:00 [minecrafthome] URL https://minecraftathome.com/minecrafthome/; Computer ID 10404; resource share 100 00:00:00 [MLC@Home] URL https://www.mlcathome.org/mlcathome/; Computer ID 789; resource share 100 00:00:00 [Moo! Wrapper] URL http://moowrap.net/; Computer ID 1309198; resource share 100 00:00:00 [nanoHUB_at_home] URL https://boinc.nanohub.org/nanoHUB_at_home/; Computer ID 9286; resource share 100 00:00:00 [NFS@Home] URL https://escatter11.fullerton.edu/nfs/; Computer ID 7174027; resource share 100 00:00:00 [NumberFields@home] URL http://numberfields.asu.edu/NumberFields/; Computer ID 2126347; resource share 100 00:00:00 [NumberFields@home] Not using CPU: project preferences 00:00:00 [odlk] URL https://boinc.progger.info/odlk/; Computer ID 13373; resource share 100 00:00:00 [PrimeGrid] URL http://www.primegrid.com/; Computer ID 979678; resource share 100 00:00:00 [PrimeGrid] Not using CPU: project preferences 00:00:00 [PRIVATE GFN SERVER] URL http://boincvm.proxyma.ru:30080/test4vm/; Computer ID 2016; resource share 100 00:00:00 [Quake-Catcher Network] URL https://quakecatcher.net/sensor/; Computer ID 88571; resource share 100 00:00:00 [QuChemPedIA@home] URL https://quchempedia.univ-angers.fr/athome/; Computer ID 2708; resource share 100 00:00:00 [Rake search of diagonal Latin squares] URL https://rake.boincfast.ru/rakesearch/; Computer ID 15009; resource share 100 00:00:00 [ralph@home] URL https://ralph.bakerlab.org/; Computer ID 48013; resource share 100 00:00:00 [RamanujanMachine] URL https://rnma.xyz/boinc/; Computer ID 1016; resource share 100 00:00:00 [RNA World] URL http://www.rnaworld.de/rnaworld/; Computer ID 105259; resource share 100 00:00:00 [Rosetta@home] URL https://boinc.bakerlab.org/rosetta/; Computer ID 6194079; resource share 100 00:00:00 [SiDock@home] URL https://www.sidock.si/sidock/; Computer ID 7174; resource share 100 00:00:00 [SRBase] URL http://srbase.my-firewall.org/sr5/; Computer ID 212640; resource share 100 00:00:00 [SRBase] Not using CPU: project preferences 00:00:00 [T.Brada Experiment Grid] URL https://boinc.tbrada.eu/; Computer ID 7397; resource share 100 00:00:00 [TN-Grid Platform] URL http://gene.disi.unitn.it/test/; Computer ID 56710; resource share 100 00:00:00 [Universe@Home] URL https://universeathome.pl/universe/; Computer ID 586913; resource share 100 00:00:00 [vdwnumbers.org: Van Der Waerden Numbers] URL http://www.vdwnumbers.org/; Computer ID 5464; resource share 100 00:00:00 [World Community Grid] URL http://www.worldcommunitygrid.org/; Computer ID 7701834; resource share 100 00:00:00 [WUProp@Home] URL https://wuprop.boinc-af.org/; Computer ID 146600; resource share 100 00:00:00 [yafu] URL http://yafu.myfirewall.org/yafu/; Computer ID 50078; resource share 100 00:00:00 [yoyo@home] URL http://www.rechenkraft.net/yoyo/; Computer ID 350166; resource share 100 00:00:00 [---] [work_fetch] Request work fetch: init 01:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:00:00 [---] [rr_sim] doing sim: CPU sched 01:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (1000000000.00G/439.78G) 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273869.96 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 227387.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227386.995965 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046482.96 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 204648.30 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 432035.292333 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841834.67 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 184183.47 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616218.759065 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657651.20 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 165765.12 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781983.879123 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491886.08 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 149188.61 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 931172.487176 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342697.47 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 134269.75 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065442.234423 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208427.73 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 120842.77 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186285.006946 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087584.95 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 108758.50 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1295043.502216 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978826.46 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 97882.65 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392926.147959 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880943.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 88094.38 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1481020.529128 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792849.43 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 79284.94 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560305.472180 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713564.49 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 71356.45 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631661.920927 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642208.04 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 64220.80 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695882.724800 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577987.23 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 57798.72 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753681.448285 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520188.51 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 52018.85 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805700.299421 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468169.66 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 46816.97 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852517.265444 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421352.69 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 42135.27 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894652.534864 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379217.42 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 37921.74 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932574.277343 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341295.68 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 34129.57 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966703.845573 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307166.11 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 30716.61 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997420.456981 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276449.50 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 27644.95 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025065.407248 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248804.55 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 24880.46 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049945.862488 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223924.10 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 22392.41 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072338.272204 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201531.69 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 20153.17 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092491.440949 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181378.52 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 18137.85 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110629.292819 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163240.67 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 16324.07 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126953.359502 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146916.60 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 14691.66 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141645.019516 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132224.94 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 13222.49 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154867.513530 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 119002.45 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 11900.24 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166767.758142 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107102.20 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 10710.22 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177477.978292 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96391.98 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 9639.20 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187117.176428 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86752.78 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 8675.28 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195792.454750 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78077.50 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 7807.75 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203600.205240 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70269.75 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 7026.98 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210627.180681 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63242.78 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 6324.28 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216951.458578 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56918.50 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 5691.85 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222643.308685 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51226.65 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 5122.67 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227765.973781 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46103.99 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 4610.40 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232376.372368 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41493.59 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 4149.36 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236525.731096 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37344.23 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3734.42 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240260.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33609.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243860.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30009.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247460.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26409.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251060.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22809.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254660.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19209.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258260.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15609.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261860.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12009.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265460.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8409.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269060.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4809.81 sec) 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272660.153952 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1209.81 sec) 01:00:00 [GPUGRID] [rr_sim] 2273869.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (1000000000.00G/439.78G) 01:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850510.87 01:00:00 [---] pick_jobs_to_run() start 01:00:00 [---] pick_jobs_to_run() end 01:00:00 [---] [rr_sim] end 01:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:00:00 [---] [cpu_sched_debug] preliminary job list: 01:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 01:00:00 [---] [cpu_sched_debug] final job list: 01:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 01:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:00:00 [WUProp@Home] [task] Starting task data_collect_v4_1665176701_461720_0: FLOPS left 1000.00G 01:00:00 [GPUGRID] [task] Starting task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 1000000000.00G 01:00:00 [---] [cpu_sched_debug] enforce_run_list: end 01:00:00 [---] choose_project(): 3600.000000 01:00:00 [iThena.Measurements] doing RPC to get work 01:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431999.09 cpu 2174672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174672.90 01:00:00 [---] Got 0 tasks 01:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:01:00 [---] [rr_sim] doing sim: CPU sched 01:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999972409.64G/439.78G) 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273807.22 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 227380.72 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227380.722277 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046426.50 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 204642.65 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 432023.372326 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841783.85 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 184178.39 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616201.757370 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657605.47 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 165760.55 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781962.303910 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491844.92 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 149184.49 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 931146.795795 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342660.43 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 134266.04 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065412.838492 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208394.38 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 120839.44 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186252.276920 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087554.95 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 108755.49 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1295007.771505 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978799.45 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 97879.95 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392887.716631 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880919.51 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 88091.95 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480979.667244 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792827.56 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 79282.76 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560262.422797 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713544.80 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 71354.48 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631616.902794 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642190.32 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 64219.03 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695835.934791 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577971.29 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 57797.13 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753633.063589 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520174.16 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 52017.42 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805650.479506 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468156.74 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 46815.67 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852466.153833 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421341.07 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 42134.11 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894600.260726 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379206.96 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 37920.70 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932520.956930 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341286.27 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 34128.63 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966649.583514 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307157.64 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 30715.76 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997365.347439 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276441.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 27644.19 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025009.534972 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248797.69 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 24879.77 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049889.303751 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223917.92 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 22391.79 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072281.095653 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201526.13 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 20152.61 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092433.708364 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181373.51 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 18137.35 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110571.059805 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163236.16 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 16323.62 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126894.676101 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146912.55 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 14691.25 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141585.930767 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132221.29 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 13222.13 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154808.059967 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118999.16 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 11899.92 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166707.976247 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107099.25 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 10709.92 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177417.900899 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96389.32 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 9638.93 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187056.833086 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86750.39 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 8675.04 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195731.872054 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78075.35 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 7807.54 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203539.407125 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70267.82 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 7026.78 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210566.188690 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63241.03 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 6324.10 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216890.292097 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56916.93 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 5691.69 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222581.985164 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51225.24 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 5122.52 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227704.508925 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46102.71 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 4610.27 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232314.780309 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41492.44 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 4149.24 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236464.024555 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37343.20 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3734.32 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240198.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33608.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243798.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30008.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247398.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26408.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250998.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22808.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254598.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19208.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258198.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15608.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261798.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12008.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265398.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8408.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268998.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4808.88 sec) 01:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272598.344376 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208.88 sec) 01:01:00 [GPUGRID] [rr_sim] 2273807.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999972409.64G/439.78G) 01:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850508.14 01:01:00 [---] pick_jobs_to_run() start 01:01:00 [---] pick_jobs_to_run() end 01:01:00 [---] [rr_sim] end 01:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:01:00 [---] [cpu_sched_debug] preliminary job list: 01:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 01:01:00 [---] [cpu_sched_debug] final job list: 01:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:01:00 [---] [cpu_sched_debug] enforce_run_list: end 01:01:00 [---] choose_project(): 3660.000000 01:01:00 [iThena.Measurements] doing RPC to get work 01:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431939.09 cpu 2174612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174612.90 01:01:00 [---] Got 0 tasks 01:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:02:00 [---] [rr_sim] doing sim: CPU sched 01:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999944819.29G/439.78G) 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273744.49 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 227374.45 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227374.448588 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046370.04 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 204637.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 432011.452318 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841733.03 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 184173.30 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616184.755675 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657559.73 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 165755.97 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781940.728696 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491803.76 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 149180.38 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 931121.104415 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342623.38 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 134262.34 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065383.442562 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208361.04 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 120836.10 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186219.546894 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087524.94 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 108752.49 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294972.040793 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978772.45 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 97877.24 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392849.285302 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880895.20 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 88089.52 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480938.805361 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792805.68 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 79280.57 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560219.373413 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713525.11 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 71352.51 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631571.884660 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642172.60 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 64217.26 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695789.144783 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577955.34 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 57795.53 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753584.678893 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520159.81 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 52015.98 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805600.659592 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468143.83 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 46814.38 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852415.042221 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421329.44 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 42132.94 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894547.986588 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379196.50 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 37919.65 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932467.636517 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341276.85 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 34127.68 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966595.321454 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307149.16 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 30714.92 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997310.237897 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276434.25 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 27643.42 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024953.662696 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248790.82 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 24879.08 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049832.745015 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223911.74 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 22391.17 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072223.919102 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201520.57 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 20152.06 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092375.975780 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181368.51 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 18136.85 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110512.826791 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163231.66 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 16323.17 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126835.992700 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146908.49 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 14690.85 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141526.842018 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132217.64 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 13221.76 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154748.606405 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118995.88 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 11899.59 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166648.194353 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107096.29 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 10709.63 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177357.823506 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96386.66 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 9638.67 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186996.489744 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86748.00 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 8674.80 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195671.289358 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78073.20 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 7807.32 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203478.609011 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70265.88 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 7026.59 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210505.196698 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63239.29 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 6323.93 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216829.125617 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56915.36 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 5691.54 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222520.661644 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51223.82 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 5122.38 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227643.044068 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46101.44 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 4610.14 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232253.188250 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41491.30 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 4149.13 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236402.318013 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37342.17 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3734.22 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240136.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33607.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243736.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30007.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247336.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26407.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250936.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22807.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254536.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19207.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258136.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15607.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261736.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12007.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265336.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8407.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268936.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4807.95 sec) 01:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272536.534800 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207.95 sec) 01:02:00 [GPUGRID] [rr_sim] 2273744.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999944819.29G/439.78G) 01:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850505.40 01:02:00 [---] pick_jobs_to_run() start 01:02:00 [---] pick_jobs_to_run() end 01:02:00 [---] [rr_sim] end 01:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:02:00 [---] [cpu_sched_debug] preliminary job list: 01:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:02:00 [---] [cpu_sched_debug] final job list: 01:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:02:00 [---] [cpu_sched_debug] enforce_run_list: end 01:02:00 [---] choose_project(): 3720.000000 01:02:00 [iThena.Measurements] doing RPC to get work 01:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431879.09 cpu 2174552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174552.90 01:02:00 [---] Got 0 tasks 01:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:03:00 [---] [rr_sim] doing sim: CPU sched 01:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999917228.93G/439.78G) 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273681.75 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 227368.17 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227368.174900 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046313.57 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 204631.36 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431999.532311 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841682.22 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 184168.22 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616167.753980 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657514.00 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 165751.40 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781919.153482 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491762.60 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 149176.26 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 931095.413034 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342586.34 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 134258.63 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065354.046631 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208327.70 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 120832.77 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186186.816868 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087494.93 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 108749.49 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294936.310082 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978745.44 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 97874.54 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392810.853974 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880870.90 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 88087.09 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480897.943477 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792783.81 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 79278.38 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560176.324029 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713505.42 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 71350.54 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631526.866527 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642154.88 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 64215.49 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695742.354774 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577939.39 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 57793.94 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753536.294197 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520145.45 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 52014.55 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805550.839678 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468130.91 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 46813.09 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852363.930610 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421317.82 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 42131.78 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894495.712449 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379186.04 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 37918.60 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932414.316105 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341267.43 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 34126.74 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966541.059394 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307140.69 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 30714.07 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997255.128355 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276426.62 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 27642.66 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024897.790420 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248783.96 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 24878.40 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049776.186278 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223905.56 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 22390.56 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072166.742551 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201515.01 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 20151.50 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092318.243196 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181363.51 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 18136.35 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110454.593777 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163227.16 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 16322.72 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126777.309299 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146904.44 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 14690.44 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141467.753270 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132214.00 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 13221.40 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154689.152843 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118992.60 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 11899.26 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166588.412459 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107093.34 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 10709.33 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177297.746113 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96384.00 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 9638.40 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186936.146402 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86745.60 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 8674.56 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195610.706662 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78071.04 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 7807.10 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203417.810896 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70263.94 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 7026.39 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210444.204707 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63237.54 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 6323.75 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216767.959136 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56913.79 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 5691.38 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222459.338123 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51222.41 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 5122.24 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227581.579211 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46100.17 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 4610.02 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232191.596190 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41490.15 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 4149.02 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236340.611471 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37341.14 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3734.11 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240074.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33607.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243674.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30007.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247274.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26407.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250874.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22807.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254474.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19207.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258074.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15607.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261674.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12007.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265274.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8407.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268874.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4807.02 sec) 01:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272474.725225 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207.02 sec) 01:03:00 [GPUGRID] [rr_sim] 2273681.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999917228.93G/439.78G) 01:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850502.66 01:03:00 [---] pick_jobs_to_run() start 01:03:00 [---] pick_jobs_to_run() end 01:03:00 [---] [rr_sim] end 01:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:03:00 [---] [cpu_sched_debug] preliminary job list: 01:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:03:00 [---] [cpu_sched_debug] final job list: 01:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:03:00 [---] [cpu_sched_debug] enforce_run_list: end 01:03:00 [---] choose_project(): 3780.000000 01:03:00 [iThena.Measurements] doing RPC to get work 01:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431819.09 cpu 2174492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174492.90 01:03:00 [---] Got 0 tasks 01:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:04:00 [---] [rr_sim] doing sim: CPU sched 01:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999889638.58G/439.78G) 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273619.01 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 227361.90 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227361.901212 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046257.11 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 204625.71 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431987.612303 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841631.40 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 184163.14 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616150.752285 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657468.26 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 165746.83 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781897.578268 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491721.43 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 149172.14 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 931069.721653 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342549.29 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 134254.93 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065324.650700 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208294.36 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 120829.44 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186154.086842 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087464.93 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 108746.49 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294900.579370 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978718.43 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 97871.84 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392772.422645 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880846.59 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 88084.66 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480857.081593 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792761.93 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 79276.19 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560133.274645 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713485.74 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 71348.57 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631481.848393 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642137.16 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 64213.72 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695695.564766 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577923.45 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 57792.34 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753487.909501 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520131.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 52013.11 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805501.019763 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468117.99 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 46811.80 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852312.818999 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421306.19 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 42130.62 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894443.438311 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379175.57 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 37917.56 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932360.995692 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341258.02 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 34125.80 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966486.797335 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307132.21 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 30713.22 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997200.018813 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276418.99 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 27641.90 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024841.918144 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248777.09 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 24877.71 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049719.627542 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223899.38 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 22389.94 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072109.566000 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201509.45 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 20150.94 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092260.510612 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181358.50 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 18135.85 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110396.360763 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163222.65 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 16322.27 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126718.625898 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146900.39 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 14690.04 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141408.664521 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132210.35 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 13221.03 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154629.699281 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118989.31 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 11898.93 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166528.630565 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107090.38 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 10709.04 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177237.668720 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96381.34 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 9638.13 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186875.803060 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86743.21 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 8674.32 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195550.123966 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78068.89 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 7806.89 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203357.012782 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70262.00 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 7026.20 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210383.212716 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63235.80 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 6323.58 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216706.792656 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56912.22 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 5691.22 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.014602 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51221.00 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 5122.10 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227520.114354 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46098.90 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 4609.89 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232130.004131 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41489.01 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 4148.90 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236278.904930 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37340.11 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3734.01 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240012.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33606.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243612.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30006.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247212.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26406.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250812.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22806.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254412.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19206.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258012.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15606.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261612.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12006.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265212.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8406.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268812.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4806.10 sec) 01:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272412.915649 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206.10 sec) 01:04:00 [GPUGRID] [rr_sim] 2273619.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999889638.58G/439.78G) 01:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850499.92 01:04:00 [---] pick_jobs_to_run() start 01:04:00 [---] pick_jobs_to_run() end 01:04:00 [---] [rr_sim] end 01:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:04:00 [---] [cpu_sched_debug] preliminary job list: 01:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:04:00 [---] [cpu_sched_debug] final job list: 01:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:04:00 [---] [cpu_sched_debug] enforce_run_list: end 01:04:00 [---] choose_project(): 3840.000000 01:04:00 [iThena.Measurements] doing RPC to get work 01:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431759.09 cpu 2174432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174432.90 01:04:00 [---] Got 0 tasks 01:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:05:00 [---] [rr_sim] doing sim: CPU sched 01:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999862048.22G/439.78G) 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273556.28 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 227355.63 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227355.627524 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046200.65 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 204620.06 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431975.692295 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841580.58 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 184158.06 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616133.750590 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657422.52 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 165742.25 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781876.003054 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491680.27 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 149168.03 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 931044.030273 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342512.24 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 134251.22 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065295.254769 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208261.02 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 120826.10 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186121.356816 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087434.92 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 108743.49 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294864.848658 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978691.43 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 97869.14 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392733.991316 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880822.28 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 88082.23 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480816.219709 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792740.06 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 79274.01 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560090.225262 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713466.05 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 71346.60 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631436.830259 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642119.44 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 64211.94 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695648.774757 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577907.50 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 57790.75 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753439.524805 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520116.75 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 52011.68 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805451.199849 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468105.08 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 46810.51 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852261.707388 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421294.57 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 42129.46 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894391.164173 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379165.11 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 37916.51 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932307.675279 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341248.60 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 34124.86 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966432.535275 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307123.74 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 30712.37 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997144.909271 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276411.37 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 27641.14 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024786.045868 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248770.23 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 24877.02 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049663.068805 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223893.21 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 22389.32 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072052.389448 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201503.89 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 20150.39 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092202.778027 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181353.50 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 18135.35 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110338.127749 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163218.15 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 16321.81 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126659.942498 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146896.33 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 14689.63 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141349.575772 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132206.70 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 13220.67 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154570.245718 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118986.03 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 11898.60 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166468.848670 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107087.43 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 10708.74 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177177.591327 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96378.68 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 9637.87 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186815.459718 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86740.82 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 8674.08 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195489.541270 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78066.73 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 7806.67 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203296.214667 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70260.06 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 7026.01 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210322.220724 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63234.05 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 6323.41 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216645.626176 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56910.65 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 5691.06 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222336.691082 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51219.58 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 5121.96 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227458.649497 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46097.63 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 4609.76 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232068.412072 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41487.86 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 4148.79 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236217.198388 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37339.08 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3733.91 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239951.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33605.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243551.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30005.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247151.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26405.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250751.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22805.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254351.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19205.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257951.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15605.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261551.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12005.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265151.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8405.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268751.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4805.17 sec) 01:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272351.106073 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205.17 sec) 01:05:00 [GPUGRID] [rr_sim] 2273556.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999862048.22G/439.78G) 01:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850497.19 01:05:00 [---] pick_jobs_to_run() start 01:05:00 [---] pick_jobs_to_run() end 01:05:00 [---] [rr_sim] end 01:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:05:00 [---] [cpu_sched_debug] preliminary job list: 01:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:05:00 [---] [cpu_sched_debug] final job list: 01:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:05:00 [---] [cpu_sched_debug] enforce_run_list: end 01:05:00 [---] choose_project(): 3900.000000 01:05:00 [iThena.Measurements] doing RPC to get work 01:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431699.09 cpu 2174372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174372.90 01:05:00 [---] Got 0 tasks 01:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:06:00 [---] [rr_sim] doing sim: CPU sched 01:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999834457.86G/439.78G) 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273493.54 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 227349.35 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227349.353836 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046144.18 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 204614.42 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431963.772288 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841529.77 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 184152.98 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616116.748895 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657376.79 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 165737.68 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781854.427841 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491639.11 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 149163.91 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 931018.338892 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342475.20 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 134247.52 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065265.858839 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208227.68 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 120822.77 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186088.626790 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087404.91 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 108740.49 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294829.117947 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978664.42 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 97866.44 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392695.559988 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880797.98 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 88079.80 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480775.357825 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792718.18 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 79271.82 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560047.175878 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713446.36 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 71344.64 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631391.812126 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642101.73 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 64210.17 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695601.984749 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577891.55 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 57789.16 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753391.140109 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520102.40 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 52010.24 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805401.379934 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468092.16 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 46809.22 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852210.595776 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421282.94 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 42128.29 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894338.890034 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379154.65 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 37915.46 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932254.354867 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341239.18 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 34123.92 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966378.273215 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307115.27 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 30711.53 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997089.799730 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276403.74 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 27640.37 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024730.173592 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248763.36 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 24876.34 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049606.510069 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223887.03 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 22388.70 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071995.212897 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201498.33 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 20149.83 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092145.045443 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181348.49 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 18134.85 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110279.894735 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163213.64 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 16321.36 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126601.259097 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146892.28 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 14689.23 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141290.487023 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132203.05 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 13220.31 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154510.792156 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118982.75 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 11898.27 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166409.066776 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107084.47 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 10708.45 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177117.513934 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96376.02 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 9637.60 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186755.116376 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86738.42 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 8673.84 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195428.958574 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78064.58 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 7806.46 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203235.416552 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70258.12 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 7025.81 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210261.228733 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63232.31 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 6323.23 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216584.459695 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56909.08 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 5690.91 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222275.367561 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51218.17 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 5121.82 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227397.184641 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46096.35 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 4609.64 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232006.820012 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41486.72 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 4148.67 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236155.491847 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37338.05 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3733.80 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239889.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33604.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243489.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30004.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247089.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26404.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250689.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22804.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254289.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19204.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257889.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15604.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261489.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12004.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265089.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8404.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268689.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4804.24 sec) 01:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272289.296498 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204.24 sec) 01:06:00 [GPUGRID] [rr_sim] 2273493.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999834457.86G/439.78G) 01:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850494.45 01:06:00 [---] pick_jobs_to_run() start 01:06:00 [---] pick_jobs_to_run() end 01:06:00 [---] [rr_sim] end 01:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:06:00 [---] [cpu_sched_debug] preliminary job list: 01:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:06:00 [---] [cpu_sched_debug] final job list: 01:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:06:00 [---] [cpu_sched_debug] enforce_run_list: end 01:06:00 [---] choose_project(): 3960.000000 01:06:00 [iThena.Measurements] doing RPC to get work 01:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431639.09 cpu 2174312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174312.90 01:06:00 [---] Got 0 tasks 01:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:07:00 [---] [rr_sim] doing sim: CPU sched 01:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999806867.51G/439.78G) 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273430.80 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 227343.08 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227343.080147 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046087.72 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 204608.77 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431951.852280 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841478.95 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 184147.89 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616099.747199 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657331.05 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 165733.11 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781832.852627 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491597.95 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 149159.79 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930992.647512 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342438.15 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 134243.82 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065236.462908 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208194.34 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 120819.43 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186055.896764 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087374.90 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 108737.49 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294793.387235 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978637.41 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 97863.74 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392657.128659 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880773.67 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 88077.37 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480734.495941 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792696.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 79269.63 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1560004.126494 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713426.67 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 71342.67 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631346.793992 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642084.01 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 64208.40 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695555.194740 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577875.61 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 57787.56 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753342.755414 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520088.05 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 52008.80 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805351.560020 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468079.24 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 46807.92 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852159.484165 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421271.32 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 42127.13 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894286.615896 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379144.19 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 37914.42 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932201.034454 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341229.77 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 34122.98 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966324.011156 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307106.79 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 30710.68 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997034.690188 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276396.11 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 27639.61 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024674.301316 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248756.50 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 24875.65 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049549.951332 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223880.85 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 22388.09 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071938.036346 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201492.77 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 20149.28 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092087.312859 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181343.49 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 18134.35 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110221.661721 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163209.14 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 16320.91 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126542.575696 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146888.23 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 14688.82 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141231.398274 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132199.40 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 13219.94 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154451.338594 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118979.46 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 11897.95 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166349.284882 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107081.52 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 10708.15 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177057.436541 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96373.36 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 9637.34 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186694.773034 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86736.03 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 8673.60 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195368.375878 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78062.43 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 7806.24 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203174.618438 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70256.18 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 7025.62 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210200.236741 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63230.56 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 6323.06 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216523.293215 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56907.51 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 5690.75 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222214.044041 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51216.76 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 5121.68 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227335.719784 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46095.08 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 4609.51 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231945.227953 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41485.57 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 4148.56 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236093.785305 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37337.02 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3733.70 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239827.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33603.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243427.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30003.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247027.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26403.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250627.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22803.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254227.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19203.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257827.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15603.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261427.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12003.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265027.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8403.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268627.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4803.31 sec) 01:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272227.486922 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203.31 sec) 01:07:00 [GPUGRID] [rr_sim] 2273430.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999806867.51G/439.78G) 01:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850491.71 01:07:00 [---] pick_jobs_to_run() start 01:07:00 [---] pick_jobs_to_run() end 01:07:00 [---] [rr_sim] end 01:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:07:00 [---] [cpu_sched_debug] preliminary job list: 01:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:07:00 [---] [cpu_sched_debug] final job list: 01:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:07:00 [---] [cpu_sched_debug] enforce_run_list: end 01:07:00 [---] choose_project(): 4020.000000 01:07:00 [iThena.Measurements] doing RPC to get work 01:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431579.09 cpu 2174252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174252.90 01:07:00 [---] Got 0 tasks 01:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:08:00 [---] [rr_sim] doing sim: CPU sched 01:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999779277.15G/439.78G) 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273368.06 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 227336.81 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227336.806459 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2046031.26 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 204603.13 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431939.932272 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841428.13 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 184142.81 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616082.745504 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657285.32 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 165728.53 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781811.277413 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491556.79 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 149155.68 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930966.956131 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342401.11 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 134240.11 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065207.066977 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208161.00 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 120816.10 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1186023.166739 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087344.90 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 108734.49 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294757.656524 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978610.41 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 97861.04 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392618.697331 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880749.37 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 88074.94 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480693.634057 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792674.43 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 79267.44 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559961.077110 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713406.99 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 71340.70 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631301.775858 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642066.29 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 64206.63 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695508.404732 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577859.66 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 57785.97 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753294.370718 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520073.69 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 52007.37 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805301.740105 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468066.32 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 46806.63 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852108.372554 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421259.69 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 42125.97 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894234.341758 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379133.72 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 37913.37 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932147.714041 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341220.35 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 34122.04 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966269.749096 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307098.32 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 30709.83 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996979.580646 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276388.48 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 27638.85 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024618.429040 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248749.64 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 24874.96 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049493.392595 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223874.67 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 22387.47 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071880.859795 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201487.20 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 20148.72 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092029.580275 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181338.48 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 18133.85 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110163.428706 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163204.64 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 16320.46 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126483.892295 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146884.17 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 14688.42 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141172.309525 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132195.76 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 13219.58 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154391.885031 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118976.18 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 11897.62 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166289.502987 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107078.56 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 10707.86 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176997.359148 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96370.71 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 9637.07 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186634.429692 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86733.63 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 8673.36 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195307.793182 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78060.27 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 7806.03 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203113.820323 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70254.24 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 7025.42 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210139.244750 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63228.82 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 6322.88 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216462.126734 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56905.94 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 5690.59 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222152.720520 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51215.34 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 5121.53 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227274.254927 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46093.81 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 4609.38 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231883.635894 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41484.43 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 4148.44 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236032.078763 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37335.99 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3733.60 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239765.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33602.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243365.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30002.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246965.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26402.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250565.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22802.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254165.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19202.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257765.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15602.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261365.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12002.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264965.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8402.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268565.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4802.39 sec) 01:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272165.677346 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202.39 sec) 01:08:00 [GPUGRID] [rr_sim] 2273368.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999779277.15G/439.78G) 01:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850488.98 01:08:00 [---] pick_jobs_to_run() start 01:08:00 [---] pick_jobs_to_run() end 01:08:00 [---] [rr_sim] end 01:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:08:00 [---] [cpu_sched_debug] preliminary job list: 01:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:08:00 [---] [cpu_sched_debug] final job list: 01:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:08:00 [---] [cpu_sched_debug] enforce_run_list: end 01:08:00 [---] choose_project(): 4080.000000 01:08:00 [iThena.Measurements] doing RPC to get work 01:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431519.09 cpu 2174192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174192.90 01:08:00 [---] Got 0 tasks 01:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:09:00 [---] [rr_sim] doing sim: CPU sched 01:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999751686.79G/439.78G) 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273305.33 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 227330.53 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227330.532771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045974.79 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 204597.48 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431928.012265 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841377.32 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 184137.73 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616065.743809 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657239.58 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 165723.96 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781789.702199 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491515.63 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 149151.56 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930941.264750 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342364.06 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 134236.41 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065177.671046 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208127.66 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 120812.77 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185990.436713 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087314.89 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 108731.49 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294721.925812 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978583.40 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 97858.34 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392580.266002 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880725.06 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 88072.51 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480652.772173 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792652.56 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 79265.26 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559918.027727 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713387.30 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 71338.73 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631256.757725 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642048.57 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 64204.86 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695461.614723 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577843.71 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 57784.37 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753245.986022 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520059.34 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 52005.93 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805251.920191 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468053.41 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 46805.34 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852057.260943 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421248.07 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 42124.81 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894182.067619 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379123.26 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 37912.33 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932094.393628 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341210.93 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 34121.09 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966215.487036 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307089.84 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 30708.98 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996924.471104 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276380.86 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 27638.09 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024562.556764 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248742.77 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 24874.28 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049436.833859 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223868.49 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 22386.85 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071823.683244 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201481.64 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 20148.16 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091971.847691 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181333.48 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 18133.35 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110105.195692 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163200.13 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 16320.01 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126425.208894 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146880.12 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 14688.01 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141113.220776 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132192.11 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 13219.21 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154332.431469 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118972.90 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 11897.29 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166229.721093 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107075.61 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 10707.56 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176937.281755 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96368.05 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 9636.80 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186574.086350 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86731.24 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 8673.12 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195247.210486 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78058.12 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 7805.81 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203053.022209 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70252.31 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 7025.23 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210078.252759 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63227.07 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 6322.71 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216400.960254 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56904.37 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 5690.44 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222091.396999 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51213.93 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 5121.39 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227212.790070 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46092.54 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 4609.25 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231822.043834 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41483.28 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 4148.33 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235970.372222 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37334.96 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3733.50 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239703.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33601.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243303.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30001.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246903.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26401.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250503.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22801.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254103.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19201.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257703.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15601.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261303.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12001.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264903.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8401.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268503.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4801.46 sec) 01:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272103.867771 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201.46 sec) 01:09:00 [GPUGRID] [rr_sim] 2273305.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999751686.79G/439.78G) 01:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850486.24 01:09:00 [---] pick_jobs_to_run() start 01:09:00 [---] pick_jobs_to_run() end 01:09:00 [---] [rr_sim] end 01:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:09:00 [---] [cpu_sched_debug] preliminary job list: 01:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:09:00 [---] [cpu_sched_debug] final job list: 01:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:09:00 [---] [cpu_sched_debug] enforce_run_list: end 01:09:00 [---] choose_project(): 4140.000000 01:09:00 [iThena.Measurements] doing RPC to get work 01:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431459.09 cpu 2174132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174132.90 01:09:00 [---] Got 0 tasks 01:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:10:00 [---] [rr_sim] doing sim: CPU sched 01:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999724096.44G/439.78G) 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273242.59 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 227324.26 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227324.259083 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045918.33 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 204591.83 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431916.092257 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841326.50 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 184132.65 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616048.742114 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657193.85 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 165719.38 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781768.126986 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491474.46 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 149147.45 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930915.573370 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342327.02 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 134232.70 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065148.275116 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208094.32 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 120809.43 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185957.706687 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087284.88 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 108728.49 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294686.195101 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978556.40 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 97855.64 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392541.834673 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880700.76 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 88070.08 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480611.910289 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792630.68 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 79263.07 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559874.978343 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713367.61 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 71336.76 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631211.739591 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642030.85 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 64203.09 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695414.824715 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577827.77 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 57782.78 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753197.601326 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520044.99 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 52004.50 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805202.100276 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468040.49 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 46804.05 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852006.149331 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421236.44 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 42123.64 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894129.793481 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379112.80 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 37911.28 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932041.073216 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341201.52 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 34120.15 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966161.224977 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307081.37 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 30708.14 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996869.361562 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276373.23 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 27637.32 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024506.684488 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248735.91 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 24873.59 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049380.275122 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223862.32 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 22386.23 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071766.506693 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201476.08 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 20147.61 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091914.115106 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181328.48 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 18132.85 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110046.962678 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163195.63 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 16319.56 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126366.525493 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146876.07 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 14687.61 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141054.132027 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132188.46 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 13218.85 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154272.977907 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118969.61 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 11896.96 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166169.939199 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107072.65 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 10707.27 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176877.204362 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96365.39 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 9636.54 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186513.743008 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86728.85 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 8672.88 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195186.627790 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78055.96 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 7805.60 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202992.224094 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70250.37 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 7025.04 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210017.260767 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63225.33 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 6322.53 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216339.793773 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56902.80 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 5690.28 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222030.073479 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51212.52 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 5121.25 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227151.325214 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46091.27 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 4609.13 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231760.451775 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41482.14 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 4148.21 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235908.665680 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37333.93 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3733.39 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239642.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33600.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243242.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30000.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246842.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26400.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250442.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22800.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254042.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19200.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257642.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15600.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261242.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12000.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264842.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8400.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268442.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4800.53 sec) 01:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2272042.058195 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200.53 sec) 01:10:00 [GPUGRID] [rr_sim] 2273242.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999724096.44G/439.78G) 01:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850483.50 01:10:00 [---] pick_jobs_to_run() start 01:10:00 [---] pick_jobs_to_run() end 01:10:00 [---] [rr_sim] end 01:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:10:00 [---] [cpu_sched_debug] preliminary job list: 01:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:10:00 [---] [cpu_sched_debug] final job list: 01:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:10:00 [---] [cpu_sched_debug] enforce_run_list: end 01:10:00 [---] choose_project(): 4200.000000 01:10:00 [iThena.Measurements] doing RPC to get work 01:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431399.09 cpu 2174072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174072.90 01:10:00 [---] Got 0 tasks 01:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:11:00 [---] [rr_sim] doing sim: CPU sched 01:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999696506.08G/439.78G) 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273179.85 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 227317.99 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227317.985395 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045861.87 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 204586.19 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431904.172250 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841275.68 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 184127.57 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616031.740419 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657148.11 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 165714.81 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781746.551772 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491433.30 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 149143.33 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930889.881989 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342289.97 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 134229.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065118.879185 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208060.97 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 120806.10 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185924.976661 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087254.88 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 108725.49 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294650.464389 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978529.39 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 97852.94 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392503.403345 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880676.45 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 88067.65 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480571.048405 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792608.81 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 79260.88 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559831.928959 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713347.92 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 71334.79 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631166.721458 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642013.13 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 64201.31 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695368.034706 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577811.82 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 57781.18 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753149.216630 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520030.64 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 52003.06 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805152.280362 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468027.57 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 46802.76 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851955.037720 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421224.82 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 42122.48 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894077.519343 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379102.33 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 37910.23 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931987.752803 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341192.10 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 34119.21 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966106.962917 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307072.89 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 30707.29 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996814.252020 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276365.60 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 27636.56 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024450.812213 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248729.04 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 24872.90 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049323.716386 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223856.14 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 22385.61 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071709.330142 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201470.52 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 20147.05 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091856.382522 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181323.47 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 18132.35 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109988.729664 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163191.12 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 16319.11 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126307.842093 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146872.01 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 14687.20 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140995.043278 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132184.81 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 13218.48 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154213.524345 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118966.33 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 11896.63 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166110.157305 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107069.70 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 10706.97 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176817.126969 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96362.73 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 9636.27 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186453.399666 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86726.45 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 8672.65 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195126.045094 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78053.81 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 7805.38 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202931.425979 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70248.43 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 7024.84 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209956.268776 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63223.59 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 6322.36 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216278.627293 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56901.23 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 5690.12 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221968.749958 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51211.10 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 5121.11 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227089.860357 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46089.99 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 4609.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231698.859716 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41480.99 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 4148.10 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235846.959139 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37332.89 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3733.29 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239580.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33599.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243180.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29999.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246780.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26399.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250380.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22799.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253980.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19199.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257580.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15599.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261180.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11999.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264780.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8399.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268380.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4799.61 sec) 01:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271980.248619 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199.61 sec) 01:11:00 [GPUGRID] [rr_sim] 2273179.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999696506.08G/439.78G) 01:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850480.77 01:11:00 [---] pick_jobs_to_run() start 01:11:00 [---] pick_jobs_to_run() end 01:11:00 [---] [rr_sim] end 01:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:11:00 [---] [cpu_sched_debug] preliminary job list: 01:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:11:00 [---] [cpu_sched_debug] final job list: 01:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:11:00 [---] [cpu_sched_debug] enforce_run_list: end 01:11:00 [---] choose_project(): 4260.000000 01:11:00 [iThena.Measurements] doing RPC to get work 01:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431339.09 cpu 2174012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2174012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2174012.90 01:11:00 [---] Got 0 tasks 01:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:12:00 [---] [rr_sim] doing sim: CPU sched 01:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999668915.73G/439.78G) 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273117.12 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 227311.71 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227311.711706 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045805.41 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 204580.54 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431892.252242 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841224.86 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 184122.49 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616014.738724 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657102.38 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 165710.24 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781724.976558 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491392.14 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 149139.21 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930864.190609 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342252.93 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 134225.29 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065089.483254 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208027.63 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 120802.76 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185892.246635 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087224.87 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 108722.49 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294614.733678 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978502.38 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 97850.24 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392464.972016 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880652.15 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 88065.21 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480530.186521 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792586.93 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 79258.69 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559788.879575 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713328.24 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 71332.82 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631121.703324 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641995.41 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 64199.54 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695321.244698 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577795.87 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 57779.59 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753100.831934 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520016.29 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 52001.63 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805102.460447 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468014.66 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 46801.47 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851903.926109 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421213.19 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 42121.32 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894025.245204 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379091.87 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 37909.19 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931934.432390 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341182.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 34118.27 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966052.700857 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307064.42 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 30706.44 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996759.142478 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276357.97 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 27635.80 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024394.939937 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248722.18 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 24872.22 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049267.157649 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223849.96 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 22385.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071652.153591 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201464.96 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 20146.50 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091798.649938 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181318.47 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 18131.85 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109930.496650 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163186.62 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 16318.66 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126249.158692 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146867.96 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 14686.80 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140935.954529 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132181.16 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 13218.12 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154154.070782 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118963.05 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 11896.30 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166050.375410 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107066.74 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 10706.67 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176757.049576 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96360.07 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 9636.01 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186393.056324 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86724.06 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 8672.41 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195065.462398 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78051.65 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 7805.17 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202870.627865 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70246.49 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 7024.65 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209895.276785 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63221.84 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 6322.18 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216217.460812 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56899.66 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 5689.97 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221907.426438 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51209.69 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 5120.97 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227028.395500 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46088.72 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 4608.87 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231637.267656 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41479.85 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 4147.98 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235785.252597 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37331.86 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3733.19 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239518.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33598.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243118.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29998.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246718.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26398.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250318.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22798.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253918.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19198.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257518.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15598.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261118.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11998.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264718.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8398.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268318.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4798.68 sec) 01:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271918.439044 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198.68 sec) 01:12:00 [GPUGRID] [rr_sim] 2273117.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999668915.73G/439.78G) 01:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850478.03 01:12:00 [---] pick_jobs_to_run() start 01:12:00 [---] pick_jobs_to_run() end 01:12:00 [---] [rr_sim] end 01:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:12:00 [---] [cpu_sched_debug] preliminary job list: 01:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:12:00 [---] [cpu_sched_debug] final job list: 01:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:12:00 [---] [cpu_sched_debug] enforce_run_list: end 01:12:00 [---] choose_project(): 4320.000000 01:12:00 [iThena.Measurements] doing RPC to get work 01:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431279.09 cpu 2173952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173952.90 01:12:00 [---] Got 0 tasks 01:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:13:00 [---] [rr_sim] doing sim: CPU sched 01:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999668915.73G/439.78G) 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273117.12 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 227311.71 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227311.711706 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045805.41 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 204580.54 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431892.252242 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841224.86 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 184122.49 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 616014.738724 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657102.38 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 165710.24 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781724.976558 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491392.14 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 149139.21 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930864.190609 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342252.93 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 134225.29 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065089.483254 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1208027.63 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 120802.76 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185892.246635 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087224.87 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 108722.49 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294614.733678 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978502.38 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 97850.24 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392464.972016 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880652.15 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 88065.21 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480530.186521 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792586.93 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 79258.69 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559788.879575 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713328.24 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 71332.82 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631121.703324 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641995.41 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 64199.54 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695321.244698 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577795.87 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 57779.59 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753100.831934 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520016.29 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 52001.63 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805102.460447 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468014.66 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 46801.47 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851903.926109 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421213.19 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 42121.32 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894025.245204 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379091.87 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 37909.19 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931934.432390 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341182.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 34118.27 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966052.700857 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307064.42 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 30706.44 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996759.142478 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276357.97 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 27635.80 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024394.939937 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248722.18 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 24872.22 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049267.157649 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223849.96 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 22385.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071652.153591 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201464.96 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 20146.50 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091798.649938 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181318.47 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 18131.85 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109930.496650 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163186.62 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 16318.66 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126249.158692 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146867.96 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 14686.80 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140935.954529 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132181.16 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 13218.12 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154154.070782 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118963.05 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 11896.30 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166050.375410 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107066.74 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 10706.67 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176757.049576 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96360.07 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 9636.01 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186393.056324 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86724.06 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 8672.41 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195065.462398 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78051.65 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 7805.17 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202870.627865 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70246.49 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 7024.65 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209895.276785 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63221.84 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 6322.18 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216217.460812 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56899.66 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 5689.97 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221907.426438 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51209.69 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 5120.97 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227028.395500 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46088.72 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 4608.87 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231637.267656 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41479.85 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 4147.98 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235785.252597 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37331.86 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3733.19 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239518.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33598.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243118.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29998.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246718.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26398.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250318.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22798.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253918.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19198.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257518.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15598.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261118.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11998.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264718.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8398.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268318.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4798.68 sec) 01:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271918.439044 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198.68 sec) 01:13:00 [GPUGRID] [rr_sim] 2273117.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999668915.73G/439.78G) 01:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850538.03 01:13:00 [---] pick_jobs_to_run() start 01:13:00 [---] pick_jobs_to_run() end 01:13:00 [---] [rr_sim] end 01:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:13:00 [---] [cpu_sched_debug] preliminary job list: 01:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:13:00 [---] [cpu_sched_debug] final job list: 01:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:13:00 [---] [cpu_sched_debug] enforce_run_list: end 01:13:00 [---] choose_project(): 4380.000000 01:13:00 [iThena.Measurements] doing RPC to get work 01:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 604019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431219.09 cpu 2173952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:13:00 [---] Got 0 tasks 01:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:14:00 [---] [rr_sim] doing sim: CPU sched 01:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999641325.37G/439.78G) 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273054.38 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 227305.44 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227305.438018 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045748.94 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 204574.89 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431880.332234 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841174.05 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 184117.40 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615997.737029 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657056.64 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 165705.66 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781703.401344 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491350.98 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 149135.10 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930838.499228 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342215.88 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 134221.59 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065060.087323 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207994.29 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 120799.43 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185859.516609 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087194.86 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 108719.49 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294579.002966 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978475.38 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 97847.54 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392426.540688 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880627.84 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 88062.78 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480489.324637 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792565.06 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 79256.51 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559745.830191 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713308.55 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 71330.85 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631076.685190 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641977.69 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 64197.77 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695274.454689 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577779.93 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 57777.99 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753052.447239 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520001.93 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 52000.19 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805052.640533 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468001.74 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 46800.17 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851852.814498 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421201.57 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 42120.16 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893972.971066 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379081.41 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 37908.14 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931881.111977 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341173.27 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 34117.33 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965998.438798 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307055.94 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 30705.59 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996704.032936 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276350.35 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 27635.03 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024339.067661 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248715.31 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 24871.53 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049210.598913 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223843.78 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 22384.38 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071594.977040 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201459.40 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 20145.94 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091740.917354 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181313.46 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 18131.35 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109872.263636 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163182.12 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 16318.21 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126190.475291 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146863.90 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 14686.39 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140876.865780 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132177.51 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 13217.75 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154094.617220 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118959.76 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 11895.98 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165990.593516 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107063.79 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 10706.38 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176696.972183 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96357.41 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 9635.74 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186332.712982 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86721.67 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 8672.17 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195004.879702 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78049.50 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 7804.95 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202809.829750 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70244.55 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 7024.46 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209834.284793 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63220.10 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 6322.01 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216156.294332 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56898.09 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 5689.81 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221846.102917 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51208.28 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 5120.83 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226966.930643 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46087.45 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 4608.74 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231575.675597 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41478.70 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 4147.87 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235723.546055 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37330.83 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3733.08 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239456.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33597.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243056.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29997.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246656.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26397.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250256.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22797.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253856.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19197.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257456.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15597.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261056.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11997.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264656.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8397.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268256.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4797.75 sec) 01:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271856.629468 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197.75 sec) 01:14:00 [GPUGRID] [rr_sim] 2273054.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999641325.37G/439.78G) 01:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850535.29 01:14:00 [---] pick_jobs_to_run() start 01:14:00 [---] pick_jobs_to_run() end 01:14:00 [---] [rr_sim] end 01:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:14:00 [---] [cpu_sched_debug] preliminary job list: 01:14:00 [---] [cpu_sched_debug] final job list: 01:14:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 01:14:00 [---] [work_fetch] Request work fetch: CPUs idle 01:14:00 [---] [cpu_sched_debug] enforce_run_list: end 01:14:00 [---] choose_project(): 4440.000000 01:14:00 [iThena.Measurements] doing RPC to get work 01:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431159.09 cpu 2173892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:14:00 [---] Got 0 tasks 01:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:15:00 [---] [rr_sim] doing sim: CPU sched 01:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999641325.37G/439.78G) 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2273054.38 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 227305.44 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227305.438018 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045748.94 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 204574.89 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431880.332234 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841174.05 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 184117.40 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615997.737029 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657056.64 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 165705.66 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781703.401344 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491350.98 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 149135.10 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930838.499228 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342215.88 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 134221.59 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065060.087323 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207994.29 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 120799.43 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185859.516609 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087194.86 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 108719.49 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294579.002966 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978475.38 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 97847.54 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392426.540688 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880627.84 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 88062.78 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480489.324637 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792565.06 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 79256.51 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559745.830191 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713308.55 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 71330.85 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631076.685190 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641977.69 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 64197.77 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695274.454689 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577779.93 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 57777.99 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753052.447239 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520001.93 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 52000.19 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805052.640533 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468001.74 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 46800.17 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851852.814498 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421201.57 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 42120.16 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893972.971066 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379081.41 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 37908.14 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931881.111977 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341173.27 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 34117.33 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965998.438798 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307055.94 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 30705.59 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996704.032936 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276350.35 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 27635.03 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024339.067661 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248715.31 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 24871.53 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049210.598913 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223843.78 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 22384.38 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071594.977040 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201459.40 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 20145.94 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091740.917354 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181313.46 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 18131.35 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109872.263636 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163182.12 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 16318.21 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126190.475291 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146863.90 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 14686.39 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140876.865780 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132177.51 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 13217.75 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154094.617220 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118959.76 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 11895.98 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165990.593516 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107063.79 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 10706.38 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176696.972183 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96357.41 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 9635.74 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186332.712982 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86721.67 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 8672.17 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195004.879702 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78049.50 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 7804.95 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202809.829750 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70244.55 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 7024.46 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209834.284793 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63220.10 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 6322.01 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216156.294332 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56898.09 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 5689.81 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221846.102917 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51208.28 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 5120.83 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226966.930643 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46087.45 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 4608.74 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231575.675597 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41478.70 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 4147.87 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235723.546055 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37330.83 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3733.08 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239456.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33597.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243056.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29997.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246656.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26397.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250256.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22797.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253856.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19197.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257456.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15597.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261056.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11997.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264656.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8397.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268256.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4797.75 sec) 01:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271856.629468 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197.75 sec) 01:15:00 [GPUGRID] [rr_sim] 2273054.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999641325.37G/439.78G) 01:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850595.29 01:15:00 [---] pick_jobs_to_run() start 01:15:00 [---] pick_jobs_to_run() end 01:15:00 [---] [rr_sim] end 01:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:15:00 [---] [cpu_sched_debug] preliminary job list: 01:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:15:00 [---] [cpu_sched_debug] final job list: 01:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 01:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:15:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 999641325.37G 01:15:00 [---] [cpu_sched_debug] enforce_run_list: end 01:15:00 [---] choose_project(): 4500.000000 01:15:00 [iThena.Measurements] doing RPC to get work 01:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431099.09 cpu 2173892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:15:00 [---] Got 0 tasks 01:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:16:00 [---] [rr_sim] doing sim: CPU sched 01:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999613735.01G/439.78G) 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272991.64 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 227299.16 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227299.164330 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045692.48 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 204569.25 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431868.412227 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841123.23 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 184112.32 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615980.735334 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1657010.91 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 165701.09 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781681.826130 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491309.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 149130.98 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930812.807847 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342178.84 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 134217.88 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065030.691392 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207960.95 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 120796.10 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185826.786583 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087164.86 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 108716.49 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294543.272255 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978448.37 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 97844.84 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392388.109359 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880603.53 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 88060.35 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480448.462753 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792543.18 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 79254.32 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559702.780808 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713288.86 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 71328.89 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1631031.667057 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641959.98 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 64196.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695227.664681 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577763.98 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 57776.40 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753004.062543 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519987.58 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 51998.76 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805002.820618 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467988.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 46798.88 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851801.702886 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421189.94 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 42118.99 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893920.696928 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379070.95 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 37907.09 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931827.791565 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341163.85 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 34116.39 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965944.176738 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307047.47 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 30704.75 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996648.923394 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276342.72 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 27634.27 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024283.195385 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248708.45 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 24870.84 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049154.040176 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223837.60 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 22383.76 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071537.800488 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201453.84 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 20145.38 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091683.184769 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181308.46 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 18130.85 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109814.030622 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163177.61 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 16317.76 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126131.791890 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146859.85 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 14685.99 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140817.777031 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132173.87 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 13217.39 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154035.163658 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118956.48 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 11895.65 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165930.811622 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107060.83 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 10706.08 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176636.894790 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96354.75 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 9635.47 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186272.369640 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86719.27 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 8671.93 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194944.297006 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78047.35 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 7804.73 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202749.031636 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70242.61 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 7024.26 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209773.292802 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63218.35 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 6321.84 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216095.127852 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56896.52 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 5689.65 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221784.779396 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51206.86 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 5120.69 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226905.465787 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46086.18 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 4608.62 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231514.083538 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41477.56 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 4147.76 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235661.839514 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37329.80 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3732.98 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239394.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33596.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242994.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29996.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246594.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26396.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250194.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22796.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253794.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19196.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257394.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15596.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260994.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11996.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264594.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8396.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268194.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4796.82 sec) 01:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271794.819892 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196.82 sec) 01:16:00 [GPUGRID] [rr_sim] 2272991.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999613735.01G/439.78G) 01:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850592.56 01:16:00 [---] pick_jobs_to_run() start 01:16:00 [---] pick_jobs_to_run() end 01:16:00 [---] [rr_sim] end 01:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:16:00 [---] [cpu_sched_debug] preliminary job list: 01:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 01:16:00 [---] [cpu_sched_debug] final job list: 01:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:16:00 [---] [cpu_sched_debug] enforce_run_list: end 01:16:00 [---] choose_project(): 4560.000000 01:16:00 [iThena.Measurements] doing RPC to get work 01:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 431039.09 cpu 2173832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:16:00 [---] Got 0 tasks 01:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:17:00 [---] [rr_sim] doing sim: CPU sched 01:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999586144.66G/439.78G) 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272928.91 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 227292.89 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227292.890642 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045636.02 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 204563.60 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431856.492219 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841072.41 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 184107.24 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615963.733639 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656965.17 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 165696.52 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781660.250917 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491268.66 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 149126.87 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930787.116467 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342141.79 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 134214.18 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1065001.295462 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207927.61 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 120792.76 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185794.056557 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087134.85 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 108713.48 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294507.541543 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978421.36 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 97842.14 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392349.678030 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880579.23 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 88057.92 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480407.600869 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792521.31 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 79252.13 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559659.731424 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713269.17 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 71326.92 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630986.648923 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641942.26 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 64194.23 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695180.874672 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577748.03 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 57774.80 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752955.677847 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519973.23 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 51997.32 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804953.000704 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467975.91 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 46797.59 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851750.591275 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421178.32 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 42117.83 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893868.422789 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379060.48 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 37906.05 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931774.471152 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341154.44 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 34115.44 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965889.914679 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307038.99 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 30703.90 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996593.813852 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276335.09 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 27633.51 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024227.323109 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248701.58 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 24870.16 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049097.481440 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223831.42 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 22383.14 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071480.623937 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201448.28 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 20144.83 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091625.452185 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181303.45 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 18130.35 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109755.797608 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163173.11 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 16317.31 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126073.108489 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146855.80 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 14685.58 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140758.688282 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132170.22 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 13217.02 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153975.710095 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118953.20 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 11895.32 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165871.029728 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107057.88 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 10705.79 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176576.817396 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96352.09 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 9635.21 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186212.026298 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86716.88 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 8671.69 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194883.714310 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78045.19 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 7804.52 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202688.233521 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70240.67 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 7024.07 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209712.300811 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63216.61 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 6321.66 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216033.961371 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56894.95 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 5689.49 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221723.455876 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51205.45 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 5120.55 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226844.000930 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46084.91 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 4608.49 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231452.491478 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41476.41 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 4147.64 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235600.132972 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37328.77 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3732.88 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239333.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33595.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242933.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29995.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246533.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26395.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250133.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22795.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253733.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19195.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257333.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15595.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260933.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11995.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264533.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8395.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268133.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4795.90 sec) 01:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271733.010317 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195.90 sec) 01:17:00 [GPUGRID] [rr_sim] 2272928.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999586144.66G/439.78G) 01:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850589.82 01:17:00 [---] pick_jobs_to_run() start 01:17:00 [---] pick_jobs_to_run() end 01:17:00 [---] [rr_sim] end 01:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:17:00 [---] [cpu_sched_debug] preliminary job list: 01:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:17:00 [---] [cpu_sched_debug] final job list: 01:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:17:00 [---] [cpu_sched_debug] enforce_run_list: end 01:17:00 [---] choose_project(): 4620.000000 01:17:00 [iThena.Measurements] doing RPC to get work 01:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430979.09 cpu 2173772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:17:00 [---] Got 0 tasks 01:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:18:00 [---] [rr_sim] doing sim: CPU sched 01:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999558554.30G/439.78G) 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272866.17 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 227286.62 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227286.616953 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045579.55 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 204557.96 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431844.572212 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1841021.60 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 184102.16 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615946.731944 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656919.44 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 165691.94 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781638.675703 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491227.49 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 149122.75 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930761.425086 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342104.74 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 134210.47 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064971.899531 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207894.27 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 120789.43 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185761.326531 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087104.84 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 108710.48 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294471.810832 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978394.36 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 97839.44 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392311.246702 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880554.92 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 88055.49 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480366.738985 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792499.43 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 79249.94 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559616.682040 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713249.49 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 71324.95 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630941.630790 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641924.54 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 64192.45 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695134.084664 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577732.08 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 57773.21 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752907.293151 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519958.88 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 51995.89 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804903.180789 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467962.99 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 46796.30 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851699.479664 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421166.69 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 42116.67 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893816.148651 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379050.02 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 37905.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931721.150739 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341145.02 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 34114.50 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965835.652619 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307030.52 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 30703.05 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996538.704310 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276327.47 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 27632.75 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024171.450833 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248694.72 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 24869.47 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049040.922703 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223825.25 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 22382.52 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071423.447386 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201442.72 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 20144.27 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091567.719601 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181298.45 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 18129.84 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109697.564594 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163168.60 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 16316.86 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126014.425088 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146851.74 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 14685.17 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140699.599533 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132166.57 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 13216.66 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153916.256533 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118949.91 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 11894.99 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165811.247833 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107054.92 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 10705.49 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176516.740003 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96349.43 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 9634.94 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186151.682956 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86714.49 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 8671.45 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194823.131614 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78043.04 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 7804.30 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202627.435406 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70238.73 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 7023.87 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209651.308819 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63214.86 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 6321.49 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215972.794891 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56893.37 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 5689.34 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221662.132355 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51204.04 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 5120.40 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226782.536073 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46083.63 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 4608.36 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231390.899419 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41475.27 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 4147.53 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235538.426431 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37327.74 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3732.77 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239271.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33594.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242871.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29994.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246471.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26394.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250071.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22794.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253671.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19194.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257271.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15594.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260871.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11994.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264471.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8394.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268071.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4794.97 sec) 01:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271671.200741 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194.97 sec) 01:18:00 [GPUGRID] [rr_sim] 2272866.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999558554.30G/439.78G) 01:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850587.08 01:18:00 [---] pick_jobs_to_run() start 01:18:00 [---] pick_jobs_to_run() end 01:18:00 [---] [rr_sim] end 01:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:18:00 [---] [cpu_sched_debug] preliminary job list: 01:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:18:00 [---] [cpu_sched_debug] final job list: 01:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:18:00 [---] [cpu_sched_debug] enforce_run_list: end 01:18:00 [---] choose_project(): 4680.000000 01:18:00 [iThena.Measurements] doing RPC to get work 01:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430919.09 cpu 2173712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:18:00 [---] Got 0 tasks 01:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:19:00 [---] [rr_sim] doing sim: CPU sched 01:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999530963.94G/439.78G) 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272803.43 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 227280.34 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227280.343265 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045523.09 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 204552.31 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431832.652204 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840970.78 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 184097.08 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615929.730249 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656873.70 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 165687.37 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781617.100489 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491186.33 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 149118.63 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930735.733705 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342067.70 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 134206.77 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064942.503600 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207860.93 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 120786.09 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185728.596505 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087074.84 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 108707.48 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294436.080120 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978367.35 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 97836.74 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392272.815373 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880530.62 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 88053.06 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480325.877101 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792477.56 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 79247.76 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559573.632656 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713229.80 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 71322.98 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630896.612656 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641906.82 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 64190.68 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695087.294656 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577716.14 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 57771.61 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752858.908455 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519944.52 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 51994.45 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804853.360875 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467950.07 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 46795.01 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851648.368053 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421155.06 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 42115.51 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893763.874513 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379039.56 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 37903.96 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931667.830327 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341135.60 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 34113.56 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965781.390559 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307022.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 30702.20 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996483.594769 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276319.84 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 27631.98 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024115.578557 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248687.85 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 24868.79 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048984.363966 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223819.07 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 22381.91 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071366.270835 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201437.16 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 20143.72 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091509.987017 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181293.45 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 18129.34 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109639.331580 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163164.10 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 16316.41 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125955.741688 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146847.69 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 14684.77 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140640.510784 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132162.92 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 13216.29 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153856.802971 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118946.63 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 11894.66 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165751.465939 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107051.97 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 10705.20 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176456.662610 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96346.77 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 9634.68 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186091.339615 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86712.09 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 8671.21 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194762.548918 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78040.88 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 7804.09 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202566.637292 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70236.80 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 7023.68 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209590.316828 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63213.12 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 6321.31 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215911.628410 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56891.80 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 5689.18 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221600.808834 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51202.62 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 5120.26 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226721.071216 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46082.36 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 4608.24 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231329.307360 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41474.13 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 4147.41 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235476.719889 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37326.71 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3732.67 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239209.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33594.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242809.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29994.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246409.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26394.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250009.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22794.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253609.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19194.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257209.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15594.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260809.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11994.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264409.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8394.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268009.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4794.04 sec) 01:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271609.391165 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194.04 sec) 01:19:00 [GPUGRID] [rr_sim] 2272803.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999530963.94G/439.78G) 01:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850584.35 01:19:00 [---] pick_jobs_to_run() start 01:19:00 [---] pick_jobs_to_run() end 01:19:00 [---] [rr_sim] end 01:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:19:00 [---] [cpu_sched_debug] preliminary job list: 01:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:19:00 [---] [cpu_sched_debug] final job list: 01:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:19:00 [---] [cpu_sched_debug] enforce_run_list: end 01:19:00 [---] choose_project(): 4740.000000 01:19:00 [iThena.Measurements] doing RPC to get work 01:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430859.09 cpu 2173652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:19:00 [---] Got 0 tasks 01:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:20:00 [---] [rr_sim] doing sim: CPU sched 01:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999503373.59G/439.78G) 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272740.70 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 227274.07 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227274.069577 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045466.63 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 204546.66 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431820.732196 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840919.96 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 184092.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615912.728554 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656827.97 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 165682.80 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781595.525275 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491145.17 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 149114.52 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930710.042325 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1342030.65 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 134203.07 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064913.107669 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207827.59 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 120782.76 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185695.866479 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087044.83 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 108704.48 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294400.349409 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978340.35 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 97834.03 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392234.384045 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880506.31 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 88050.63 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480285.015217 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792455.68 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 79245.57 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559530.583273 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713210.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 71321.01 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630851.594522 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641889.10 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 64188.91 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695040.504647 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577700.19 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 57770.02 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752810.523759 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519930.17 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 51993.02 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804803.540960 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467937.15 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 46793.72 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851597.256441 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421143.44 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 42114.34 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893711.600374 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379029.10 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 37902.91 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931614.509914 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341126.19 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 34112.62 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965727.128500 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307013.57 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 30701.36 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996428.485227 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276312.21 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 27631.22 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024059.706281 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248680.99 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 24868.10 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048927.805230 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223812.89 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 22381.29 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071309.094284 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201431.60 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 20143.16 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091452.254433 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181288.44 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 18128.84 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109581.098566 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163159.60 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 16315.96 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125897.058287 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146843.64 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 14684.36 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140581.422035 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132159.27 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 13215.93 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153797.349409 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118943.35 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 11894.33 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165691.684045 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107049.01 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 10704.90 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176396.585217 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96344.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 9634.41 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186030.996273 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86709.70 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 8670.97 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194701.966222 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78038.73 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 7803.87 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202505.839177 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70234.86 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 7023.49 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209529.324836 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63211.37 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 6321.14 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215850.461930 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56890.23 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 5689.02 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221539.485314 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51201.21 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 5120.12 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226659.606359 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46081.09 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 4608.11 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231267.715301 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41472.98 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 4147.30 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235415.013347 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37325.68 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3732.57 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239147.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33593.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242747.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29993.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246347.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26393.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249947.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22793.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253547.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19193.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257147.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15593.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260747.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11993.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264347.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8393.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267947.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4793.11 sec) 01:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271547.581590 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193.11 sec) 01:20:00 [GPUGRID] [rr_sim] 2272740.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999503373.59G/439.78G) 01:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850581.61 01:20:00 [---] pick_jobs_to_run() start 01:20:00 [---] pick_jobs_to_run() end 01:20:00 [---] [rr_sim] end 01:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:20:00 [---] [cpu_sched_debug] preliminary job list: 01:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:20:00 [---] [cpu_sched_debug] final job list: 01:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:20:00 [---] [cpu_sched_debug] enforce_run_list: end 01:20:00 [---] choose_project(): 4800.000000 01:20:00 [iThena.Measurements] doing RPC to get work 01:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430799.09 cpu 2173592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:20:00 [---] Got 0 tasks 01:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:21:00 [---] [rr_sim] doing sim: CPU sched 01:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999475783.23G/439.78G) 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272677.96 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 227267.80 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227267.795889 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045410.16 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 204541.02 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431808.812189 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840869.15 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 184086.91 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615895.726859 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656782.23 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 165678.22 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781573.950062 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491104.01 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 149110.40 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930684.350944 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341993.61 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 134199.36 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064883.711739 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207794.25 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 120779.42 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185663.136454 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087014.82 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 108701.48 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294364.618697 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978313.34 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 97831.33 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392195.952716 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880482.01 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 88048.20 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480244.153333 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792433.81 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 79243.38 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559487.533889 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713190.42 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 71319.04 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630806.576389 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641871.38 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 64187.14 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694993.714639 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577684.24 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 57768.42 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752762.139064 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519915.82 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 51991.58 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804753.721046 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467924.24 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 46792.42 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851546.144830 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421131.81 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 42113.18 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893659.326236 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379018.63 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 37901.86 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931561.189501 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341116.77 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 34111.68 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965672.866440 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307005.09 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 30700.51 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996373.375685 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276304.58 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 27630.46 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024003.834005 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248674.12 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 24867.41 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048871.246493 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223806.71 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 22380.67 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071251.917733 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201426.04 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 20142.60 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091394.521848 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181283.44 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 18128.34 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109522.865552 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163155.09 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 16315.51 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125838.374886 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146839.58 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 14683.96 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140522.333286 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132155.63 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 13215.56 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153737.895846 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118940.06 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 11894.01 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165631.902150 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107046.06 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 10704.61 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176336.507824 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96341.45 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 9634.15 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185970.652931 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86707.31 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 8670.73 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194641.383526 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78036.58 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 7803.66 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202445.041062 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70232.92 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 7023.29 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209468.332845 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63209.63 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 6320.96 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215789.295449 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56888.66 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 5688.87 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221478.161793 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51199.80 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 5119.98 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226598.141503 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46079.82 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 4607.98 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231206.123241 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41471.84 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 4147.18 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235353.306806 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37324.65 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3732.47 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239085.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33592.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242685.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29992.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246285.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26392.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249885.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22792.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253485.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19192.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257085.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15592.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260685.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11992.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264285.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8392.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267885.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4792.19 sec) 01:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271485.772014 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192.19 sec) 01:21:00 [GPUGRID] [rr_sim] 2272677.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999475783.23G/439.78G) 01:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850578.87 01:21:00 [---] pick_jobs_to_run() start 01:21:00 [---] pick_jobs_to_run() end 01:21:00 [---] [rr_sim] end 01:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:21:00 [---] [cpu_sched_debug] preliminary job list: 01:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:21:00 [---] [cpu_sched_debug] final job list: 01:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:21:00 [---] [cpu_sched_debug] enforce_run_list: end 01:21:00 [---] choose_project(): 4860.000000 01:21:00 [iThena.Measurements] doing RPC to get work 01:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430739.09 cpu 2173532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:21:00 [---] Got 0 tasks 01:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:22:00 [---] [rr_sim] doing sim: CPU sched 01:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999448192.88G/439.78G) 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272615.22 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 227261.52 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227261.522201 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045353.70 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 204535.37 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431796.892181 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840818.33 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 184081.83 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615878.725164 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656736.50 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 165673.65 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781552.374848 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491062.85 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 149106.28 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930658.659564 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341956.56 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 134195.66 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064854.315808 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207760.91 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 120776.09 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185630.406428 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086984.82 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 108698.48 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294328.887985 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978286.33 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 97828.63 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392157.521387 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880457.70 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 88045.77 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480203.291449 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792411.93 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 79241.19 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559444.484505 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713170.74 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 71317.07 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630761.558255 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641853.66 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 64185.37 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694946.924630 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577668.30 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 57766.83 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752713.754368 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519901.47 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 51990.15 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804703.901132 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467911.32 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 46791.13 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851495.033219 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421120.19 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 42112.02 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893607.052098 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379008.17 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 37900.82 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931507.869088 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341107.35 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 34110.74 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965618.604380 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306996.62 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 30699.66 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996318.266143 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276296.96 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 27629.70 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023947.961729 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248667.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 24866.73 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048814.687757 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223800.53 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 22380.05 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071194.741182 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201420.48 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 20142.05 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091336.789264 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181278.43 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 18127.84 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109464.632538 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163150.59 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 16315.06 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125779.691485 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146835.53 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 14683.55 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140463.244537 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132151.98 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 13215.20 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153678.442284 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118936.78 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 11893.68 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165572.120256 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107043.10 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 10704.31 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176276.430431 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96338.79 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 9633.88 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185910.309589 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86704.91 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 8670.49 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194580.800830 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78034.42 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 7803.44 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202384.242948 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70230.98 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 7023.10 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209407.340854 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63207.88 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 6320.79 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215728.128969 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56887.09 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 5688.71 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221416.838273 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51198.38 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 5119.84 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226536.676646 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46078.55 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 4607.85 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231144.531182 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41470.69 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 4147.07 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235291.600264 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37323.62 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3732.36 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239023.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33591.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242623.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29991.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246223.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26391.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249823.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22791.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253423.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19191.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257023.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15591.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260623.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11991.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264223.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8391.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267823.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4791.26 sec) 01:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271423.962438 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191.26 sec) 01:22:00 [GPUGRID] [rr_sim] 2272615.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999448192.88G/439.78G) 01:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850576.13 01:22:00 [---] pick_jobs_to_run() start 01:22:00 [---] pick_jobs_to_run() end 01:22:00 [---] [rr_sim] end 01:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:22:00 [---] [cpu_sched_debug] preliminary job list: 01:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:22:00 [---] [cpu_sched_debug] final job list: 01:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:22:00 [---] [cpu_sched_debug] enforce_run_list: end 01:22:00 [---] choose_project(): 4920.000000 01:22:00 [iThena.Measurements] doing RPC to get work 01:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430679.09 cpu 2173472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:22:00 [---] Got 0 tasks 01:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:23:00 [---] [rr_sim] doing sim: CPU sched 01:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999420602.52G/439.78G) 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272552.49 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 227255.25 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227255.248512 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045297.24 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 204529.72 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431784.972173 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840767.51 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 184076.75 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615861.723469 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656690.76 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 165669.08 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781530.799634 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1491021.69 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 149102.17 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930632.968183 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341919.52 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 134191.95 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064824.919877 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207727.57 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 120772.76 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185597.676402 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086954.81 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 108695.48 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294293.157274 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978259.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 97825.93 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392119.090059 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880433.40 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 88043.34 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480162.429565 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792390.06 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 79239.01 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559401.435121 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713151.05 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 71315.11 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630716.540121 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641835.95 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 64183.59 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694900.134622 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577652.35 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 57765.24 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752665.369672 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519887.12 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 51988.71 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804654.081217 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467898.40 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 46789.84 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851443.921608 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421108.56 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 42110.86 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893554.777959 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378997.71 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 37899.77 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931454.548676 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341097.94 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 34109.79 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965564.342321 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306988.14 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 30698.81 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996263.156601 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276289.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 27628.93 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023892.089453 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248660.40 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 24866.04 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048758.129020 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223794.36 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 22379.44 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071137.564631 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201414.92 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 20141.49 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091279.056680 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181273.43 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 18127.34 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109406.399524 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163146.09 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 16314.61 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125721.008084 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146831.48 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 14683.15 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140404.155788 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132148.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 13214.83 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153618.988722 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118933.50 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 11893.35 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165512.338362 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107040.15 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 10704.01 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176216.353038 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96336.13 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 9633.61 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185849.966247 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86702.52 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 8670.25 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194520.218134 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78032.27 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 7803.23 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202323.444833 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70229.04 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 7022.90 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209346.348862 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63206.14 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 6320.61 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215666.962488 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56885.52 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 5688.55 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221355.514752 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51196.97 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 5119.70 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226475.211789 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46077.27 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 4607.73 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231082.939123 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41469.55 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 4146.95 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235229.893723 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37322.59 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3732.26 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238962.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33590.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242562.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29990.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246162.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26390.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249762.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22790.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253362.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19190.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256962.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15590.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260562.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11990.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264162.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8390.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267762.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4790.33 sec) 01:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271362.152863 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190.33 sec) 01:23:00 [GPUGRID] [rr_sim] 2272552.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999420602.52G/439.78G) 01:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850573.40 01:23:00 [---] pick_jobs_to_run() start 01:23:00 [---] pick_jobs_to_run() end 01:23:00 [---] [rr_sim] end 01:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:23:00 [---] [cpu_sched_debug] preliminary job list: 01:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:23:00 [---] [cpu_sched_debug] final job list: 01:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:23:00 [---] [cpu_sched_debug] enforce_run_list: end 01:23:00 [---] choose_project(): 4980.000000 01:23:00 [iThena.Measurements] doing RPC to get work 01:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430619.09 cpu 2173412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:23:00 [---] Got 0 tasks 01:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:24:00 [---] [rr_sim] doing sim: CPU sched 01:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999393012.16G/439.78G) 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272489.75 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 227248.97 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227248.974824 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045240.77 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 204524.08 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431773.052166 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840716.70 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 184071.67 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615844.721773 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656645.03 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 165664.50 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781509.224420 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490980.52 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 149098.05 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930607.276802 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341882.47 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 134188.25 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064795.523946 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207694.22 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 120769.42 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185564.946376 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086924.80 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 108692.48 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294257.426562 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978232.32 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 97823.23 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392080.658730 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880409.09 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 88040.91 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480121.567681 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792368.18 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 79236.82 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559358.385737 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713131.36 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 71313.14 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630671.521988 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641818.23 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 64181.82 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694853.344613 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577636.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 57763.64 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752616.984976 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519872.76 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 51987.28 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804604.261303 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467885.49 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 46788.55 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851392.809996 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421096.94 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 42109.69 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893502.503821 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378987.24 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 37898.72 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931401.228263 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341088.52 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 34108.85 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965510.080261 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306979.67 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 30697.97 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996208.047059 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276281.70 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 27628.17 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023836.217177 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248653.53 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 24865.35 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048701.570284 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223788.18 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 22378.82 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071080.388079 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201409.36 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 20140.94 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091221.324096 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181268.42 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 18126.84 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109348.166510 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163141.58 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 16314.16 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125662.324683 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146827.42 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 14682.74 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140345.067039 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132144.68 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 13214.47 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153559.535159 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118930.21 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 11893.02 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165452.556468 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107037.19 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 10703.72 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176156.275645 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96333.47 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 9633.35 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185789.622905 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86700.13 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 8670.01 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194459.635438 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78030.11 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 7803.01 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202262.646719 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70227.10 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 7022.71 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209285.356871 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63204.39 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 6320.44 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215605.796008 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56883.95 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 5688.40 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221294.191231 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51195.56 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 5119.56 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226413.746932 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46076.00 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 4607.60 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231021.347063 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41468.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 4146.84 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235168.187181 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37321.56 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3732.16 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238900.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33589.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242500.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29989.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246100.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26389.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249700.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22789.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253300.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19189.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256900.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15589.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260500.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11989.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264100.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8389.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267700.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4789.40 sec) 01:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271300.343287 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189.40 sec) 01:24:00 [GPUGRID] [rr_sim] 2272489.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999393012.16G/439.78G) 01:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850570.66 01:24:00 [---] pick_jobs_to_run() start 01:24:00 [---] pick_jobs_to_run() end 01:24:00 [---] [rr_sim] end 01:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:24:00 [---] [cpu_sched_debug] preliminary job list: 01:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:24:00 [---] [cpu_sched_debug] final job list: 01:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:24:00 [---] [cpu_sched_debug] enforce_run_list: end 01:24:00 [---] choose_project(): 5040.000000 01:24:00 [iThena.Measurements] doing RPC to get work 01:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430559.09 cpu 2173352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:24:00 [---] Got 0 tasks 01:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:25:00 [---] [rr_sim] doing sim: CPU sched 01:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999365421.81G/439.78G) 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272427.01 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 227242.70 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227242.701136 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045184.31 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 204518.43 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431761.132158 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840665.88 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 184066.59 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615827.720078 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656599.29 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 165659.93 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781487.649206 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490939.36 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 149093.94 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930581.585422 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341845.43 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 134184.54 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064766.128016 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207660.88 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 120766.09 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185532.216350 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086894.80 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 108689.48 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294221.695851 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978205.32 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 97820.53 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392042.227402 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880384.78 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 88038.48 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480080.705797 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792346.31 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 79234.63 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559315.336354 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713111.68 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 71311.17 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630626.503854 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641800.51 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 64180.05 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694806.554605 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577620.46 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 57762.05 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752568.600280 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519858.41 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 51985.84 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804554.441388 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467872.57 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 46787.26 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851341.698385 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421085.31 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 42108.53 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893450.229683 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378976.78 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 37897.68 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931347.907850 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341079.10 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 34107.91 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965455.818201 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306971.19 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 30697.12 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996152.937517 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276274.07 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 27627.41 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023780.344901 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248646.67 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 24864.67 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048645.011547 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223782.00 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 22378.20 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071023.211528 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201403.80 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 20140.38 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091163.591511 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181263.42 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 18126.34 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109289.933496 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163137.08 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 16313.71 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125603.641283 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146823.37 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 14682.34 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140285.978290 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132141.03 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 13214.10 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153500.081597 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118926.93 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 11892.69 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165392.774573 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107034.24 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 10703.42 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176096.198252 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96330.81 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 9633.08 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185729.279563 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86697.73 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 8669.77 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194399.052742 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78027.96 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 7802.80 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202201.848604 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70225.16 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 7022.52 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209224.364880 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63202.65 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 6320.26 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215544.629528 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56882.38 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 5688.24 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221232.867711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51194.14 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 5119.41 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226352.282076 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46074.73 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 4607.47 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230959.755004 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41467.26 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 4146.73 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235106.480640 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37320.53 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3732.05 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238838.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33588.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242438.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29988.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246038.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26388.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249638.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22788.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253238.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19188.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256838.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15588.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260438.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11988.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264038.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8388.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267638.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4788.48 sec) 01:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271238.533711 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188.48 sec) 01:25:00 [GPUGRID] [rr_sim] 2272427.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999365421.81G/439.78G) 01:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850567.92 01:25:00 [---] pick_jobs_to_run() start 01:25:00 [---] pick_jobs_to_run() end 01:25:00 [---] [rr_sim] end 01:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:25:00 [---] [cpu_sched_debug] preliminary job list: 01:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:25:00 [---] [cpu_sched_debug] final job list: 01:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:25:00 [---] [cpu_sched_debug] enforce_run_list: end 01:25:00 [---] choose_project(): 5100.000000 01:25:00 [iThena.Measurements] doing RPC to get work 01:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430499.09 cpu 2173292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:25:00 [---] Got 0 tasks 01:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:26:00 [---] [rr_sim] doing sim: CPU sched 01:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999337831.45G/439.78G) 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272364.27 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 227236.43 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227236.427448 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045127.85 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 204512.78 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431749.212151 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840615.06 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 184061.51 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615810.718383 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656553.56 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 165655.36 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781466.073993 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490898.20 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 149089.82 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930555.894041 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341808.38 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 134180.84 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064736.732085 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207627.54 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 120762.75 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185499.486324 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086864.79 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 108686.48 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294185.965139 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978178.31 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 97817.83 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1392003.796073 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880360.48 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 88036.05 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1480039.843914 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792324.43 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 79232.44 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559272.286970 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713091.99 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 71309.20 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630581.485721 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641782.79 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 64178.28 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694759.764596 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577604.51 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 57760.45 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752520.215584 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519844.06 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 51984.41 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804504.621474 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467859.65 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 46785.97 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851290.586774 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421073.69 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 42107.37 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893397.955544 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378966.32 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 37896.63 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931294.587438 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341069.69 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 34106.97 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965401.556142 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306962.72 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 30696.27 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996097.827975 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276266.45 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 27626.64 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023724.472625 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248639.80 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 24863.98 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048588.452811 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223775.82 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 22377.58 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070966.034977 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201398.24 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 20139.82 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091105.858927 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181258.42 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 18125.84 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109231.700482 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163132.57 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 16313.26 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125544.957882 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146819.32 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 14681.93 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140226.889541 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132137.38 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 13213.74 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153440.628035 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118923.65 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 11892.36 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165332.992679 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107031.28 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 10703.13 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176036.120859 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96328.15 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 9632.82 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185668.936221 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86695.34 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 8669.53 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194338.470046 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78025.80 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 7802.58 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202141.050489 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70223.22 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 7022.32 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209163.372888 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63200.90 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 6320.09 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215483.463047 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56880.81 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 5688.08 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221171.544190 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51192.73 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 5119.27 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226290.817219 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46073.46 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 4607.35 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230898.162945 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41466.11 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 4146.61 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235044.774098 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37319.50 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3731.95 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238776.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33587.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242376.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29987.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245976.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26387.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249576.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22787.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253176.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19187.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256776.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15587.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260376.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11987.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263976.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8387.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267576.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4787.55 sec) 01:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271176.724136 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187.55 sec) 01:26:00 [GPUGRID] [rr_sim] 2272364.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999337831.45G/439.78G) 01:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850565.19 01:26:00 [---] pick_jobs_to_run() start 01:26:00 [---] pick_jobs_to_run() end 01:26:00 [---] [rr_sim] end 01:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:26:00 [---] [cpu_sched_debug] preliminary job list: 01:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:26:00 [---] [cpu_sched_debug] final job list: 01:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:26:00 [---] [cpu_sched_debug] enforce_run_list: end 01:26:00 [---] choose_project(): 5160.000000 01:26:00 [iThena.Measurements] doing RPC to get work 01:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430439.09 cpu 2173232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:26:00 [---] Got 0 tasks 01:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:27:00 [---] [rr_sim] doing sim: CPU sched 01:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999310241.09G/439.78G) 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272301.54 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 227230.15 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227230.153760 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045071.38 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 204507.14 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431737.292143 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840564.25 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 184056.42 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615793.716688 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656507.82 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 165650.78 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781444.498779 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490857.04 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 149085.70 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930530.202661 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341771.33 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 134177.13 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064707.336154 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207594.20 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 120759.42 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185466.756298 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086834.78 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 108683.48 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294150.234428 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978151.30 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 97815.13 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391965.364745 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880336.17 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 88033.62 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479998.982030 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792302.56 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 79230.26 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559229.237586 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713072.30 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 71307.23 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630536.467587 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641765.07 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 64176.51 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694712.974588 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577588.56 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 57758.86 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752471.830889 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519829.71 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 51982.97 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804454.801559 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467846.74 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 46784.67 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851239.475163 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421062.06 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 42106.21 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893345.681406 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378955.86 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 37895.59 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931241.267025 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341060.27 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 34106.03 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965347.294082 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306954.24 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 30695.42 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996042.718433 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276258.82 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 27625.88 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023668.600349 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248632.94 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 24863.29 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048531.894074 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223769.64 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 22376.96 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070908.858426 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201392.68 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 20139.27 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091048.126343 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181253.41 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 18125.34 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109173.467468 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163128.07 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 16312.81 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125486.274481 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146815.26 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 14681.53 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140167.800792 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132133.74 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 13213.37 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153381.174473 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118920.36 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 11892.04 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165273.210785 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107028.33 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 10702.83 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175976.043466 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96325.49 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 9632.55 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185608.592879 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86692.94 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 8669.29 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194277.887350 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78023.65 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 7802.37 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202080.252375 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70221.29 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 7022.13 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209102.380897 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63199.16 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 6319.92 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215422.296567 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56879.24 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 5687.92 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221110.220670 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51191.32 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 5119.13 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226229.352362 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46072.19 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 4607.22 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230836.570885 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41464.97 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 4146.50 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234983.067556 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37318.47 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3731.85 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238714.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33586.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242314.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29986.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245914.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26386.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249514.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22786.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253114.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19186.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256714.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15586.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260314.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11986.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263914.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8386.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267514.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4786.62 sec) 01:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271114.914560 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186.62 sec) 01:27:00 [GPUGRID] [rr_sim] 2272301.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999310241.09G/439.78G) 01:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850562.45 01:27:00 [---] pick_jobs_to_run() start 01:27:00 [---] pick_jobs_to_run() end 01:27:00 [---] [rr_sim] end 01:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:27:00 [---] [cpu_sched_debug] preliminary job list: 01:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:27:00 [---] [cpu_sched_debug] final job list: 01:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:27:00 [---] [cpu_sched_debug] enforce_run_list: end 01:27:00 [---] choose_project(): 5220.000000 01:27:00 [iThena.Measurements] doing RPC to get work 01:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430379.09 cpu 2173172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:27:00 [---] Got 0 tasks 01:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:28:00 [---] [rr_sim] doing sim: CPU sched 01:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999282650.74G/439.78G) 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272238.80 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 227223.88 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227223.880071 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2045014.92 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 204501.49 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431725.372135 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840513.43 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 184051.34 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615776.714993 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656462.09 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 165646.21 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781422.923565 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490815.88 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 149081.59 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930504.511280 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341734.29 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 134173.43 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064677.940223 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207560.86 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 120756.09 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185434.026272 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086804.77 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 108680.48 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294114.503716 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978124.30 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 97812.43 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391926.933416 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880311.87 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 88031.19 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479958.120146 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792280.68 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 79228.07 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559186.188202 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713052.61 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 71305.26 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630491.449453 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641747.35 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 64174.74 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694666.184579 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577572.62 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 57757.26 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752423.446193 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519815.35 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 51981.54 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804404.981645 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467833.82 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 46783.38 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851188.363552 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421050.44 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 42105.04 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893293.407268 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378945.39 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 37894.54 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931187.946612 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341050.85 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 34105.09 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965293.032022 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306945.77 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 30694.58 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995987.608891 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276251.19 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 27625.12 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023612.728073 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248626.07 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 24862.61 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048475.335337 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223763.47 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 22376.35 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070851.681875 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201387.12 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 20138.71 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090990.393759 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181248.41 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 18124.84 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109115.234454 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163123.57 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 16312.36 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125427.591080 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146811.21 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 14681.12 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140108.712043 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132130.09 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 13213.01 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153321.720910 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118917.08 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 11891.71 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165213.428891 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107025.37 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 10702.54 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175915.966073 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96322.83 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 9632.28 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185548.249537 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86690.55 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 8669.06 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194217.304654 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78021.50 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 7802.15 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202019.454260 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70219.35 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 7021.93 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209041.388905 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63197.41 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 6319.74 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215361.130086 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56877.67 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 5687.77 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221048.897149 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51189.90 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 5118.99 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226167.887505 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46070.91 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 4607.09 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230774.978826 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41463.82 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 4146.38 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234921.361015 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37317.44 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3731.74 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238653.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33585.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242253.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29985.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245853.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26385.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249453.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22785.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253053.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19185.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256653.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15585.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260253.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11985.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263853.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8385.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267453.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4785.70 sec) 01:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2271053.104985 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185.70 sec) 01:28:00 [GPUGRID] [rr_sim] 2272238.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999282650.74G/439.78G) 01:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850559.71 01:28:00 [---] pick_jobs_to_run() start 01:28:00 [---] pick_jobs_to_run() end 01:28:00 [---] [rr_sim] end 01:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:28:00 [---] [cpu_sched_debug] preliminary job list: 01:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:28:00 [---] [cpu_sched_debug] final job list: 01:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:28:00 [---] [cpu_sched_debug] enforce_run_list: end 01:28:00 [---] choose_project(): 5280.000000 01:28:00 [iThena.Measurements] doing RPC to get work 01:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430319.09 cpu 2173112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:28:00 [---] Got 0 tasks 01:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:29:00 [---] [rr_sim] doing sim: CPU sched 01:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999255060.38G/439.78G) 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272176.06 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 227217.61 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227217.606383 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044958.46 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 204495.85 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431713.452128 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840462.61 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 184046.26 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615759.713298 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656416.35 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 165641.64 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781401.348351 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490774.72 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 149077.47 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930478.819899 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341697.24 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 134169.72 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064648.544292 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207527.52 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 120752.75 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185401.296246 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086774.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 108677.48 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294078.773005 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978097.29 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 97809.73 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391888.502087 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880287.56 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 88028.76 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479917.258262 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792258.81 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 79225.88 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559143.138819 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713032.93 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 71303.29 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630446.431320 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641729.63 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 64172.96 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694619.394571 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577556.67 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 57755.67 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752375.061497 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519801.00 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 51980.10 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804355.161730 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467820.90 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 46782.09 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851137.251940 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421038.81 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 42103.88 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893241.133129 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378934.93 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 37893.49 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931134.626199 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341041.44 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 34104.14 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965238.769963 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306937.29 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 30693.73 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995932.499349 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276243.56 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 27624.36 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023556.855798 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248619.21 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 24861.92 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048418.776601 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223757.29 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 22375.73 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070794.505324 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201381.56 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 20138.16 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090932.661175 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181243.40 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 18124.34 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109057.001440 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163119.06 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 16311.91 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125368.907679 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146807.16 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 14680.72 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140049.623294 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132126.44 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 13212.64 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153262.267348 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118913.80 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 11891.38 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165153.646996 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107022.42 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 10702.24 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175855.888680 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96320.18 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 9632.02 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185487.906195 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86688.16 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 8668.82 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194156.721958 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78019.34 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 7801.93 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201958.656146 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70217.41 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 7021.74 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208980.396914 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63195.67 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 6319.57 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215299.963606 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56876.10 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 5687.61 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220987.573628 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51188.49 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 5118.85 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226106.422649 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46069.64 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 4606.96 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230713.386767 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41462.68 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 4146.27 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234859.654473 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37316.41 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3731.64 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238591.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33584.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242191.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29984.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245791.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26384.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249391.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22784.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252991.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19184.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256591.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15584.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260191.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11984.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263791.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8384.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267391.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4784.77 sec) 01:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270991.295409 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184.77 sec) 01:29:00 [GPUGRID] [rr_sim] 2272176.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999255060.38G/439.78G) 01:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850556.98 01:29:00 [---] pick_jobs_to_run() start 01:29:00 [---] pick_jobs_to_run() end 01:29:00 [---] [rr_sim] end 01:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:29:00 [---] [cpu_sched_debug] preliminary job list: 01:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:29:00 [---] [cpu_sched_debug] final job list: 01:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:29:00 [---] [cpu_sched_debug] enforce_run_list: end 01:29:00 [---] choose_project(): 5340.000000 01:29:00 [iThena.Measurements] doing RPC to get work 01:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 603059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430259.09 cpu 2173052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2173052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2173052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:29:00 [---] Got 0 tasks 01:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:31:00 [---] [rr_sim] doing sim: CPU sched 01:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999227470.03G/439.78G) 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272113.33 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 227211.33 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227211.332695 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044901.99 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 204490.20 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431701.532120 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840411.79 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 184041.18 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615742.711603 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656370.62 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 165637.06 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781379.773138 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490733.55 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 149073.36 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930453.128519 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341660.20 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 134166.02 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064619.148362 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207494.18 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 120749.42 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185368.566220 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086744.76 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 108674.48 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294043.042293 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978070.28 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 97807.03 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391850.070759 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880263.26 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 88026.33 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479876.396378 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792236.93 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 79223.69 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559100.089435 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713013.24 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 71301.32 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630401.413186 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641711.91 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 64171.19 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694572.604562 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577540.72 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 57754.07 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752326.676801 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519786.65 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 51978.67 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804305.341816 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467807.99 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 46780.80 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851086.140329 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421027.19 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 42102.72 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893188.858991 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378924.47 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 37892.45 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931081.305787 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341032.02 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 34103.20 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965184.507903 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306928.82 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 30692.88 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995877.389807 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276235.94 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 27623.59 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023500.983522 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248612.34 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 24861.23 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048362.217864 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223751.11 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 22375.11 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070737.328773 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201376.00 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 20137.60 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090874.928590 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181238.40 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 18123.84 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108998.768426 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163114.56 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 16311.46 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125310.224278 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146803.10 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 14680.31 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139990.534545 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132122.79 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 13212.28 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153202.813786 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118910.51 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 11891.05 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165093.865102 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107019.46 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 10701.95 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175795.811287 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96317.52 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 9631.75 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185427.562853 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86685.76 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 8668.58 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194096.139262 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78017.19 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 7801.72 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201897.858031 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70215.47 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 7021.55 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208919.404923 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63193.92 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 6319.39 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215238.797125 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56874.53 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 5687.45 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220926.250108 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51187.08 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 5118.71 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226044.957792 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46068.37 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 4606.84 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230651.794707 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41461.53 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 4146.15 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234797.947932 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37315.38 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3731.54 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238529.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33583.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242129.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29983.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245729.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26383.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249329.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22783.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252929.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19183.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256529.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15583.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260129.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11983.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263729.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8383.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267329.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4783.84 sec) 01:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270929.485833 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183.84 sec) 01:31:00 [GPUGRID] [rr_sim] 2272113.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999227470.03G/439.78G) 01:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850614.24 01:31:00 [---] pick_jobs_to_run() start 01:31:00 [---] pick_jobs_to_run() end 01:31:00 [---] [rr_sim] end 01:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:31:00 [---] [cpu_sched_debug] preliminary job list: 01:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:31:00 [---] [cpu_sched_debug] final job list: 01:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:31:00 [---] [cpu_sched_debug] enforce_run_list: end 01:31:00 [---] choose_project(): 5460.000000 01:31:00 [iThena.Measurements] doing RPC to get work 01:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430139.09 cpu 2172992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:31:00 [---] Got 0 tasks 01:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:32:00 [---] [rr_sim] doing sim: CPU sched 01:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999199879.67G/439.78G) 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2272050.59 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 227205.06 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227205.059007 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044845.53 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 204484.55 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431689.612113 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840360.98 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 184036.10 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615725.709908 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656324.88 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 165632.49 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781358.197924 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490692.39 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 149069.24 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930427.437138 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341623.15 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 134162.32 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064589.752431 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207460.84 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 120746.08 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185335.836194 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086714.75 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 108671.48 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1294007.311582 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978043.28 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 97804.33 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391811.639430 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880238.95 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 88023.90 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479835.534494 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792215.06 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 79221.51 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559057.040051 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712993.55 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 71299.36 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630356.395053 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641694.20 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 64169.42 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694525.814554 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577524.78 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 57752.48 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752278.292105 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519772.30 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 51977.23 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804255.521901 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467795.07 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 46779.51 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851035.028718 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421015.56 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 42101.56 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893136.584853 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378914.01 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 37891.40 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931027.985374 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341022.60 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 34102.26 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965130.245843 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306920.34 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 30692.03 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995822.280266 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276228.31 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 27622.83 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023445.111246 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248605.48 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 24860.55 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048305.659128 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223744.93 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 22374.49 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070680.152222 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201370.44 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 20137.04 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090817.196006 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181233.39 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 18123.34 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108940.535412 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163110.05 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 16311.01 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125251.540878 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146799.05 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 14679.90 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139931.445796 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132119.14 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 13211.91 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153143.360223 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118907.23 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 11890.72 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165034.083208 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107016.51 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 10701.65 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175735.733894 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96314.86 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 9631.49 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185367.219511 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86683.37 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 8668.34 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194035.556566 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78015.03 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 7801.50 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201837.059916 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70213.53 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 7021.35 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208858.412931 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63192.18 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 6319.22 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215177.630645 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56872.96 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 5687.30 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220864.926587 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51185.66 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 5118.57 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225983.492935 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46067.10 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 4606.71 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230590.202648 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41460.39 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 4146.04 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234736.241390 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37314.35 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3731.43 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238467.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33582.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242067.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29982.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245667.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26382.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249267.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22782.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252867.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19182.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256467.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15582.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260067.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11982.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263667.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8382.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267267.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4782.91 sec) 01:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270867.676258 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182.91 sec) 01:32:00 [GPUGRID] [rr_sim] 2272050.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999199879.67G/439.78G) 01:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850611.50 01:32:00 [---] pick_jobs_to_run() start 01:32:00 [---] pick_jobs_to_run() end 01:32:00 [---] [rr_sim] end 01:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:32:00 [---] [cpu_sched_debug] preliminary job list: 01:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:32:00 [---] [cpu_sched_debug] final job list: 01:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:32:00 [---] [cpu_sched_debug] enforce_run_list: end 01:32:00 [---] choose_project(): 5520.000000 01:32:00 [iThena.Measurements] doing RPC to get work 01:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430079.09 cpu 2172932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:32:00 [---] Got 0 tasks 01:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:33:00 [---] [rr_sim] doing sim: CPU sched 01:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999172289.31G/439.78G) 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271987.85 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 227198.79 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227198.785318 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044789.07 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 204478.91 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431677.692105 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840310.16 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 184031.02 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615708.708213 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656279.14 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 165627.91 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781336.622710 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490651.23 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 149065.12 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930401.745757 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341586.11 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 134158.61 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064560.356500 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207427.50 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 120742.75 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185303.106169 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086684.75 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 108668.47 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293971.580870 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978016.27 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 97801.63 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391773.208102 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880214.65 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 88021.46 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479794.672610 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792193.18 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 79219.32 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1559013.990667 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712973.86 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 71297.39 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630311.376919 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641676.48 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 64167.65 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694479.024545 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577508.83 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 57750.88 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752229.907409 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519757.95 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 51975.79 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804205.701987 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467782.15 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 46778.22 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850983.917107 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421003.94 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 42100.39 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893084.310714 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378903.54 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 37890.35 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930974.664961 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341013.19 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 34101.32 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965075.983784 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306911.87 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 30691.19 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995767.170724 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276220.68 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 27622.07 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023389.238970 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248598.61 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 24859.86 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048249.100391 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223738.75 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 22373.88 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070622.975670 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201364.88 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 20136.49 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090759.463422 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181228.39 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 18122.84 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108882.302398 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163105.55 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 16310.56 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125192.857477 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146795.00 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 14679.50 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139872.357047 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132115.50 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 13211.55 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153083.906661 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118903.95 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 11890.39 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164974.301313 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107013.55 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 10701.36 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175675.656501 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96312.20 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 9631.22 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185306.876169 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86680.98 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 8668.10 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193974.973870 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78012.88 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 7801.29 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201776.261802 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70211.59 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 7021.16 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208797.420940 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63190.43 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 6319.04 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215116.464164 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56871.39 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 5687.14 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220803.603066 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51184.25 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 5118.43 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225922.028078 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46065.83 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 4606.58 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230528.610589 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41459.24 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 4145.92 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234674.534848 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37313.32 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3731.33 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238405.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33581.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242005.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29981.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245605.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26381.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249205.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22781.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252805.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19181.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256405.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15581.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260005.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11981.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263605.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8381.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267205.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4781.99 sec) 01:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270805.866682 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181.99 sec) 01:33:00 [GPUGRID] [rr_sim] 2271987.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999172289.31G/439.78G) 01:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850608.77 01:33:00 [---] pick_jobs_to_run() start 01:33:00 [---] pick_jobs_to_run() end 01:33:00 [---] [rr_sim] end 01:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:33:00 [---] [cpu_sched_debug] preliminary job list: 01:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:33:00 [---] [cpu_sched_debug] final job list: 01:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:33:00 [---] [cpu_sched_debug] enforce_run_list: end 01:33:00 [---] choose_project(): 5580.000000 01:33:00 [iThena.Measurements] doing RPC to get work 01:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 430019.09 cpu 2172872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:33:00 [---] Got 0 tasks 01:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:34:00 [---] [rr_sim] doing sim: CPU sched 01:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999144698.96G/439.78G) 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271925.12 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 227192.51 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227192.511630 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044732.60 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 204473.26 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431665.772097 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840259.34 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 184025.93 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615691.706518 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656233.41 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 165623.34 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781315.047496 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490610.07 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 149061.01 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930376.054377 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341549.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 134154.91 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064530.960569 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207394.16 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 120739.42 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185270.376143 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086654.74 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 108665.47 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293935.850159 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977989.27 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 97798.93 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391734.776773 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880190.34 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 88019.03 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479753.810726 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792171.31 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 79217.13 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558970.941283 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712954.18 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 71295.42 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630266.358785 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641658.76 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 64165.88 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694432.234537 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577492.88 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 57749.29 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752181.522713 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519743.59 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 51974.36 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804155.882072 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467769.23 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 46776.92 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850932.805495 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420992.31 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 42099.23 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893032.036576 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378893.08 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 37889.31 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930921.344549 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341003.77 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 34100.38 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965021.721724 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306903.39 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 30690.34 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995712.061182 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276213.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 27621.31 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023333.366694 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248591.75 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 24859.17 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048192.541655 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223732.57 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 22373.26 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070565.799119 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201359.32 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 20135.93 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090701.730838 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181223.39 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 18122.34 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108824.069384 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163101.05 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 16310.10 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125134.174076 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146790.94 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 14679.09 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139813.268298 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132111.85 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 13211.18 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153024.453099 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118900.66 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 11890.07 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164914.519419 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107010.60 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 10701.06 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175615.579107 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96309.54 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 9630.95 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185246.532827 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86678.58 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 8667.86 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193914.391174 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78010.73 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 7801.07 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201715.463687 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70209.65 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 7020.97 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208736.428949 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63188.69 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 6318.87 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215055.297684 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56869.82 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 5686.98 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220742.279546 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51182.84 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 5118.28 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225860.563221 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46064.55 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 4606.46 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230467.018529 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41458.10 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 4145.81 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234612.828307 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37312.29 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3731.23 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238344.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33581.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241944.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29981.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245544.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26381.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249144.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22781.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252744.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19181.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256344.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15581.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259944.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11981.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263544.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8381.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267144.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4781.06 sec) 01:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270744.057106 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181.06 sec) 01:34:00 [GPUGRID] [rr_sim] 2271925.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999144698.96G/439.78G) 01:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850606.03 01:34:00 [---] pick_jobs_to_run() start 01:34:00 [---] pick_jobs_to_run() end 01:34:00 [---] [rr_sim] end 01:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:34:00 [---] [cpu_sched_debug] preliminary job list: 01:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:34:00 [---] [cpu_sched_debug] final job list: 01:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:34:00 [---] [cpu_sched_debug] enforce_run_list: end 01:34:00 [---] choose_project(): 5640.000000 01:34:00 [iThena.Measurements] doing RPC to get work 01:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429959.09 cpu 2172812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:34:00 [---] Got 0 tasks 01:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:35:00 [---] [rr_sim] doing sim: CPU sched 01:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999117108.60G/439.78G) 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271862.38 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 227186.24 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227186.237942 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044676.14 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 204467.61 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431653.852090 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840208.53 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 184020.85 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615674.704823 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656187.67 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 165618.77 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781293.472282 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490568.91 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 149056.89 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930350.362996 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341512.02 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 134151.20 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064501.564639 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207360.81 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 120736.08 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185237.646117 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086624.73 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 108662.47 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293900.119447 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977962.26 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 97796.23 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391696.345444 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880166.03 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 88016.60 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479712.948842 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792149.43 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 79214.94 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558927.891900 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712934.49 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 71293.45 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630221.340652 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641641.04 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 64164.10 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694385.444529 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577476.93 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 57747.69 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752133.138018 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519729.24 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 51972.92 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804106.062158 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467756.32 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 46775.63 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850881.693884 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420980.69 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 42098.07 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892979.762438 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378882.62 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 37888.26 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930868.024136 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340994.36 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 34099.44 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964967.459664 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306894.92 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 30689.49 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995656.951640 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276205.43 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 27620.54 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023277.494418 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248584.89 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 24858.49 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048135.982918 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223726.40 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 22372.64 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070508.622568 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201353.76 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 20135.38 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090643.998253 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181218.38 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 18121.84 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108765.836370 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163096.54 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 16309.65 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125075.490675 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146786.89 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 14678.69 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139754.179550 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132108.20 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 13210.82 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152964.999537 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118897.38 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 11889.74 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164854.737525 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107007.64 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 10700.76 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175555.501714 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96306.88 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 9630.69 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185186.189485 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86676.19 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 8667.62 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193853.808478 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78008.57 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 7800.86 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201654.665573 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70207.71 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 7020.77 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208675.436957 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63186.94 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 6318.69 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214994.131204 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56868.25 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 5686.82 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220680.956025 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51181.42 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 5118.14 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225799.098365 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46063.28 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 4606.33 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230405.426470 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41456.95 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 4145.70 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234551.121765 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37311.26 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3731.13 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238282.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33580.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241882.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29980.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245482.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26380.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249082.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22780.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252682.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19180.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256282.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15580.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259882.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11980.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263482.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8380.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267082.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4780.13 sec) 01:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270682.247531 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180.13 sec) 01:35:00 [GPUGRID] [rr_sim] 2271862.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999117108.60G/439.78G) 01:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850603.29 01:35:00 [---] pick_jobs_to_run() start 01:35:00 [---] pick_jobs_to_run() end 01:35:00 [---] [rr_sim] end 01:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:35:00 [---] [cpu_sched_debug] preliminary job list: 01:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:35:00 [---] [cpu_sched_debug] final job list: 01:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:35:00 [---] [cpu_sched_debug] enforce_run_list: end 01:35:00 [---] choose_project(): 5700.000000 01:35:00 [iThena.Measurements] doing RPC to get work 01:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429899.09 cpu 2172752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:35:00 [---] Got 0 tasks 01:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:36:00 [---] [rr_sim] doing sim: CPU sched 01:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999089518.25G/439.78G) 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271799.64 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 227179.96 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227179.964254 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044619.68 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 204461.97 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431641.932082 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840157.71 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 184015.77 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615657.703128 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656141.94 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 165614.19 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781271.897069 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490527.75 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 149052.77 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930324.671616 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341474.97 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 134147.50 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064472.168708 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207327.47 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 120732.75 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185204.916091 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086594.73 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 108659.47 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293864.388736 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977935.25 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 97793.53 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391657.914116 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880141.73 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 88014.17 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479672.086958 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792127.56 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 79212.76 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558884.842516 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712914.80 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 71291.48 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630176.322518 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641623.32 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 64162.33 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694338.654520 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577460.99 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 57746.10 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752084.753322 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519714.89 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 51971.49 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804056.242243 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467743.40 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 46774.34 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850830.582273 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420969.06 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 42096.91 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892927.488299 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378872.15 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 37887.22 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930814.703723 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340984.94 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 34098.49 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964913.197605 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306886.44 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 30688.64 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995601.842098 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276197.80 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 27619.78 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023221.622142 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248578.02 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 24857.80 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048079.424181 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223720.22 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 22372.02 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070451.446017 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201348.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 20134.82 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090586.265669 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181213.38 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 18121.34 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108707.603356 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163092.04 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 16309.20 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125016.807274 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146782.84 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 14678.28 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139695.090801 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132104.55 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 13210.46 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152905.545974 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118894.10 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 11889.41 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164794.955631 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107004.69 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 10700.47 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175495.424321 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96304.22 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 9630.42 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185125.846143 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86673.80 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 8667.38 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193793.225782 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78006.42 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 7800.64 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201593.867458 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70205.78 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 7020.58 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208614.444966 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63185.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 6318.52 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214932.964723 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56866.68 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 5686.67 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220619.632505 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51180.01 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 5118.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225737.633508 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46062.01 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 4606.20 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230343.834411 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41455.81 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 4145.58 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234489.415224 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37310.23 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3731.02 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238220.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33579.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241820.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29979.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245420.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26379.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249020.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22779.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252620.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19179.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256220.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15579.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259820.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11979.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263420.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8379.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267020.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4779.20 sec) 01:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270620.437955 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179.20 sec) 01:36:00 [GPUGRID] [rr_sim] 2271799.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999089518.25G/439.78G) 01:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850600.56 01:36:00 [---] pick_jobs_to_run() start 01:36:00 [---] pick_jobs_to_run() end 01:36:00 [---] [rr_sim] end 01:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:36:00 [---] [cpu_sched_debug] preliminary job list: 01:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:36:00 [---] [cpu_sched_debug] final job list: 01:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:36:00 [---] [cpu_sched_debug] enforce_run_list: end 01:36:00 [---] choose_project(): 5760.000000 01:36:00 [iThena.Measurements] doing RPC to get work 01:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429839.09 cpu 2172692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:36:00 [---] Got 0 tasks 01:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:37:00 [---] [rr_sim] doing sim: CPU sched 01:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999061927.89G/439.78G) 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271736.91 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 227173.69 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227173.690566 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044563.22 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 204456.32 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431630.012075 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840106.89 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 184010.69 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615640.701433 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656096.20 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 165609.62 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781250.321855 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490486.58 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 149048.66 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930298.980235 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341437.93 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 134143.79 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064442.772777 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207294.13 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 120729.41 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185172.186065 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086564.72 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 108656.47 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293828.658024 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977908.25 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 97790.82 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391619.482787 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880117.42 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 88011.74 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479631.225074 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792105.68 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 79210.57 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558841.793132 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712895.11 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 71289.51 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630131.304384 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641605.60 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 64160.56 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694291.864512 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577445.04 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 57744.50 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752036.368626 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519700.54 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 51970.05 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804006.422329 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467730.48 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 46773.05 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850779.470662 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420957.43 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 42095.74 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892875.214161 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378861.69 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 37886.17 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930761.383310 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340975.52 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 34097.55 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964858.935545 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306877.97 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 30687.80 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995546.732556 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276190.17 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 27619.02 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023165.749866 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248571.16 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 24857.12 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048022.865445 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223714.04 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 22371.40 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070394.269466 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201342.64 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 20134.26 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090528.533085 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181208.37 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 18120.84 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108649.370342 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163087.54 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 16308.75 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124958.123873 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146778.78 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 14677.88 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139636.002052 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132100.90 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 13210.09 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152846.092412 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118890.81 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 11889.08 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164735.173736 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107001.73 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 10700.17 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175435.346928 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96301.56 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 9630.16 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185065.502801 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86671.40 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 8667.14 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193732.643086 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78004.26 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 7800.43 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201533.069343 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70203.84 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 7020.38 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208553.452975 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63183.45 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 6318.35 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214871.798243 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56865.11 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 5686.51 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220558.308984 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51178.60 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 5117.86 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225676.168651 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46060.74 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 4606.07 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230282.242352 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41454.66 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 4145.47 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234427.708682 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37309.20 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3730.92 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238158.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33578.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241758.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29978.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245358.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26378.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248958.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22778.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252558.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19178.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256158.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15578.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259758.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11978.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263358.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8378.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266958.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4778.28 sec) 01:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270558.628379 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178.28 sec) 01:37:00 [GPUGRID] [rr_sim] 2271736.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999061927.89G/439.78G) 01:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850597.82 01:37:00 [---] pick_jobs_to_run() start 01:37:00 [---] pick_jobs_to_run() end 01:37:00 [---] [rr_sim] end 01:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:37:00 [---] [cpu_sched_debug] preliminary job list: 01:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:37:00 [---] [cpu_sched_debug] final job list: 01:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:37:00 [---] [cpu_sched_debug] enforce_run_list: end 01:37:00 [---] choose_project(): 5820.000000 01:37:00 [iThena.Measurements] doing RPC to get work 01:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429779.09 cpu 2172632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:37:00 [---] Got 0 tasks 01:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:38:00 [---] [rr_sim] doing sim: CPU sched 01:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999034337.53G/439.78G) 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271674.17 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 227167.42 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227167.416877 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044506.75 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 204450.68 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431618.092067 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840056.08 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 184005.61 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615623.699738 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656050.47 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 165605.05 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781228.746641 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490445.42 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 149044.54 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930273.288854 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341400.88 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 134140.09 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064413.376846 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207260.79 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 120726.08 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185139.456039 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086534.71 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 108653.47 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293792.927312 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977881.24 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 97788.12 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391581.051459 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880093.12 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 88009.31 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479590.363190 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792083.81 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 79208.38 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558798.743748 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712875.43 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 71287.54 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630086.286251 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641587.88 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 64158.79 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694245.074503 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577429.09 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 57742.91 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751987.983930 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519686.18 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 51968.62 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803956.602414 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467717.57 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 46771.76 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850728.359050 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420945.81 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 42094.58 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892822.940023 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378851.23 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 37885.12 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930708.062898 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340966.11 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 34096.61 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964804.673485 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306869.50 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 30686.95 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995491.623014 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276182.55 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 27618.25 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023109.877590 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248564.29 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 24856.43 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047966.306708 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223707.86 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 22370.79 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070337.092915 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201337.08 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 20133.71 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090470.800501 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181203.37 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 18120.34 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108591.137328 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163083.03 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 16308.30 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124899.440473 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146774.73 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 14677.47 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139576.913303 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132097.26 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 13209.73 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152786.638850 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118887.53 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 11888.75 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164675.391842 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106998.78 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 10699.88 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175375.269535 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96298.90 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 9629.89 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185005.159459 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86669.01 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 8666.90 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193672.060390 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78002.11 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 7800.21 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201472.271229 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70201.90 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 7020.19 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208492.460983 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63181.71 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 6318.17 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214810.631762 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56863.54 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 5686.35 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220496.985463 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51177.18 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 5117.72 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225614.703794 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46059.46 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 4605.95 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230220.650292 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41453.52 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 4145.35 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234366.002140 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37308.17 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3730.82 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238096.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33577.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241696.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29977.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245296.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26377.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248896.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22777.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252496.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19177.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256096.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15577.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259696.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11977.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263296.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8377.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266896.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4777.35 sec) 01:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270496.818804 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177.35 sec) 01:38:00 [GPUGRID] [rr_sim] 2271674.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999034337.53G/439.78G) 01:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850595.08 01:38:00 [---] pick_jobs_to_run() start 01:38:00 [---] pick_jobs_to_run() end 01:38:00 [---] [rr_sim] end 01:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:38:00 [---] [cpu_sched_debug] preliminary job list: 01:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:38:00 [---] [cpu_sched_debug] final job list: 01:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:38:00 [---] [cpu_sched_debug] enforce_run_list: end 01:38:00 [---] choose_project(): 5880.000000 01:38:00 [iThena.Measurements] doing RPC to get work 01:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429719.09 cpu 2172572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:38:00 [---] Got 0 tasks 01:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:39:00 [---] [rr_sim] doing sim: CPU sched 01:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (999006747.18G/439.78G) 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271611.43 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 227161.14 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227161.143189 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044450.29 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 204445.03 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431606.172059 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1840005.26 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 184000.53 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615606.698043 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1656004.73 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 165600.47 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781207.171427 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490404.26 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 149040.43 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930247.597474 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341363.83 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 134136.38 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064383.980916 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207227.45 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 120722.75 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185106.726013 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086504.71 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 108650.47 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293757.196601 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977854.24 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 97785.42 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391542.620130 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880068.81 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 88006.88 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479549.501306 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792061.93 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 79206.19 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558755.694365 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712855.74 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 71285.57 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1630041.268117 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641570.16 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 64157.02 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694198.284495 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577413.15 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 57741.31 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751939.599234 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519671.83 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 51967.18 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803906.782500 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467704.65 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 46770.46 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850677.247439 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420934.18 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 42093.42 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892770.665884 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378840.77 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 37884.08 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930654.742485 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340956.69 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 34095.67 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964750.411426 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306861.02 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 30686.10 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995436.513472 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276174.92 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 27617.49 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023054.005314 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248557.43 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 24855.74 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047909.747972 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223701.68 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 22370.17 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070279.916364 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201331.52 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 20133.15 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090413.067916 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181198.36 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 18119.84 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108532.904314 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163078.53 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 16307.85 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124840.757072 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146770.67 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 14677.07 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139517.824554 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132093.61 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 13209.36 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152727.185287 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118884.25 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 11888.42 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164615.609948 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106995.82 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 10699.58 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175315.192142 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96296.24 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 9629.62 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184944.816117 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86666.62 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 8666.66 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193611.477694 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77999.95 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 7800.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201411.473114 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70199.96 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 7020.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208431.468992 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63179.96 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 6318.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214749.465282 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56861.97 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 5686.20 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220435.661943 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51175.77 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 5117.58 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225553.238938 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46058.19 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 4605.82 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230159.058233 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41452.37 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 4145.24 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234304.295599 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37307.14 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3730.71 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238035.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33576.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241635.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29976.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245235.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26376.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248835.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22776.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252435.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19176.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256035.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15576.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259635.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11976.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263235.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8376.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266835.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4776.42 sec) 01:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270435.009228 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176.42 sec) 01:39:00 [GPUGRID] [rr_sim] 2271611.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (999006747.18G/439.78G) 01:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850592.34 01:39:00 [---] pick_jobs_to_run() start 01:39:00 [---] pick_jobs_to_run() end 01:39:00 [---] [rr_sim] end 01:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:39:00 [---] [cpu_sched_debug] preliminary job list: 01:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:39:00 [---] [cpu_sched_debug] final job list: 01:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:39:00 [---] [cpu_sched_debug] enforce_run_list: end 01:39:00 [---] choose_project(): 5940.000000 01:39:00 [iThena.Measurements] doing RPC to get work 01:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429659.09 cpu 2172512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:39:00 [---] Got 0 tasks 01:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:40:00 [---] [rr_sim] doing sim: CPU sched 01:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998979156.82G/439.78G) 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271548.70 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 227154.87 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227154.869501 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044393.83 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 204439.38 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431594.252052 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839954.44 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 183995.44 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615589.696347 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655959.00 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 165595.90 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781185.596214 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490363.10 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 149036.31 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930221.906093 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341326.79 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 134132.68 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064354.584985 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207194.11 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 120719.41 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185073.995987 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086474.70 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 108647.47 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293721.465889 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977827.23 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 97782.72 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391504.188801 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880044.51 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 88004.45 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479508.639422 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792040.06 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 79204.01 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558712.644981 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712836.05 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 71283.61 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629996.249984 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641552.45 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 64155.24 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694151.494486 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577397.20 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 57739.72 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751891.214538 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519657.48 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 51965.75 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803856.962586 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467691.73 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 46769.17 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850626.135828 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420922.56 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 42092.26 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892718.391746 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378830.30 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 37883.03 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930601.422072 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340947.27 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 34094.73 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964696.149366 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306852.55 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 30685.25 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995381.403930 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276167.29 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 27616.73 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022998.133038 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248550.56 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 24855.06 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047853.189235 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223695.51 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 22369.55 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070222.739813 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201325.96 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 20132.60 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090355.335332 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181193.36 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 18119.34 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108474.671300 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163074.02 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 16307.40 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124782.073671 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146766.62 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 14676.66 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139458.735805 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132089.96 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 13209.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152667.731725 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118880.96 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 11888.10 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164555.828054 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106992.87 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 10699.29 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175255.114749 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96293.58 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 9629.36 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184884.472775 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86664.22 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 8666.42 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193550.894998 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77997.80 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 7799.78 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201350.675000 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70198.02 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 7019.80 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208370.477000 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63178.22 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 6317.82 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214688.298801 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56860.40 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 5686.04 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220374.338422 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51174.36 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 5117.44 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225491.774081 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46056.92 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 4605.69 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230097.466174 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41451.23 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 4145.12 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234242.589057 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37306.11 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3730.61 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237973.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33575.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241573.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29975.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245173.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26375.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248773.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22775.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252373.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19175.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255973.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15575.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259573.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11975.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263173.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8375.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266773.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4775.50 sec) 01:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270373.199652 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175.50 sec) 01:40:00 [GPUGRID] [rr_sim] 2271548.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998979156.82G/439.78G) 01:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850589.61 01:40:00 [---] pick_jobs_to_run() start 01:40:00 [---] pick_jobs_to_run() end 01:40:00 [---] [rr_sim] end 01:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:40:00 [---] [cpu_sched_debug] preliminary job list: 01:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:40:00 [---] [cpu_sched_debug] final job list: 01:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:40:00 [---] [cpu_sched_debug] enforce_run_list: end 01:40:00 [---] choose_project(): 6000.000000 01:40:00 [iThena.Measurements] doing RPC to get work 01:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429599.09 cpu 2172452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:40:00 [---] Got 0 tasks 01:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:41:00 [---] [rr_sim] doing sim: CPU sched 01:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998951566.46G/439.78G) 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271485.96 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 227148.60 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227148.595813 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044337.36 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 204433.74 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431582.332044 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839903.63 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 183990.36 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615572.694652 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655913.26 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 165591.33 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781164.021000 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490321.94 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 149032.19 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930196.214713 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341289.74 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 134128.97 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064325.189054 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207160.77 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 120716.08 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185041.265961 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086444.69 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 108644.47 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293685.735178 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977800.22 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 97780.02 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391465.757473 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880020.20 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 88002.02 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479467.777538 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792018.18 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 79201.82 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558669.595597 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712816.36 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 71281.64 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629951.231850 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641534.73 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 64153.47 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694104.704478 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577381.25 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 57738.13 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751842.829843 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519643.13 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 51964.31 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803807.142671 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467678.82 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 46767.88 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850575.024217 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420910.93 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 42091.09 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892666.117608 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378819.84 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 37881.98 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930548.101660 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340937.86 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 34093.79 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964641.887306 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306844.07 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 30684.41 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995326.294388 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276159.66 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 27615.97 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022942.260762 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248543.70 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 24854.37 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047796.630499 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223689.33 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 22368.93 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070165.563262 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201320.39 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 20132.04 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090297.602748 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181188.36 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 18118.84 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108416.438286 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163069.52 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 16306.95 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124723.390270 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146762.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 14676.26 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139399.647056 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132086.31 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 13208.63 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152608.278163 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118877.68 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 11887.77 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164496.046159 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106989.91 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 10698.99 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175195.037356 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96290.92 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 9629.09 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184824.129433 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86661.83 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 8666.18 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193490.312302 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77995.65 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 7799.56 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201289.876885 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70196.08 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 7019.61 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208309.485009 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63176.47 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 6317.65 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214627.132321 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56858.83 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 5685.88 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220313.014901 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51172.94 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 5117.29 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225430.309224 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46055.65 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 4605.56 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230035.874114 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41450.08 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 4145.01 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234180.882516 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37305.08 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3730.51 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237911.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33574.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241511.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29974.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245111.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26374.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248711.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22774.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252311.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19174.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255911.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15574.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259511.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11974.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263111.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8374.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266711.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4774.57 sec) 01:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270311.390077 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174.57 sec) 01:41:00 [GPUGRID] [rr_sim] 2271485.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998951566.46G/439.78G) 01:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850586.87 01:41:00 [---] pick_jobs_to_run() start 01:41:00 [---] pick_jobs_to_run() end 01:41:00 [---] [rr_sim] end 01:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:41:00 [---] [cpu_sched_debug] preliminary job list: 01:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:41:00 [---] [cpu_sched_debug] final job list: 01:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:41:00 [---] [cpu_sched_debug] enforce_run_list: end 01:41:00 [---] choose_project(): 6060.000000 01:41:00 [iThena.Measurements] doing RPC to get work 01:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429539.09 cpu 2172392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:41:00 [---] Got 0 tasks 01:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:42:00 [---] [rr_sim] doing sim: CPU sched 01:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998923976.11G/439.78G) 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271423.22 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 227142.32 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227142.322124 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044280.90 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 204428.09 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431570.412036 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839852.81 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 183985.28 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615555.692957 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655867.53 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 165586.75 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781142.445786 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490280.78 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 149028.08 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930170.523332 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341252.70 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 134125.27 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064295.793123 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207127.43 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 120712.74 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1185008.535935 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086414.69 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 108641.47 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293650.004466 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977773.22 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 97777.32 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391427.326144 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879995.90 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 87999.59 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479426.915654 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791996.31 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 79199.63 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558626.546213 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712796.68 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 71279.67 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629906.213716 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641517.01 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 64151.70 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694057.914469 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577365.31 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 57736.53 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751794.445147 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519628.78 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 51962.88 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803757.322757 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467665.90 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 46766.59 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850523.912605 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420899.31 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 42089.93 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892613.843469 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378809.38 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 37880.94 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930494.781247 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340928.44 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 34092.84 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964587.625247 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306835.60 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 30683.56 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995271.184846 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276152.04 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 27615.20 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022886.388486 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248536.83 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 24853.68 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047740.071762 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223683.15 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 22368.31 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070108.386710 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201314.83 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 20131.48 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090239.870164 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181183.35 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 18118.34 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108358.205272 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163065.02 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 16306.50 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124664.706869 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146758.51 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 14675.85 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139340.558307 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132082.66 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 13208.27 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152548.824601 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118874.40 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 11887.44 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164436.264265 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106986.96 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 10698.70 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175134.959963 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96288.26 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 9628.83 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184763.786091 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86659.44 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 8665.94 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193429.729606 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77993.49 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 7799.35 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201229.078770 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70194.14 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 7019.41 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208248.493018 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63174.73 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 6317.47 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214565.965840 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56857.26 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 5685.73 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220251.691381 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51171.53 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 5117.15 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225368.844367 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46054.38 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 4605.44 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229974.282055 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41448.94 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 4144.89 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234119.175974 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37304.05 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3730.40 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237849.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33573.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241449.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29973.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245049.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26373.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248649.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22773.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252249.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19173.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255849.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15573.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259449.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11973.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263049.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8373.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266649.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4773.64 sec) 01:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270249.580501 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173.64 sec) 01:42:00 [GPUGRID] [rr_sim] 2271423.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998923976.11G/439.78G) 01:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850584.13 01:42:00 [---] pick_jobs_to_run() start 01:42:00 [---] pick_jobs_to_run() end 01:42:00 [---] [rr_sim] end 01:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:42:00 [---] [cpu_sched_debug] preliminary job list: 01:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:42:00 [---] [cpu_sched_debug] final job list: 01:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:42:00 [---] [cpu_sched_debug] enforce_run_list: end 01:42:00 [---] choose_project(): 6120.000000 01:42:00 [iThena.Measurements] doing RPC to get work 01:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429479.09 cpu 2172332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:42:00 [---] Got 0 tasks 01:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:43:00 [---] [rr_sim] doing sim: CPU sched 01:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998896385.75G/439.78G) 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271360.48 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 227136.05 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227136.048436 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044224.44 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 204422.44 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431558.492029 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839801.99 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 183980.20 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615538.691262 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655821.79 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 165582.18 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781120.870572 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490239.61 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 149023.96 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930144.831951 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341215.65 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 134121.57 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064266.397192 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207094.09 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 120709.41 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184975.805909 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086384.68 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 108638.47 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293614.273755 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977746.21 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 97774.62 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391388.894816 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879971.59 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 87997.16 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479386.053770 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791974.43 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 79197.44 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558583.496829 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712776.99 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 71277.70 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629861.195583 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641499.29 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 64149.93 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694011.124461 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577349.36 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 57734.94 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751746.060451 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519614.42 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 51961.44 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803707.502842 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467652.98 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 46765.30 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850472.800994 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420887.68 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 42088.77 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892561.569331 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378798.92 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 37879.89 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930441.460834 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340919.02 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 34091.90 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964533.363187 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306827.12 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 30682.71 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995216.075305 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276144.41 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 27614.44 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022830.516210 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248529.97 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 24853.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047683.513026 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223676.97 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 22367.70 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070051.210159 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201309.27 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 20130.93 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090182.137580 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181178.35 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 18117.83 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108299.972258 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163060.51 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 16306.05 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124606.023468 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146754.46 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 14675.45 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139281.469558 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132079.01 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 13207.90 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152489.371038 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118871.11 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 11887.11 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164376.482371 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106984.00 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 10698.40 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175074.882570 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96285.60 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 9628.56 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184703.442749 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86657.04 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 8665.70 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193369.146910 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77991.34 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 7799.13 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201168.280656 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70192.20 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 7019.22 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208187.501026 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63172.98 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 6317.30 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214504.799360 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56855.69 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 5685.57 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220190.367860 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51170.12 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 5117.01 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225307.379510 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46053.10 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 4605.31 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229912.689996 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41447.79 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 4144.78 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234057.469432 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37303.01 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3730.30 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237787.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33572.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241387.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29972.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244987.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26372.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248587.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22772.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252187.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19172.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255787.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15572.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259387.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11972.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262987.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8372.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266587.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4772.71 sec) 01:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270187.770925 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172.71 sec) 01:43:00 [GPUGRID] [rr_sim] 2271360.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998896385.75G/439.78G) 01:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850581.40 01:43:00 [---] pick_jobs_to_run() start 01:43:00 [---] pick_jobs_to_run() end 01:43:00 [---] [rr_sim] end 01:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:43:00 [---] [cpu_sched_debug] preliminary job list: 01:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:43:00 [---] [cpu_sched_debug] final job list: 01:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:43:00 [---] [cpu_sched_debug] enforce_run_list: end 01:43:00 [---] choose_project(): 6180.000000 01:43:00 [iThena.Measurements] doing RPC to get work 01:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429419.09 cpu 2172272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:43:00 [---] Got 0 tasks 01:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:44:00 [---] [rr_sim] doing sim: CPU sched 01:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998868795.40G/439.78G) 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271297.75 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 227129.77 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227129.774748 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044167.97 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 204416.80 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431546.572021 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839751.18 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 183975.12 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615521.689567 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655776.06 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 165577.61 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781099.295359 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490198.45 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 149019.85 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930119.140571 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341178.61 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 134117.86 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064237.001262 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207060.75 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 120706.07 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184943.075884 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086354.67 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 108635.47 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293578.543043 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977719.20 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 97771.92 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391350.463487 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879947.28 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 87994.73 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479345.191886 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791952.56 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 79195.26 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558540.447446 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712757.30 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 71275.73 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629816.177449 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641481.57 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 64148.16 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693964.334452 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577333.41 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 57733.34 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751697.675755 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519600.07 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 51960.01 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803657.682928 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467640.06 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 46764.01 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850421.689383 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420876.06 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 42087.61 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892509.295193 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378788.45 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 37878.85 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930388.140421 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340909.61 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 34090.96 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964479.101127 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306818.65 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 30681.86 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995160.965763 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276136.78 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 27613.68 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022774.643934 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248523.10 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 24852.31 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047626.954289 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223670.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 22367.08 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069994.033608 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201303.71 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 20130.37 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090124.404995 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181173.34 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 18117.33 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108241.739244 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163056.01 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 16305.60 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124547.340068 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146750.41 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 14675.04 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139222.380809 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132075.37 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 13207.54 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152429.917476 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118867.83 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 11886.78 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164316.700476 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106981.05 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 10698.10 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175014.805177 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96282.94 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 9628.29 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184643.099407 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86654.65 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 8665.46 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193308.564214 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77989.18 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 7798.92 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201107.482541 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70190.26 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 7019.03 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208126.509035 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63171.24 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 6317.12 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214443.632880 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56854.11 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 5685.41 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220129.044340 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51168.70 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 5116.87 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225245.914654 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46051.83 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 4605.18 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229851.097936 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41446.65 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 4144.66 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233995.762891 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37301.98 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3730.20 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237725.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33571.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241325.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29971.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244925.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26371.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248525.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22771.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252125.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19171.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255725.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15571.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259325.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11971.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262925.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8371.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266525.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4771.79 sec) 01:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270125.961350 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171.79 sec) 01:44:00 [GPUGRID] [rr_sim] 2271297.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998868795.40G/439.78G) 01:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850578.66 01:44:00 [---] pick_jobs_to_run() start 01:44:00 [---] pick_jobs_to_run() end 01:44:00 [---] [rr_sim] end 01:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:44:00 [---] [cpu_sched_debug] preliminary job list: 01:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:44:00 [---] [cpu_sched_debug] final job list: 01:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:44:00 [---] [cpu_sched_debug] enforce_run_list: end 01:44:00 [---] choose_project(): 6240.000000 01:44:00 [iThena.Measurements] doing RPC to get work 01:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429359.09 cpu 2172212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:44:00 [---] Got 0 tasks 01:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:45:00 [---] [rr_sim] doing sim: CPU sched 01:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998841205.04G/439.78G) 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271235.01 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 227123.50 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227123.501060 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044111.51 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 204411.15 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431534.652014 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839700.36 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 183970.04 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615504.687872 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655730.32 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 165573.03 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781077.720145 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490157.29 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 149015.73 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930093.449190 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341141.56 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 134114.16 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064207.605331 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1207027.41 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 120702.74 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184910.345858 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086324.66 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 108632.47 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293542.812332 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977692.20 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 97769.22 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391312.032158 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879922.98 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 87992.30 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479304.330002 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791930.68 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 79193.07 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558497.398062 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712737.61 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 71273.76 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629771.159316 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641463.85 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 64146.39 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693917.544444 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577317.47 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 57731.75 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751649.291059 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519585.72 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 51958.57 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803607.863013 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467627.15 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 46762.71 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850370.577772 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420864.43 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 42086.44 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892457.021054 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378777.99 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 37877.80 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930334.820009 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340900.19 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 34090.02 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964424.839068 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306810.17 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 30681.02 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995105.856221 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276129.15 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 27612.92 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022718.771658 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248516.24 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 24851.62 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047570.395552 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223664.62 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 22366.46 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069936.857057 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201298.15 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 20129.82 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090066.672411 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181168.34 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 18116.83 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108183.506230 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163051.50 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 16305.15 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124488.656667 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146746.35 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 14674.64 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139163.292060 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132071.72 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 13207.17 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152370.463914 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118864.55 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 11886.45 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164256.918582 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106978.09 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 10697.81 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174954.727784 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96280.28 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 9628.03 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184582.756065 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86652.25 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 8665.23 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193247.981518 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77987.03 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 7798.70 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201046.684426 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70188.33 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 7018.83 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208065.517044 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63169.49 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 6316.95 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214382.466399 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56852.54 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 5685.25 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220067.720819 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51167.29 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 5116.73 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225184.449797 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46050.56 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 4605.06 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229789.505877 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41445.50 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 4144.55 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233934.056349 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37300.95 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3730.10 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237664.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33570.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241264.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29970.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244864.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26370.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248464.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22770.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252064.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19170.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255664.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15570.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259264.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11970.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262864.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8370.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266464.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4770.86 sec) 01:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270064.151774 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170.86 sec) 01:45:00 [GPUGRID] [rr_sim] 2271235.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998841205.04G/439.78G) 01:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850575.92 01:45:00 [---] pick_jobs_to_run() start 01:45:00 [---] pick_jobs_to_run() end 01:45:00 [---] [rr_sim] end 01:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:45:00 [---] [cpu_sched_debug] preliminary job list: 01:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:45:00 [---] [cpu_sched_debug] final job list: 01:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:45:00 [---] [cpu_sched_debug] enforce_run_list: end 01:45:00 [---] choose_project(): 6300.000000 01:45:00 [iThena.Measurements] doing RPC to get work 01:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429299.09 cpu 2172152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:45:00 [---] Got 0 tasks 01:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:46:00 [---] [rr_sim] doing sim: CPU sched 01:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998813614.68G/439.78G) 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271172.27 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 227117.23 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227117.227372 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2044055.05 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 204405.50 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431522.732006 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839649.54 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 183964.95 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615487.686177 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655684.59 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 165568.46 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781056.144931 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490116.13 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 149011.61 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930067.757809 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341104.52 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 134110.45 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064178.209400 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206994.06 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 120699.41 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184877.615832 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086294.66 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 108629.47 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293507.081620 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977665.19 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 97766.52 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391273.600830 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879898.67 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 87989.87 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479263.468118 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791908.81 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 79190.88 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558454.348678 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712717.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 71271.79 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629726.141182 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641446.13 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 64144.61 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693870.754435 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577301.52 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 57730.15 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751600.906363 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519571.37 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 51957.14 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803558.043099 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467614.23 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 46761.42 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850319.466160 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420852.81 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 42085.28 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892404.746916 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378767.53 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 37876.75 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930281.499596 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340890.77 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 34089.08 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964370.577008 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306801.70 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 30680.17 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995050.746679 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276121.53 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 27612.15 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022662.899383 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248509.37 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 24850.94 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047513.836816 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223658.44 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 22365.84 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069879.680506 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201292.59 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 20129.26 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090008.939827 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181163.33 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 18116.33 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108125.273216 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163047.00 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 16304.70 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124429.973266 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146742.30 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 14674.23 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139104.203311 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132068.07 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 13206.81 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152311.010351 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118861.26 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 11886.13 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164197.136688 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106975.14 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 10697.51 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174894.650391 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96277.62 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 9627.76 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184522.412723 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86649.86 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 8664.99 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193187.398823 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77984.87 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 7798.49 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200985.886312 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70186.39 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 7018.64 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208004.525052 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63167.75 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 6316.77 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214321.299919 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56850.97 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 5685.10 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220006.397298 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51165.88 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 5116.59 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225122.984940 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46049.29 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 4604.93 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229727.913818 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41444.36 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 4144.44 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233872.349808 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37299.92 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3729.99 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237602.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33569.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241202.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29969.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244802.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26369.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248402.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22769.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252002.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19169.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255602.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15569.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259202.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11969.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262802.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8369.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266402.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4769.93 sec) 01:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2270002.342198 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169.93 sec) 01:46:00 [GPUGRID] [rr_sim] 2271172.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998813614.68G/439.78G) 01:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850573.19 01:46:00 [---] pick_jobs_to_run() start 01:46:00 [---] pick_jobs_to_run() end 01:46:00 [---] [rr_sim] end 01:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:46:00 [---] [cpu_sched_debug] preliminary job list: 01:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:46:00 [---] [cpu_sched_debug] final job list: 01:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:46:00 [---] [cpu_sched_debug] enforce_run_list: end 01:46:00 [---] choose_project(): 6360.000000 01:46:00 [iThena.Measurements] doing RPC to get work 01:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 602039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429239.09 cpu 2172092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:46:00 [---] Got 0 tasks 01:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:47:00 [---] [rr_sim] doing sim: CPU sched 01:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998786024.33G/439.78G) 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271109.54 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 227110.95 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227110.953683 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043998.58 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 204399.86 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431510.811998 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839598.72 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 183959.87 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615470.684482 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655638.85 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 165563.89 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781034.569717 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490074.97 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 149007.50 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930042.066429 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341067.47 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 134106.75 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064148.813469 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206960.72 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 120696.07 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184844.885806 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086264.65 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 108626.47 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293471.350909 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977638.19 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 97763.82 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391235.169501 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879874.37 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 87987.44 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479222.606234 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791886.93 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 79188.69 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558411.299294 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712698.24 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 71269.82 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629681.123048 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641428.41 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 64142.84 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693823.964427 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577285.57 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 57728.56 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751552.521668 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519557.02 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 51955.70 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803508.223184 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467601.31 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 46760.13 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850268.354549 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420841.18 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 42084.12 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892352.472778 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378757.06 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 37875.71 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930228.179183 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340881.36 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 34088.14 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964316.314948 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306793.22 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 30679.32 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994995.637137 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276113.90 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 27611.39 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022607.027107 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248502.51 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 24850.25 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047457.278079 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223652.26 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 22365.23 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069822.503955 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201287.03 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 20128.70 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089951.207243 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181158.33 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 18115.83 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108067.040202 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163042.50 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 16304.25 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124371.289865 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146738.25 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 14673.82 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139045.114562 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132064.42 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 13206.44 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152251.556789 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118857.98 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 11885.80 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164137.354794 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106972.18 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 10697.22 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174834.572998 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96274.96 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 9627.50 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184462.069381 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86647.47 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 8664.75 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193126.816127 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77982.72 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 7798.27 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200925.088197 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70184.45 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 7018.44 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207943.533061 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63166.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 6316.60 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214260.133438 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56849.40 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 5684.94 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219945.073778 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51164.46 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 5116.45 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225061.520083 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46048.02 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 4604.80 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229666.321758 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41443.22 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 4144.32 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233810.643266 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37298.89 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3729.89 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237540.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33569.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241140.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29969.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244740.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26369.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248340.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22769.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251940.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19169.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255540.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15569.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259140.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11969.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262740.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8369.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266340.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4769.00 sec) 01:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269940.532623 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169.00 sec) 01:47:00 [GPUGRID] [rr_sim] 2271109.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998786024.33G/439.78G) 01:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850570.45 01:47:00 [---] pick_jobs_to_run() start 01:47:00 [---] pick_jobs_to_run() end 01:47:00 [---] [rr_sim] end 01:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:47:00 [---] [cpu_sched_debug] preliminary job list: 01:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:47:00 [---] [cpu_sched_debug] final job list: 01:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:47:00 [---] [cpu_sched_debug] enforce_run_list: end 01:47:00 [---] choose_project(): 6420.000000 01:47:00 [iThena.Measurements] doing RPC to get work 01:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429179.09 cpu 2172032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2172032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2172032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:47:00 [---] Got 0 tasks 01:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:48:00 [---] [rr_sim] doing sim: CPU sched 01:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998758433.97G/439.78G) 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2271046.80 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 227104.68 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227104.679995 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043942.12 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 204394.21 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431498.891991 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839547.91 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 183954.79 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615453.682787 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655593.12 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 165559.31 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 781012.994503 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1490033.81 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 149003.38 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 930016.375048 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1341030.42 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 134103.04 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064119.417539 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206927.38 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 120692.74 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184812.155780 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086234.64 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 108623.46 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293435.620197 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977611.18 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 97761.12 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391196.738173 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879850.06 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 87985.01 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479181.744350 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791865.06 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 79186.51 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558368.249911 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712678.55 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 71267.86 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629636.104915 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641410.70 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 64141.07 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693777.174418 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577269.63 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 57726.96 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751504.136972 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519542.66 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 51954.27 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803458.403270 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467588.40 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 46758.84 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850217.242938 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420829.56 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 42082.96 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892300.198639 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378746.60 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 37874.66 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930174.858771 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340871.94 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 34087.19 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964262.052889 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306784.75 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 30678.47 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994940.527595 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276106.27 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 27610.63 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022551.154831 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248495.65 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 24849.56 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047400.719343 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223646.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 22364.61 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069765.327404 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201281.47 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 20128.15 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089893.474658 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181153.33 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 18115.33 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108008.807188 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163037.99 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 16303.80 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124312.606464 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146734.19 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 14673.42 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138986.025813 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132060.77 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 13206.08 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152192.103227 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118854.70 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 11885.47 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164077.572899 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106969.23 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 10696.92 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174774.495605 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96272.30 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 9627.23 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184401.726039 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86645.07 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 8664.51 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193066.233431 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77980.57 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 7798.06 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200864.290083 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70182.51 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 7018.25 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207882.541070 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63164.26 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 6316.43 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214198.966958 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56847.83 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 5684.78 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219883.750257 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51163.05 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 5116.30 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225000.055227 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46046.74 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 4604.67 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229604.729699 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41442.07 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 4144.21 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233748.936724 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37297.86 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3729.79 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237478.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33568.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241078.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29968.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244678.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26368.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248278.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22768.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251878.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19168.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255478.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15568.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259078.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11968.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262678.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8368.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266278.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4768.08 sec) 01:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269878.723047 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168.08 sec) 01:48:00 [GPUGRID] [rr_sim] 2271046.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998758433.97G/439.78G) 01:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850567.71 01:48:00 [---] pick_jobs_to_run() start 01:48:00 [---] pick_jobs_to_run() end 01:48:00 [---] [rr_sim] end 01:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:48:00 [---] [cpu_sched_debug] preliminary job list: 01:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:48:00 [---] [cpu_sched_debug] final job list: 01:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:48:00 [---] [cpu_sched_debug] enforce_run_list: end 01:48:00 [---] choose_project(): 6480.000000 01:48:00 [iThena.Measurements] doing RPC to get work 01:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429119.09 cpu 2171972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:48:00 [---] Got 0 tasks 01:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:49:00 [---] [rr_sim] doing sim: CPU sched 01:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998730843.61G/439.78G) 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270984.06 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 227098.41 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227098.406307 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043885.66 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 204388.57 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431486.971983 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839497.09 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 183949.71 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615436.681092 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655547.38 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 165554.74 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780991.419290 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489992.64 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 148999.26 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929990.683668 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340993.38 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 134099.34 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064090.021608 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206894.04 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 120689.40 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184779.425754 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086204.64 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 108620.46 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293399.889486 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977584.17 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 97758.42 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391158.306844 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879825.76 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 87982.58 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479140.882467 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791843.18 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 79184.32 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558325.200527 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712658.86 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 71265.89 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629591.086781 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641392.98 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 64139.30 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693730.384410 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577253.68 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 57725.37 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751455.752276 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519528.31 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 51952.83 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803408.583355 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467575.48 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 46757.55 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850166.131327 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420817.93 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 42081.79 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892247.924501 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378736.14 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 37873.61 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930121.538358 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340862.52 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 34086.25 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964207.790829 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306776.27 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 30677.63 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994885.418053 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276098.65 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 27609.86 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022495.282555 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248488.78 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 24848.88 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047344.160606 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223639.90 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 22363.99 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069708.150853 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201275.91 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 20127.59 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089835.742074 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181148.32 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 18114.83 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107950.574174 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163033.49 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 16303.35 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124253.923063 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146730.14 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 14673.01 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138926.937064 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132057.13 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 13205.71 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152132.649665 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118851.41 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 11885.14 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164017.791005 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106966.27 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 10696.63 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174714.418211 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96269.64 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 9626.96 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184341.382697 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86642.68 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 8664.27 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193005.650735 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77978.41 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 7797.84 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200803.491968 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70180.57 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 7018.06 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207821.549078 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63162.51 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 6316.25 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214137.800477 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56846.26 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 5684.63 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219822.426737 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51161.64 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 5116.16 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224938.590370 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46045.47 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 4604.55 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229543.137640 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41440.93 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 4144.09 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233687.230183 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37296.83 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3729.68 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237416.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33567.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241016.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29967.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244616.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26367.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248216.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22767.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251816.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19167.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255416.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15567.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259016.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11967.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262616.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8367.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266216.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4767.15 sec) 01:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269816.913471 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167.15 sec) 01:49:00 [GPUGRID] [rr_sim] 2270984.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998730843.61G/439.78G) 01:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850564.98 01:49:00 [---] pick_jobs_to_run() start 01:49:00 [---] pick_jobs_to_run() end 01:49:00 [---] [rr_sim] end 01:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:49:00 [---] [cpu_sched_debug] preliminary job list: 01:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:49:00 [---] [cpu_sched_debug] final job list: 01:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:49:00 [---] [cpu_sched_debug] enforce_run_list: end 01:49:00 [---] choose_project(): 6540.000000 01:49:00 [iThena.Measurements] doing RPC to get work 01:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 429059.09 cpu 2171912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:49:00 [---] Got 0 tasks 01:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:50:00 [---] [rr_sim] doing sim: CPU sched 01:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998703253.26G/439.78G) 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270921.33 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 227092.13 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227092.132619 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043829.19 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 204382.92 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431475.051976 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839446.27 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 183944.63 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615419.679397 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655501.65 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 165550.16 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780969.844076 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489951.48 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 148995.15 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929964.992287 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340956.33 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 134095.63 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064060.625677 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206860.70 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 120686.07 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184746.695728 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086174.63 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 108617.46 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293364.158774 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977557.17 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 97755.72 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391119.875515 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879801.45 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 87980.15 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479100.020583 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791821.31 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 79182.13 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558282.151143 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712639.18 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 71263.92 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629546.068647 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641375.26 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 64137.53 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693683.594401 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577237.73 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 57723.77 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751407.367580 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519513.96 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 51951.40 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803358.763441 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467562.56 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 46756.26 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850115.019715 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420806.31 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 42080.63 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892195.650363 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378725.68 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 37872.57 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930068.217945 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340853.11 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 34085.31 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964153.528769 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306767.80 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 30676.78 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994830.308511 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276091.02 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 27609.10 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022439.410279 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248481.92 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 24848.19 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047287.601870 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223633.72 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 22363.37 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069650.974301 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201270.35 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 20127.04 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089778.009490 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181143.32 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 18114.33 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107892.341160 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163028.99 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 16302.90 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124195.239663 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146726.09 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 14672.61 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138867.848315 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132053.48 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 13205.35 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152073.196102 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118848.13 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 11884.81 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163958.009111 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106963.32 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 10696.33 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174654.340818 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96266.99 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 9626.70 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184281.039355 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86640.29 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 8664.03 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192945.068039 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77976.26 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 7797.63 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200742.693853 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70178.63 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 7017.86 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207760.557087 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63160.77 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 6316.08 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214076.633997 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56844.69 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 5684.47 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219761.103216 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51160.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 5116.02 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224877.125513 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46044.20 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 4604.42 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229481.545581 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41439.78 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 4143.98 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233625.523641 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37295.80 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3729.58 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237355.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33566.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240955.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29966.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244555.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26366.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248155.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22766.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251755.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19166.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255355.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15566.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258955.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11966.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262555.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8366.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266155.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4766.22 sec) 01:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269755.103896 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166.22 sec) 01:50:00 [GPUGRID] [rr_sim] 2270921.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998703253.26G/439.78G) 01:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850562.24 01:50:00 [---] pick_jobs_to_run() start 01:50:00 [---] pick_jobs_to_run() end 01:50:00 [---] [rr_sim] end 01:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:50:00 [---] [cpu_sched_debug] preliminary job list: 01:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:50:00 [---] [cpu_sched_debug] final job list: 01:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:50:00 [---] [cpu_sched_debug] enforce_run_list: end 01:50:00 [---] choose_project(): 6600.000000 01:50:00 [iThena.Measurements] doing RPC to get work 01:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428999.09 cpu 2171852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:50:00 [---] Got 0 tasks 01:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:51:00 [---] [rr_sim] doing sim: CPU sched 01:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998675662.90G/439.78G) 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270858.59 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 227085.86 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227085.858931 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043772.73 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 204377.27 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431463.131968 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839395.46 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 183939.55 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615402.677702 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655455.91 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 165545.59 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780948.268862 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489910.32 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 148991.03 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929939.300906 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340919.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 134091.93 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064031.229746 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206827.36 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 120682.74 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184713.965702 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086144.62 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 108614.46 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293328.428062 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977530.16 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 97753.02 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391081.444187 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879777.15 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 87977.71 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479059.158699 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791799.43 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 79179.94 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558239.101759 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712619.49 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 71261.95 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629501.050514 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641357.54 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 64135.75 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693636.804393 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577221.78 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 57722.18 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751358.982884 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519499.61 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 51949.96 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803308.943526 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467549.65 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 46754.96 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850063.908104 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420794.68 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 42079.47 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892143.376224 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378715.21 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 37871.52 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930014.897532 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340843.69 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 34084.37 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964099.266710 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306759.32 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 30675.93 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994775.198969 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276083.39 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 27608.34 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022383.538003 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248475.05 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 24847.51 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047231.043133 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223627.55 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 22362.75 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069593.797750 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201264.79 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 20126.48 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089720.276906 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181138.31 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 18113.83 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107834.108146 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163024.48 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 16302.45 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124136.556262 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146722.03 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 14672.20 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138808.759566 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132049.83 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 13204.98 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152013.742540 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118844.85 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 11884.48 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163898.227216 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106960.36 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 10696.04 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174594.263425 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96264.33 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 9626.43 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184220.696013 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86637.89 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 8663.79 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192884.485343 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77974.10 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 7797.41 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200681.895739 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70176.69 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 7017.67 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207699.565095 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63159.02 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 6315.90 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214015.467516 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56843.12 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 5684.31 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219699.779695 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51158.81 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 5115.88 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224815.660656 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46042.93 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 4604.29 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229419.953521 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41438.64 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 4143.86 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233563.817100 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37294.77 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3729.48 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237293.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33565.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240893.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29965.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244493.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26365.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248093.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22765.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251693.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19165.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255293.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15565.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258893.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11965.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262493.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8365.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266093.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4765.29 sec) 01:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269693.294320 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165.29 sec) 01:51:00 [GPUGRID] [rr_sim] 2270858.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998675662.90G/439.78G) 01:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850559.50 01:51:00 [---] pick_jobs_to_run() start 01:51:00 [---] pick_jobs_to_run() end 01:51:00 [---] [rr_sim] end 01:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:51:00 [---] [cpu_sched_debug] preliminary job list: 01:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:51:00 [---] [cpu_sched_debug] final job list: 01:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:51:00 [---] [cpu_sched_debug] enforce_run_list: end 01:51:00 [---] choose_project(): 6660.000000 01:51:00 [iThena.Measurements] doing RPC to get work 01:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428939.09 cpu 2171792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:51:00 [---] Got 0 tasks 01:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:52:00 [---] [rr_sim] doing sim: CPU sched 01:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998648072.55G/439.78G) 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270795.85 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 227079.59 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227079.585242 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043716.27 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 204371.63 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431451.211960 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839344.64 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 183934.46 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615385.676007 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655410.18 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 165541.02 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780926.693648 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489869.16 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 148986.92 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929913.609526 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340882.24 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 134088.22 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1064001.833816 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206794.02 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 120679.40 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184681.235676 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086114.62 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 108611.46 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293292.697351 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977503.16 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 97750.32 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391043.012858 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879752.84 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 87975.28 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1479018.296815 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791777.56 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 79177.76 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558196.052375 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712599.80 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 71259.98 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629456.032380 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641339.82 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 64133.98 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693590.014385 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577205.84 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 57720.58 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751310.598188 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519485.25 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 51948.53 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803259.123612 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467536.73 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 46753.67 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850012.796493 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420783.06 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 42078.31 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892091.102086 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378704.75 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 37870.48 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929961.577120 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340834.28 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 34083.43 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964045.004650 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306750.85 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 30675.08 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994720.089427 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276075.76 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 27607.58 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022327.665727 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248468.19 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 24846.82 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047174.484397 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223621.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 22362.14 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069536.621199 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201259.23 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 20125.92 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089662.544322 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181133.31 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 18113.33 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107775.875132 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163019.98 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 16302.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124077.872861 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146717.98 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 14671.80 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138749.670817 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132046.18 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 13204.62 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151954.288978 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118841.56 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 11884.16 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163838.445322 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106957.41 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 10695.74 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174534.186032 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96261.67 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 9626.17 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184160.352671 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86635.50 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 8663.55 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192823.902647 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77971.95 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 7797.19 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200621.097624 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70174.75 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 7017.48 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207638.573104 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63157.28 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 6315.73 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213954.301036 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56841.55 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 5684.16 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219638.456175 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51157.40 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 5115.74 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224754.195800 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46041.66 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 4604.17 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229358.361462 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41437.49 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 4143.75 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233502.110558 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37293.74 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3729.37 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237231.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33564.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240831.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29964.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244431.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26364.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248031.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22764.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251631.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19164.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255231.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15564.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258831.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11964.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262431.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8364.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266031.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4764.37 sec) 01:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269631.484745 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164.37 sec) 01:52:00 [GPUGRID] [rr_sim] 2270795.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998648072.55G/439.78G) 01:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850556.77 01:52:00 [---] pick_jobs_to_run() start 01:52:00 [---] pick_jobs_to_run() end 01:52:00 [---] [rr_sim] end 01:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:52:00 [---] [cpu_sched_debug] preliminary job list: 01:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:52:00 [---] [cpu_sched_debug] final job list: 01:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:52:00 [---] [cpu_sched_debug] enforce_run_list: end 01:52:00 [---] choose_project(): 6720.000000 01:52:00 [iThena.Measurements] doing RPC to get work 01:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428879.09 cpu 2171732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:52:00 [---] Got 0 tasks 01:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:53:00 [---] [rr_sim] doing sim: CPU sched 01:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998620482.19G/439.78G) 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270733.12 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 227073.31 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227073.311554 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043659.80 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 204365.98 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431439.291953 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839293.82 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 183929.38 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615368.674312 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655364.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 165536.44 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780905.118435 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489828.00 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 148982.80 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929887.918145 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340845.20 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 134084.52 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063972.437885 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206760.68 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 120676.07 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184648.505650 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086084.61 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 108608.46 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293256.966639 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977476.15 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 97747.61 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1391004.581530 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879728.53 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 87972.85 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478977.434931 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791755.68 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 79175.57 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558153.002992 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712580.11 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 71258.01 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629411.014247 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641322.10 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 64132.21 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693543.224376 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577189.89 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 57718.99 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751262.213493 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519470.90 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 51947.09 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803209.303697 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467523.81 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 46752.38 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849961.684882 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420771.43 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 42077.14 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892038.827948 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378694.29 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 37869.43 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929908.256707 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340824.86 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 34082.49 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963990.742590 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306742.37 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 30674.24 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994664.979885 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276068.14 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 27606.81 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022271.793451 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248461.32 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 24846.13 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047117.925660 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223615.19 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 22361.52 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069479.444648 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201253.67 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 20125.37 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089604.811737 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181128.30 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 18112.83 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107717.642118 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163015.47 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 16301.55 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124019.189460 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146713.93 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 14671.39 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138690.582068 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132042.53 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 13204.25 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151894.835415 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118838.28 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 11883.83 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163778.663428 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106954.45 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 10695.45 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174474.108639 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96259.01 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 9625.90 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184100.009329 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86633.11 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 8663.31 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192763.319951 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77969.80 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 7796.98 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200560.299510 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70172.82 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 7017.28 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207577.581113 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63155.53 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 6315.55 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213893.134556 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56839.98 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 5684.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219577.132654 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51155.98 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 5115.60 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224692.730943 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46040.38 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 4604.04 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229296.769403 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41436.35 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 4143.63 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233440.404016 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37292.71 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3729.27 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237169.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33563.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240769.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29963.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244369.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26363.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247969.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22763.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251569.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19163.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255169.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15563.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258769.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11963.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262369.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8363.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265969.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4763.44 sec) 01:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269569.675169 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1163.44 sec) 01:53:00 [GPUGRID] [rr_sim] 2270733.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998620482.19G/439.78G) 01:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850554.03 01:53:00 [---] pick_jobs_to_run() start 01:53:00 [---] pick_jobs_to_run() end 01:53:00 [---] [rr_sim] end 01:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:53:00 [---] [cpu_sched_debug] preliminary job list: 01:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:53:00 [---] [cpu_sched_debug] final job list: 01:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:53:00 [---] [cpu_sched_debug] enforce_run_list: end 01:53:00 [---] choose_project(): 6780.000000 01:53:00 [iThena.Measurements] doing RPC to get work 01:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428819.09 cpu 2171672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:53:00 [---] Got 0 tasks 01:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:54:00 [---] [rr_sim] doing sim: CPU sched 01:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998592891.83G/439.78G) 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270670.38 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 227067.04 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227067.037866 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043603.34 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 204360.33 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431427.371945 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839243.01 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 183924.30 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615351.672617 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655318.71 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 165531.87 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780883.543221 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489786.84 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 148978.68 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929862.226765 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340808.15 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 134080.82 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063943.041954 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206727.34 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 120672.73 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184615.775624 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086054.60 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 108605.46 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293221.235928 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977449.14 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 97744.91 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390966.150201 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879704.23 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 87970.42 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478936.573047 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791733.81 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 79173.38 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558109.953608 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712560.43 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 71256.04 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629365.996113 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641304.38 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 64130.44 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693496.434368 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577173.94 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 57717.39 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751213.828797 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519456.55 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 51945.65 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803159.483783 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467510.89 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 46751.09 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849910.573270 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420759.81 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 42075.98 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891986.553809 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378683.82 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 37868.38 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929854.936294 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340815.44 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 34081.54 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963936.480531 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306733.90 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 30673.39 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994609.870344 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276060.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 27606.05 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022215.921175 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248454.46 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 24845.45 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047061.366923 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223609.01 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 22360.90 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069422.268097 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201248.11 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 20124.81 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089547.079153 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181123.30 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 18112.33 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107659.409104 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163010.97 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 16301.10 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123960.506059 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146709.87 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 14670.99 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138631.493319 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132038.89 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 13203.89 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151835.381853 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118835.00 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 11883.50 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163718.881534 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106951.50 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 10695.15 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174414.031246 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96256.35 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 9625.63 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184039.665987 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86630.71 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 8663.07 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192702.737255 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77967.64 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 7796.76 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200499.501395 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70170.88 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 7017.09 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207516.589121 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63153.79 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 6315.38 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213831.968075 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56838.41 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 5683.84 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219515.809133 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51154.57 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 5115.46 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224631.266086 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46039.11 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 4603.91 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229235.177343 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41435.20 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 4143.52 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233378.697475 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37291.68 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3729.17 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237107.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33562.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240707.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29962.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244307.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26362.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247907.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22762.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251507.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19162.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255107.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15562.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258707.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11962.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262307.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8362.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265907.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4762.51 sec) 01:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269507.865593 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1162.51 sec) 01:54:00 [GPUGRID] [rr_sim] 2270670.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998592891.83G/439.78G) 01:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850551.29 01:54:00 [---] pick_jobs_to_run() start 01:54:00 [---] pick_jobs_to_run() end 01:54:00 [---] [rr_sim] end 01:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:54:00 [---] [cpu_sched_debug] preliminary job list: 01:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:54:00 [---] [cpu_sched_debug] final job list: 01:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:54:00 [---] [cpu_sched_debug] enforce_run_list: end 01:54:00 [---] choose_project(): 6840.000000 01:54:00 [iThena.Measurements] doing RPC to get work 01:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428759.09 cpu 2171612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:54:00 [---] Got 0 tasks 01:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:55:00 [---] [rr_sim] doing sim: CPU sched 01:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998565301.48G/439.78G) 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270607.64 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 227060.76 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227060.764178 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043546.88 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 204354.69 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431415.451938 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839192.19 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 183919.22 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615334.670921 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655272.97 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 165527.30 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780861.968007 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489745.67 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 148974.57 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929836.535384 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340771.11 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 134077.11 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063913.646023 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206694.00 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 120669.40 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184583.045599 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086024.60 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 108602.46 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293185.505216 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977422.14 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 97742.21 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390927.718872 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879679.92 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 87967.99 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478895.711163 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791711.93 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 79171.19 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558066.904224 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712540.74 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 71254.07 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629320.977979 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641286.66 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 64128.67 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693449.644359 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577158.00 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 57715.80 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751165.444101 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519442.20 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 51944.22 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803109.663868 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467497.98 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 46749.80 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849859.461659 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420748.18 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 42074.82 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891934.279671 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378673.36 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 37867.34 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929801.615882 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340806.03 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 34080.60 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963882.218471 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306725.42 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 30672.54 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994554.760802 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276052.88 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 27605.29 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022160.048899 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248447.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 24844.76 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047004.808187 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223602.83 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 22360.28 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069365.091546 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201242.55 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 20124.26 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089489.346569 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181118.30 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 18111.83 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107601.176090 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163006.47 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 16300.65 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123901.822658 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146705.82 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 14670.58 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138572.404570 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132035.24 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 13203.52 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151775.928291 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118831.71 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 11883.17 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163659.099639 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106948.54 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 10694.85 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174353.953853 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96253.69 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 9625.37 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183979.322645 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86628.32 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 8662.83 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192642.154559 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77965.49 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 7796.55 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200438.703280 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70168.94 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 7016.89 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207455.597130 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63152.04 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 6315.20 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213770.801595 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56836.84 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 5683.68 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219454.485613 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51153.16 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 5115.32 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224569.801229 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46037.84 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 4603.78 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229173.585284 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41434.06 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 4143.41 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233316.990933 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37290.65 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3729.07 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237046.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33561.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240646.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29961.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244246.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26361.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247846.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22761.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251446.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19161.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255046.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15561.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258646.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11961.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262246.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8361.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265846.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4761.59 sec) 01:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269446.056018 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1161.59 sec) 01:55:00 [GPUGRID] [rr_sim] 2270607.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998565301.48G/439.78G) 01:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850548.55 01:55:00 [---] pick_jobs_to_run() start 01:55:00 [---] pick_jobs_to_run() end 01:55:00 [---] [rr_sim] end 01:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:55:00 [---] [cpu_sched_debug] preliminary job list: 01:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:55:00 [---] [cpu_sched_debug] final job list: 01:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:55:00 [---] [cpu_sched_debug] enforce_run_list: end 01:55:00 [---] choose_project(): 6900.000000 01:55:00 [iThena.Measurements] doing RPC to get work 01:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428699.09 cpu 2171552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:55:00 [---] Got 0 tasks 01:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:56:00 [---] [rr_sim] doing sim: CPU sched 01:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998537711.12G/439.78G) 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270544.90 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 227054.49 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227054.490489 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043490.41 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 204349.04 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431403.531930 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839141.37 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 183914.14 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615317.669226 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655227.24 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 165522.72 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780840.392793 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489704.51 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 148970.45 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929810.844003 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340734.06 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 134073.41 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063884.250092 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206660.65 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 120666.07 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184550.315573 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085994.59 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 108599.46 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293149.774505 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977395.13 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 97739.51 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390889.287544 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879655.62 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 87965.56 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478854.849279 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791690.06 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 79169.01 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1558023.854840 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712521.05 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 71252.11 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629275.959846 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641268.95 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 64126.89 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693402.854351 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577142.05 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 57714.21 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751117.059405 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519427.85 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 51942.78 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803059.843954 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467485.06 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 46748.51 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849808.350048 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420736.55 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 42073.66 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891882.005533 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378662.90 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 37866.29 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929748.295469 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340796.61 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 34079.66 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963827.956411 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306716.95 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 30671.69 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994499.651260 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276045.25 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 27604.53 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022104.176623 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248440.73 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 24844.07 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046948.249450 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223596.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 22359.67 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069307.914995 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201236.99 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 20123.70 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089431.613985 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181113.29 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 18111.33 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107542.943076 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163001.96 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 16300.20 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123843.139258 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146701.77 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 14670.18 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138513.315821 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132031.59 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 13203.16 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151716.474729 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118828.43 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 11882.84 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163599.317745 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106945.59 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 10694.56 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174293.876460 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96251.03 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 9625.10 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183918.979303 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86625.93 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 8662.59 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192581.571863 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77963.33 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 7796.33 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200377.905166 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70167.00 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 7016.70 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207394.605139 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63150.30 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 6315.03 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213709.635114 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56835.27 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 5683.53 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219393.162092 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51151.74 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 5115.17 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224508.336372 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46036.57 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 4603.66 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229111.993225 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41432.91 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 4143.29 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233255.284392 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37289.62 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3728.96 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236984.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33560.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240584.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29960.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244184.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26360.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247784.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22760.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251384.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19160.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254984.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15560.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258584.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11960.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262184.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8360.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265784.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4760.66 sec) 01:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269384.246442 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1160.66 sec) 01:56:00 [GPUGRID] [rr_sim] 2270544.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998537711.12G/439.78G) 01:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850545.82 01:56:00 [---] pick_jobs_to_run() start 01:56:00 [---] pick_jobs_to_run() end 01:56:00 [---] [rr_sim] end 01:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:56:00 [---] [cpu_sched_debug] preliminary job list: 01:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:56:00 [---] [cpu_sched_debug] final job list: 01:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:56:00 [---] [cpu_sched_debug] enforce_run_list: end 01:56:00 [---] choose_project(): 6960.000000 01:56:00 [iThena.Measurements] doing RPC to get work 01:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428639.09 cpu 2171492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:56:00 [---] Got 0 tasks 01:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:57:00 [---] [rr_sim] doing sim: CPU sched 01:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998510120.76G/439.78G) 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270482.17 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 227048.22 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227048.216801 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043433.95 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 204343.40 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431391.611922 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839090.56 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 183909.06 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615300.667531 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655181.50 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 165518.15 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780818.817579 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489663.35 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 148966.34 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929785.152623 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340697.02 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 134069.70 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063854.854162 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206627.31 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 120662.73 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184517.585547 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085964.58 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 108596.46 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293114.043793 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977368.12 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 97736.81 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390850.856215 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879631.31 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 87963.13 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478813.987395 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791668.18 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 79166.82 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557980.805457 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712501.36 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 71250.14 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629230.941712 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641251.23 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 64125.12 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693356.064342 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577126.10 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 57712.61 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751068.674709 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519413.49 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 51941.35 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803010.024039 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467472.14 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 46747.21 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849757.238437 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420724.93 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 42072.49 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891829.731394 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378652.44 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 37865.24 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929694.975056 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340787.19 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 34078.72 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963773.694352 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306708.47 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 30670.85 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994444.541718 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276037.63 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 27603.76 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022048.304347 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248433.86 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 24843.39 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046891.690714 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223590.48 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 22359.05 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069250.738444 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201231.43 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 20123.14 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089373.881400 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181108.29 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 18110.83 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107484.710062 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162997.46 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 16299.75 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123784.455857 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146697.71 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 14669.77 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138454.227072 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132027.94 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 13202.79 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151657.021166 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118825.15 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 11882.51 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163539.535851 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106942.63 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 10694.26 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174233.799067 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96248.37 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 9624.84 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183858.635962 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86623.53 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 8662.35 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192520.989167 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77961.18 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 7796.12 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200317.107051 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70165.06 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 7016.51 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207333.613147 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63148.55 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 6314.86 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213648.468634 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56833.70 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 5683.37 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219331.838572 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51150.33 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 5115.03 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224446.871516 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46035.30 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 4603.53 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229050.401165 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41431.77 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 4143.18 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233193.577850 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37288.59 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3728.86 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236922.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33559.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240522.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29959.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244122.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26359.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247722.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22759.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251322.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19159.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254922.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15559.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258522.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11959.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262122.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8359.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265722.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4759.73 sec) 01:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269322.436866 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1159.73 sec) 01:57:00 [GPUGRID] [rr_sim] 2270482.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998510120.76G/439.78G) 01:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850543.08 01:57:00 [---] pick_jobs_to_run() start 01:57:00 [---] pick_jobs_to_run() end 01:57:00 [---] [rr_sim] end 01:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:57:00 [---] [cpu_sched_debug] preliminary job list: 01:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:57:00 [---] [cpu_sched_debug] final job list: 01:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:57:00 [---] [cpu_sched_debug] enforce_run_list: end 01:57:00 [---] choose_project(): 7020.000000 01:57:00 [iThena.Measurements] doing RPC to get work 01:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428579.09 cpu 2171432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:57:00 [---] Got 0 tasks 01:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:58:00 [---] [rr_sim] doing sim: CPU sched 01:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998482530.41G/439.78G) 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270419.43 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 227041.94 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227041.943113 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043377.49 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 204337.75 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431379.691915 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1839039.74 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 183903.97 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615283.665836 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655135.77 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 165513.58 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780797.242366 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489622.19 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 148962.22 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929759.461242 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340659.97 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 134066.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063825.458231 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206593.97 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 120659.40 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184484.855521 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085934.58 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 108593.46 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293078.313082 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977341.12 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 97734.11 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390812.424887 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879607.01 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 87960.70 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478773.125511 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791646.31 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 79164.63 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557937.756073 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712481.68 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 71248.17 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629185.923579 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641233.51 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 64123.35 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693309.274334 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577110.16 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 57711.02 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751020.290013 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519399.14 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 51939.91 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802960.204125 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467459.23 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 46745.92 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849706.126826 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420713.30 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 42071.33 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891777.457256 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378641.97 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 37864.20 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929641.654643 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340777.78 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 34077.78 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963719.432292 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306700.00 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 30670.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994389.432176 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276030.00 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 27603.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021992.432071 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248427.00 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 24842.70 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046835.131977 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223584.30 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 22358.43 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069193.561892 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201225.87 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 20122.59 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089316.148816 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181103.28 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 18110.33 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107426.477048 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162992.95 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 16299.30 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123725.772456 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146693.66 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 14669.37 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138395.138323 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132024.29 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 13202.43 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151597.567604 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118821.86 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 11882.19 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163479.753957 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106939.68 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 10693.97 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174173.721674 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96245.71 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 9624.57 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183798.292620 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86621.14 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 8662.11 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192460.406471 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77959.02 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 7795.90 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200256.308937 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70163.12 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 7016.31 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207272.621156 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63146.81 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 6314.68 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213587.302153 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56832.13 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 5683.21 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219270.515051 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51148.92 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 5114.89 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224385.406659 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46034.02 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 4603.40 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228988.809106 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41430.62 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 4143.06 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233131.871308 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37287.56 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3728.76 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236860.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33558.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240460.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29958.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244060.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26358.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247660.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22758.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251260.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19158.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254860.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15558.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258460.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11958.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262060.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8358.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265660.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4758.80 sec) 01:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269260.627291 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1158.80 sec) 01:58:00 [GPUGRID] [rr_sim] 2270419.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998482530.41G/439.78G) 01:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850540.34 01:58:00 [---] pick_jobs_to_run() start 01:58:00 [---] pick_jobs_to_run() end 01:58:00 [---] [rr_sim] end 01:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:58:00 [---] [cpu_sched_debug] preliminary job list: 01:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:58:00 [---] [cpu_sched_debug] final job list: 01:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:58:00 [---] [cpu_sched_debug] enforce_run_list: end 01:58:00 [---] choose_project(): 7080.000000 01:58:00 [iThena.Measurements] doing RPC to get work 01:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428519.09 cpu 2171372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:58:00 [---] Got 0 tasks 01:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 01:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 01:59:00 [---] [rr_sim] doing sim: CPU sched 01:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998454940.05G/439.78G) 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270356.69 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 227035.67 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227035.669425 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043321.02 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 204332.10 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431367.771907 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838988.92 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 183898.89 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615266.664141 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655090.03 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 165509.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780775.667152 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489581.03 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 148958.10 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929733.769861 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340622.92 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 134062.29 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063796.062300 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206560.63 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 120656.06 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184452.125495 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085904.57 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 108590.46 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293042.582370 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977314.11 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 97731.41 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390773.993558 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879582.70 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 87958.27 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478732.263627 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791624.43 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 79162.44 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557894.706689 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712461.99 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 71246.20 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629140.905445 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641215.79 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 64121.58 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693262.484325 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577094.21 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 57709.42 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750971.905318 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519384.79 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 51938.48 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802910.384211 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467446.31 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 46744.63 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849655.015214 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420701.68 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 42070.17 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891725.183118 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378631.51 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 37863.15 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929588.334231 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340768.36 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 34076.84 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963665.170232 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306691.52 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 30669.15 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994334.322634 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276022.37 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 27602.24 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021936.559795 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248420.13 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 24842.01 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046778.573241 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223578.12 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 22357.81 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069136.385341 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201220.31 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 20122.03 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089258.416232 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181098.28 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 18109.83 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107368.244034 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162988.45 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 16298.85 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123667.089055 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146689.61 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 14668.96 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138336.049574 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132020.64 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 13202.06 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151538.114042 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118818.58 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 11881.86 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163419.972062 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106936.72 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 10693.67 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174113.644281 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96243.05 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 9624.30 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183737.949278 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86618.74 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 8661.87 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192399.823775 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77956.87 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 7795.69 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200195.510822 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70161.18 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 7016.12 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207211.629165 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63145.07 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 6314.51 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213526.135673 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56830.56 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 5683.06 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219209.191530 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51147.50 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 5114.75 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224323.941802 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46032.75 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 4603.28 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228927.217047 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41429.48 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 4142.95 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233070.164767 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37286.53 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3728.65 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236798.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33557.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240398.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29957.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243998.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26357.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247598.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22757.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251198.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19157.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254798.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15557.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258398.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11957.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261998.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8357.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265598.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4757.88 sec) 01:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269198.817715 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1157.88 sec) 01:59:00 [GPUGRID] [rr_sim] 2270356.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998454940.05G/439.78G) 01:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850537.61 01:59:00 [---] pick_jobs_to_run() start 01:59:00 [---] pick_jobs_to_run() end 01:59:00 [---] [rr_sim] end 01:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 01:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 01:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 01:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 01:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 01:59:00 [---] [cpu_sched_debug] preliminary job list: 01:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:59:00 [---] [cpu_sched_debug] final job list: 01:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 01:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 01:59:00 [---] [cpu_sched_debug] enforce_run_list: end 01:59:00 [---] choose_project(): 7140.000000 01:59:00 [iThena.Measurements] doing RPC to get work 01:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428459.09 cpu 2171312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 01:59:00 [---] Got 0 tasks 02:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:00:00 [---] [rr_sim] doing sim: CPU sched 02:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998427349.70G/439.78G) 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270293.96 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 227029.40 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227029.395737 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043264.56 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 204326.46 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431355.851899 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838938.11 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 183893.81 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615249.662446 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1655044.29 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 165504.43 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780754.091938 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489539.87 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 148953.99 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929708.078481 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340585.88 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 134058.59 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063766.666369 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206527.29 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 120652.73 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184419.395469 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085874.56 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 108587.46 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1293006.851659 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977287.11 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 97728.71 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390735.562229 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879558.40 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 87955.84 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478691.401743 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791602.56 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 79160.26 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557851.657305 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712442.30 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 71244.23 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629095.887311 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641198.07 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 64119.81 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693215.694317 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577078.26 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 57707.83 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750923.520622 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519370.44 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 51937.04 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802860.564296 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467433.39 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 46743.34 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849603.903603 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420690.05 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 42069.01 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891672.908979 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378621.05 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 37862.10 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929535.013818 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340758.94 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 34075.89 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963610.908173 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306683.05 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 30668.30 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994279.213092 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276014.74 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 27601.47 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021880.687519 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248413.27 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 24841.33 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046722.014504 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223571.94 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 22357.19 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069079.208790 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201214.75 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 20121.47 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089200.683648 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181093.27 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 18109.33 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107310.011020 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162983.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 16298.39 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123608.405654 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146685.55 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 14668.56 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138276.960825 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132017.00 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 13201.70 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151478.660479 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118815.30 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 11881.53 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163360.190168 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106933.77 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 10693.38 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174053.566888 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96240.39 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 9624.04 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183677.605936 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86616.35 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 8661.64 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192339.241079 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77954.72 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 7795.47 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200134.712707 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70159.24 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 7015.92 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207150.637173 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63143.32 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 6314.33 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213464.969192 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56828.99 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 5682.90 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219147.868010 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51146.09 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 5114.61 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224262.476945 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46031.48 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 4603.15 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228865.624987 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41428.33 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 4142.83 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233008.458225 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37285.50 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3728.55 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236737.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33556.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240337.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29956.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243937.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26356.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247537.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22756.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251137.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19156.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254737.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15556.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258337.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11956.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261937.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8356.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265537.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4756.95 sec) 02:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269137.008139 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1156.95 sec) 02:00:00 [GPUGRID] [rr_sim] 2270293.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998427349.70G/439.78G) 02:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850534.87 02:00:00 [---] pick_jobs_to_run() start 02:00:00 [---] pick_jobs_to_run() end 02:00:00 [---] [rr_sim] end 02:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:00:00 [---] [cpu_sched_debug] preliminary job list: 02:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:00:00 [---] [cpu_sched_debug] final job list: 02:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:00:00 [---] [cpu_sched_debug] enforce_run_list: end 02:00:00 [---] choose_project(): 7200.000000 02:00:00 [iThena.Measurements] doing RPC to get work 02:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428399.09 cpu 2171252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:00:00 [---] Got 0 tasks 02:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:01:00 [---] [rr_sim] doing sim: CPU sched 02:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998399759.34G/439.78G) 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270231.22 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 227023.12 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227023.122048 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043208.10 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 204320.81 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431343.931892 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838887.29 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 183888.73 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615232.660751 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654998.56 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 165499.86 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780732.516724 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489498.70 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 148949.87 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929682.387100 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340548.83 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 134054.88 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063737.270439 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206493.95 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 120649.40 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184386.665443 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085844.56 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 108584.46 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292971.120947 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977260.10 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 97726.01 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390697.130901 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879534.09 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 87953.41 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478650.539859 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791580.68 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 79158.07 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557808.607922 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712422.61 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 71242.26 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629050.869178 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641180.35 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 64118.04 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693168.904308 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577062.32 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 57706.23 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750875.135926 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519356.08 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 51935.61 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802810.744382 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467420.48 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 46742.05 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849552.791992 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420678.43 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 42067.84 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891620.634841 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378610.59 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 37861.06 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929481.693405 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340749.53 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 34074.95 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963556.646113 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306674.57 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 30667.46 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994224.103550 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276007.12 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 27600.71 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021824.815243 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248406.41 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 24840.64 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046665.455767 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223565.76 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 22356.58 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069022.032239 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201209.19 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 20120.92 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089142.951064 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181088.27 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 18108.83 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107251.778006 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162979.44 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 16297.94 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123549.722253 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146681.50 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 14668.15 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138217.872076 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132013.35 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 13201.33 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151419.206917 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118812.01 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 11881.20 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163300.408274 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106930.81 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 10693.08 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173993.489495 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96237.73 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 9623.77 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183617.262594 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86613.96 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 8661.40 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192278.658383 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77952.56 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 7795.26 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200073.914593 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70157.31 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 7015.73 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207089.645182 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63141.58 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 6314.16 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213403.802712 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56827.42 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 5682.74 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219086.544489 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51144.68 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 5114.47 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224201.012089 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46030.21 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 4603.02 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228804.032928 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41427.19 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 4142.72 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232946.751684 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37284.47 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3728.45 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236675.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33556.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240275.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29956.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243875.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26356.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247475.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22756.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251075.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19156.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254675.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15556.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258275.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11956.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261875.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8356.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265475.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4756.02 sec) 02:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269075.198564 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1156.02 sec) 02:01:00 [GPUGRID] [rr_sim] 2270231.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998399759.34G/439.78G) 02:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850532.13 02:01:00 [---] pick_jobs_to_run() start 02:01:00 [---] pick_jobs_to_run() end 02:01:00 [---] [rr_sim] end 02:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:01:00 [---] [cpu_sched_debug] preliminary job list: 02:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:01:00 [---] [cpu_sched_debug] final job list: 02:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:01:00 [---] [cpu_sched_debug] enforce_run_list: end 02:01:00 [---] choose_project(): 7260.000000 02:01:00 [iThena.Measurements] doing RPC to get work 02:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428339.09 cpu 2171192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:01:00 [---] Got 0 tasks 02:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:02:00 [---] [rr_sim] doing sim: CPU sched 02:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998399759.34G/439.78G) 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270231.22 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 227023.12 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227023.122048 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043208.10 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 204320.81 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431343.931892 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838887.29 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 183888.73 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615232.660751 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654998.56 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 165499.86 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780732.516724 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489498.70 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 148949.87 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929682.387100 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340548.83 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 134054.88 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063737.270439 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206493.95 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 120649.40 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184386.665443 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085844.56 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 108584.46 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292971.120947 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977260.10 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 97726.01 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390697.130901 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879534.09 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 87953.41 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478650.539859 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791580.68 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 79158.07 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557808.607922 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712422.61 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 71242.26 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629050.869178 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641180.35 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 64118.04 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693168.904308 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577062.32 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 57706.23 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750875.135926 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519356.08 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 51935.61 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802810.744382 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467420.48 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 46742.05 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849552.791992 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420678.43 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 42067.84 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891620.634841 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378610.59 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 37861.06 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929481.693405 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340749.53 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 34074.95 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963556.646113 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306674.57 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 30667.46 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994224.103550 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276007.12 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 27600.71 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021824.815243 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248406.41 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 24840.64 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046665.455767 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223565.76 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 22356.58 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069022.032239 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201209.19 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 20120.92 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089142.951064 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181088.27 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 18108.83 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107251.778006 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162979.44 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 16297.94 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123549.722253 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146681.50 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 14668.15 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138217.872076 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132013.35 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 13201.33 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151419.206917 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118812.01 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 11881.20 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163300.408274 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106930.81 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 10693.08 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173993.489495 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96237.73 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 9623.77 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183617.262594 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86613.96 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 8661.40 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192278.658383 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77952.56 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 7795.26 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200073.914593 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70157.31 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 7015.73 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207089.645182 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63141.58 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 6314.16 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213403.802712 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56827.42 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 5682.74 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219086.544489 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51144.68 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 5114.47 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224201.012089 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46030.21 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 4603.02 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228804.032928 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41427.19 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 4142.72 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232946.751684 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37284.47 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3728.45 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236675.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33556.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240275.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29956.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243875.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26356.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247475.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22756.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251075.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19156.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254675.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15556.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258275.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11956.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261875.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8356.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265475.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4756.02 sec) 02:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269075.198564 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1156.02 sec) 02:02:00 [GPUGRID] [rr_sim] 2270231.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998399759.34G/439.78G) 02:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850592.13 02:02:00 [---] pick_jobs_to_run() start 02:02:00 [---] pick_jobs_to_run() end 02:02:00 [---] [rr_sim] end 02:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:02:00 [---] [cpu_sched_debug] preliminary job list: 02:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:02:00 [---] [cpu_sched_debug] final job list: 02:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:02:00 [---] [cpu_sched_debug] enforce_run_list: end 02:02:00 [---] choose_project(): 7320.000000 02:02:00 [iThena.Measurements] doing RPC to get work 02:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428279.09 cpu 2171192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:02:00 [---] Got 0 tasks 02:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:03:00 [---] [rr_sim] doing sim: CPU sched 02:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998372168.98G/439.78G) 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270168.48 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 227016.85 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227016.848360 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043151.64 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 204315.16 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431332.011884 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838836.47 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 183883.65 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615215.659056 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654952.82 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 165495.28 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780710.941511 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489457.54 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 148945.75 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929656.695720 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340511.79 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 134051.18 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063707.874508 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206460.61 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 120646.06 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184353.935417 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085814.55 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 108581.45 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292935.390236 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977233.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 97723.31 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390658.699572 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879509.78 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 87950.98 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478609.677975 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791558.81 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 79155.88 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557765.558538 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712402.93 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 71240.29 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629005.851044 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641162.63 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 64116.26 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693122.114300 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577046.37 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 57704.64 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750826.751230 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519341.73 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 51934.17 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802760.924467 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467407.56 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 46740.76 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849501.680381 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420666.80 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 42066.68 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891568.360703 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378600.12 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 37860.01 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929428.372993 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340740.11 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 34074.01 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963502.384053 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306666.10 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 30666.61 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994168.994008 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275999.49 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 27599.95 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021768.942968 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248399.54 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 24839.95 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046608.897031 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223559.59 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 22355.96 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068964.855688 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201203.63 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 20120.36 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089085.218479 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181083.27 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 18108.33 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107193.544992 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162974.94 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 16297.49 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123491.038853 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146677.44 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 14667.74 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138158.783327 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132009.70 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 13200.97 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151359.753355 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118808.73 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 11880.87 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163240.626379 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106927.86 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 10692.79 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173933.412102 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96235.07 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 9623.51 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183556.919252 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86611.56 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 8661.16 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192218.075687 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77950.41 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 7795.04 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200013.116478 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70155.37 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 7015.54 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207028.653190 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63139.83 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 6313.98 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213342.636232 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56825.85 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 5682.58 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219025.220969 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51143.26 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 5114.33 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224139.547232 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46028.94 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 4602.89 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228742.440869 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41426.04 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 4142.60 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232885.045142 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37283.44 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3728.34 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236613.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33555.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240213.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29955.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243813.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26355.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247413.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22755.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251013.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19155.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254613.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15555.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258213.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11955.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261813.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8355.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265413.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4755.09 sec) 02:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269013.388988 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1155.09 sec) 02:03:00 [GPUGRID] [rr_sim] 2270168.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998372168.98G/439.78G) 02:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850589.40 02:03:00 [---] pick_jobs_to_run() start 02:03:00 [---] pick_jobs_to_run() end 02:03:00 [---] [rr_sim] end 02:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:03:00 [---] [cpu_sched_debug] preliminary job list: 02:03:00 [---] [cpu_sched_debug] final job list: 02:03:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 02:03:00 [---] [work_fetch] Request work fetch: CPUs idle 02:03:00 [---] [cpu_sched_debug] enforce_run_list: end 02:03:00 [---] choose_project(): 7380.000000 02:03:00 [iThena.Measurements] doing RPC to get work 02:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 601019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428219.09 cpu 2171132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:03:00 [---] Got 0 tasks 02:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:04:00 [---] [rr_sim] doing sim: CPU sched 02:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998372168.98G/439.78G) 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270168.48 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 227016.85 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227016.848360 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043151.64 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 204315.16 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431332.011884 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838836.47 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 183883.65 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615215.659056 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654952.82 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 165495.28 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780710.941511 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489457.54 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 148945.75 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929656.695720 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340511.79 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 134051.18 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063707.874508 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206460.61 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 120646.06 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184353.935417 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085814.55 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 108581.45 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292935.390236 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977233.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 97723.31 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390658.699572 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879509.78 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 87950.98 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478609.677975 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791558.81 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 79155.88 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557765.558538 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712402.93 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 71240.29 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1629005.851044 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641162.63 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 64116.26 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693122.114300 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577046.37 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 57704.64 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750826.751230 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519341.73 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 51934.17 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802760.924467 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467407.56 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 46740.76 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849501.680381 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420666.80 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 42066.68 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891568.360703 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378600.12 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 37860.01 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929428.372993 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340740.11 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 34074.01 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963502.384053 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306666.10 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 30666.61 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994168.994008 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275999.49 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 27599.95 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021768.942968 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248399.54 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 24839.95 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046608.897031 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223559.59 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 22355.96 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068964.855688 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201203.63 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 20120.36 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089085.218479 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181083.27 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 18108.33 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107193.544992 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162974.94 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 16297.49 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123491.038853 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146677.44 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 14667.74 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138158.783327 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132009.70 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 13200.97 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151359.753355 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118808.73 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 11880.87 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163240.626379 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106927.86 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 10692.79 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173933.412102 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96235.07 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 9623.51 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183556.919252 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86611.56 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 8661.16 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192218.075687 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77950.41 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 7795.04 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200013.116478 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70155.37 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 7015.54 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207028.653190 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63139.83 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 6313.98 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213342.636232 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56825.85 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 5682.58 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219025.220969 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51143.26 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 5114.33 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224139.547232 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46028.94 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 4602.89 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228742.440869 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41426.04 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 4142.60 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232885.045142 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37283.44 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3728.34 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236613.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33555.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240213.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29955.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243813.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26355.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247413.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22755.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251013.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19155.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254613.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15555.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258213.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11955.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261813.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8355.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265413.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4755.09 sec) 02:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2269013.388988 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1155.09 sec) 02:04:00 [GPUGRID] [rr_sim] 2270168.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998372168.98G/439.78G) 02:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850649.40 02:04:00 [---] pick_jobs_to_run() start 02:04:00 [---] pick_jobs_to_run() end 02:04:00 [---] [rr_sim] end 02:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:04:00 [---] [cpu_sched_debug] preliminary job list: 02:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:04:00 [---] [cpu_sched_debug] final job list: 02:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 02:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:04:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 998372168.98G 02:04:00 [---] [cpu_sched_debug] enforce_run_list: end 02:04:00 [---] choose_project(): 7440.000000 02:04:00 [iThena.Measurements] doing RPC to get work 02:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428159.09 cpu 2171132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:04:00 [---] Got 0 tasks 02:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:05:00 [---] [rr_sim] doing sim: CPU sched 02:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998344578.63G/439.78G) 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270105.75 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 227010.57 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227010.574672 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043095.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 204309.52 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431320.091877 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838785.65 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 183878.57 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615198.657361 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654907.09 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 165490.71 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780689.366297 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489416.38 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 148941.64 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929631.004339 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340474.74 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 134047.47 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063678.478577 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206427.27 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 120642.73 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184321.205391 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085784.54 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 108578.45 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292899.659524 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977206.09 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 97720.61 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390620.268244 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879485.48 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 87948.55 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478568.816091 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791536.93 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 79153.69 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557722.509154 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712383.24 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 71238.32 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628960.832910 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641144.91 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 64114.49 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693075.324291 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577030.42 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 57703.04 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750778.366534 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519327.38 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 51932.74 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802711.104553 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467394.64 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 46739.46 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849450.568769 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420655.18 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 42065.52 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891516.086564 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378589.66 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 37858.97 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929375.052580 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340730.69 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 34073.07 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963448.121994 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306657.62 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 30665.76 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994113.884466 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275991.86 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 27599.19 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021713.070692 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248392.68 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 24839.27 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046552.338294 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223553.41 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 22355.34 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068907.679137 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201198.07 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 20119.81 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089027.485895 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181078.26 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 18107.83 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107135.311978 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162970.43 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 16297.04 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123432.355452 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146673.39 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 14667.34 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138099.694578 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132006.05 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 13200.61 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151300.299793 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118805.45 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 11880.54 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163180.844485 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106924.90 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 10692.49 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173873.334709 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96232.41 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 9623.24 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183496.575910 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86609.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 8660.92 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192157.492991 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77948.25 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 7794.83 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199952.318363 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70153.43 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 7015.34 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206967.661199 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63138.09 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 6313.81 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213281.469751 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56824.28 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 5682.43 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218963.897448 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51141.85 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 5114.18 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224078.082375 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46027.66 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 4602.77 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228680.848809 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41424.90 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 4142.49 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232823.338600 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37282.41 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3728.24 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236551.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33554.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240151.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29954.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243751.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26354.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247351.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22754.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250951.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19154.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254551.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15554.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258151.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11954.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261751.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8354.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265351.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4754.17 sec) 02:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268951.579412 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1154.17 sec) 02:05:00 [GPUGRID] [rr_sim] 2270105.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998344578.63G/439.78G) 02:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850646.66 02:05:00 [---] pick_jobs_to_run() start 02:05:00 [---] pick_jobs_to_run() end 02:05:00 [---] [rr_sim] end 02:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:05:00 [---] [cpu_sched_debug] preliminary job list: 02:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 02:05:00 [---] [cpu_sched_debug] final job list: 02:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:05:00 [---] [cpu_sched_debug] enforce_run_list: end 02:05:00 [---] choose_project(): 7500.000000 02:05:00 [iThena.Measurements] doing RPC to get work 02:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428099.09 cpu 2171072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:05:00 [---] Got 0 tasks 02:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:06:00 [---] [rr_sim] doing sim: CPU sched 02:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998316988.27G/439.78G) 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2270043.01 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 227004.30 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 227004.300984 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2043038.71 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 204303.87 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431308.171869 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838734.84 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 183873.48 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615181.655666 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654861.35 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 165486.14 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780667.791083 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489375.22 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 148937.52 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929605.312958 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340437.70 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 134043.77 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063649.082646 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206393.93 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 120639.39 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184288.475365 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085754.53 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 108575.45 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292863.928813 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977179.08 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 97717.91 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390581.836915 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879461.17 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 87946.12 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478527.954207 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791515.06 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 79151.51 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557679.459770 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712363.55 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 71236.36 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628915.814777 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641127.20 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 64112.72 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693028.534283 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577014.48 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 57701.45 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750729.981838 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519313.03 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 51931.30 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802661.284638 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467381.73 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 46738.17 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849399.457158 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420643.55 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 42064.36 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891463.812426 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378579.20 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 37857.92 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929321.732167 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340721.28 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 34072.13 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963393.859934 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306649.15 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 30664.91 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994058.774924 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275984.23 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 27598.42 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021657.198416 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248385.81 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 24838.58 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046495.779558 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223547.23 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 22354.72 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068850.502586 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201192.51 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 20119.25 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088969.753311 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181073.26 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 18107.33 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107077.078963 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162965.93 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 16296.59 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123373.672051 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146669.34 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 14666.93 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138040.605829 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132002.40 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 13200.24 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151240.846230 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118802.16 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 11880.22 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163121.062591 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106921.95 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 10692.19 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173813.257316 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96229.75 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 9622.98 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183436.232568 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86606.78 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 8660.68 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192096.910295 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77946.10 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 7794.61 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199891.520249 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70151.49 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 7015.15 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206906.669208 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63136.34 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 6313.63 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213220.303271 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56822.71 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 5682.27 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218902.573927 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51140.44 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 5114.04 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224016.617518 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46026.39 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 4602.64 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228619.256750 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41423.75 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 4142.38 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232761.632059 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37281.38 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3728.14 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236489.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33553.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240089.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29953.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243689.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26353.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247289.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22753.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250889.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19153.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254489.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15553.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258089.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11953.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261689.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8353.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265289.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4753.24 sec) 02:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268889.769837 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1153.24 sec) 02:06:00 [GPUGRID] [rr_sim] 2270043.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998316988.27G/439.78G) 02:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850643.92 02:06:00 [---] pick_jobs_to_run() start 02:06:00 [---] pick_jobs_to_run() end 02:06:00 [---] [rr_sim] end 02:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:06:00 [---] [cpu_sched_debug] preliminary job list: 02:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:06:00 [---] [cpu_sched_debug] final job list: 02:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:06:00 [---] [cpu_sched_debug] enforce_run_list: end 02:06:00 [---] choose_project(): 7560.000000 02:06:00 [iThena.Measurements] doing RPC to get work 02:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 428039.09 cpu 2171012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2171012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2171012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:06:00 [---] Got 0 tasks 02:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:07:00 [---] [rr_sim] doing sim: CPU sched 02:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998289397.91G/439.78G) 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269980.27 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 226998.03 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226998.027295 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042982.25 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 204298.22 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431296.251861 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838684.02 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 183868.40 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615164.653971 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654815.62 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 165481.56 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780646.215869 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489334.06 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 148933.41 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929579.621578 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340400.65 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 134040.07 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063619.686715 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206360.59 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 120636.06 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184255.745339 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085724.53 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 108572.45 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292828.198101 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977152.07 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 97715.21 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390543.405586 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879436.87 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 87943.69 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478487.092323 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791493.18 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 79149.32 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557636.410386 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712343.86 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 71234.39 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628870.796643 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641109.48 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 64110.95 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692981.744274 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576998.53 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 57699.85 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750681.597142 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519298.68 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 51929.87 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802611.464724 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467368.81 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 46736.88 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849348.345547 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420631.93 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 42063.19 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891411.538288 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378568.73 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 37856.87 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929268.411754 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340711.86 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 34071.19 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963339.597874 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306640.68 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 30664.07 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994003.665382 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275976.61 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 27597.66 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021601.326140 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248378.95 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 24837.89 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046439.220821 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223541.05 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 22354.11 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068793.326035 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201186.95 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 20118.69 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088912.020727 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181068.25 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 18106.83 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107018.845949 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162961.43 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 16296.14 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123314.988650 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146665.28 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 14666.53 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137981.517081 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131998.76 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 13199.88 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151181.392668 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118798.88 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 11879.89 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163061.280697 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106918.99 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 10691.90 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173753.179922 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96227.09 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 9622.71 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183375.889226 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86604.38 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 8660.44 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192036.327599 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77943.95 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 7794.39 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199830.722134 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70149.55 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 7014.96 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206845.677216 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63134.60 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 6313.46 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213159.136790 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56821.14 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 5682.11 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218841.250407 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51139.02 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 5113.90 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223955.152661 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46025.12 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 4602.51 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228557.664691 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41422.61 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 4142.26 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232699.925517 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37280.35 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3728.03 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236427.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33552.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240027.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29952.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243627.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26352.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247227.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22752.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250827.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19152.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254427.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15552.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258027.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11952.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261627.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8352.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265227.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4752.31 sec) 02:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268827.960261 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1152.31 sec) 02:07:00 [GPUGRID] [rr_sim] 2269980.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998289397.91G/439.78G) 02:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850641.19 02:07:00 [---] pick_jobs_to_run() start 02:07:00 [---] pick_jobs_to_run() end 02:07:00 [---] [rr_sim] end 02:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:07:00 [---] [cpu_sched_debug] preliminary job list: 02:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:07:00 [---] [cpu_sched_debug] final job list: 02:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:07:00 [---] [cpu_sched_debug] enforce_run_list: end 02:07:00 [---] choose_project(): 7620.000000 02:07:00 [iThena.Measurements] doing RPC to get work 02:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427979.09 cpu 2170952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:07:00 [---] Got 0 tasks 02:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:08:00 [---] [rr_sim] doing sim: CPU sched 02:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998261807.56G/439.78G) 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269917.54 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 226991.75 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226991.753607 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042925.78 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 204292.58 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431284.331854 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838633.20 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 183863.32 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615147.652276 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654769.88 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 165476.99 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780624.640655 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489292.90 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 148929.29 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929553.930197 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340363.61 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 134036.36 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063590.290785 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206327.25 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 120632.72 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184223.015314 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085694.52 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 108569.45 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292792.467389 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977125.07 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 97712.51 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390504.974258 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879412.56 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 87941.26 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478446.230439 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791471.31 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 79147.13 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557593.361003 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712324.18 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 71232.42 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628825.778510 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641091.76 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 64109.18 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692934.954266 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576982.58 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 57698.26 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750633.212447 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519284.32 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 51928.43 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802561.644809 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467355.89 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 46735.59 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849297.233936 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420620.30 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 42062.03 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891359.264149 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378558.27 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 37855.83 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929215.091342 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340702.44 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 34070.24 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963285.335815 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306632.20 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 30663.22 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993948.555841 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275968.98 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 27596.90 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021545.453864 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248372.08 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 24837.21 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046382.662085 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223534.87 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 22353.49 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068736.149483 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201181.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 20118.14 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088854.288142 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181063.25 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 18106.32 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106960.612935 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162956.92 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 16295.69 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123256.305249 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146661.23 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 14666.12 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137922.428332 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131995.11 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 13199.51 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151121.939106 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118795.60 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 11879.56 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163001.498802 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106916.04 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 10691.60 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173693.102529 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96224.43 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 9622.44 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183315.545884 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86601.99 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 8660.20 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191975.744903 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77941.79 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 7794.18 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199769.924020 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70147.61 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 7014.76 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206784.685225 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63132.85 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 6313.29 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213097.970310 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56819.57 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 5681.96 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218779.926886 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51137.61 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 5113.76 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223893.687805 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46023.85 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 4602.38 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228496.072632 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41421.46 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 4142.15 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232638.218976 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37279.32 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3727.93 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236366.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33551.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239966.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29951.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243566.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26351.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247166.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22751.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250766.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19151.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254366.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15551.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257966.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11951.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261566.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8351.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265166.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4751.39 sec) 02:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268766.150685 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1151.39 sec) 02:08:00 [GPUGRID] [rr_sim] 2269917.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998261807.56G/439.78G) 02:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850638.45 02:08:00 [---] pick_jobs_to_run() start 02:08:00 [---] pick_jobs_to_run() end 02:08:00 [---] [rr_sim] end 02:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:08:00 [---] [cpu_sched_debug] preliminary job list: 02:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:08:00 [---] [cpu_sched_debug] final job list: 02:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:08:00 [---] [cpu_sched_debug] enforce_run_list: end 02:08:00 [---] choose_project(): 7680.000000 02:08:00 [iThena.Measurements] doing RPC to get work 02:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427919.09 cpu 2170892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:08:00 [---] Got 0 tasks 02:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:09:00 [---] [rr_sim] doing sim: CPU sched 02:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998234217.20G/439.78G) 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269854.80 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 226985.48 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226985.479919 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042869.32 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 204286.93 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431272.411846 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838582.39 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 183858.24 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615130.650581 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654724.15 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 165472.41 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780603.065442 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489251.73 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 148925.17 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929528.238817 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340326.56 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 134032.66 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063560.894854 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206293.90 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 120629.39 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184190.285288 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085664.51 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 108566.45 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292756.736678 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977098.06 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 97709.81 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390466.542929 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879388.26 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 87938.83 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478405.368555 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791449.43 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 79144.94 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557550.311619 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712304.49 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 71230.45 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628780.760376 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641074.04 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 64107.40 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692888.164257 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576966.63 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 57696.66 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750584.827751 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519269.97 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 51927.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802511.824895 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467342.97 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 46734.30 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849246.122324 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420608.68 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 42060.87 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891306.990011 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378547.81 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 37854.78 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929161.770929 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340693.03 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 34069.30 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963231.073755 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306623.73 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 30662.37 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993893.446299 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275961.35 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 27596.14 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021489.581588 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248365.22 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 24836.52 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046326.103348 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223528.70 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 22352.87 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068678.972932 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201175.83 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 20117.58 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088796.555558 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181058.24 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 18105.82 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106902.379921 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162952.42 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 16295.24 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123197.621848 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146657.18 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 14665.72 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137863.339583 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131991.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 13199.15 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151062.485543 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118792.31 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 11879.23 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162941.716908 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106913.08 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 10691.31 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173633.025136 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96221.77 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 9622.18 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183255.202542 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86599.60 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 8659.96 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191915.162207 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77939.64 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 7793.96 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199709.125905 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70145.67 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 7014.57 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206723.693234 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63131.11 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 6313.11 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213036.803829 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56818.00 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 5681.80 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218718.603365 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51136.20 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 5113.62 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223832.222948 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46022.58 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 4602.26 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228434.480572 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41420.32 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 4142.03 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232576.512434 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37278.29 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3727.83 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236304.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33550.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239904.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29950.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243504.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26350.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247104.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22750.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250704.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19150.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254304.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15550.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257904.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11950.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261504.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8350.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265104.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4750.46 sec) 02:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268704.341110 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1150.46 sec) 02:09:00 [GPUGRID] [rr_sim] 2269854.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998234217.20G/439.78G) 02:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850635.71 02:09:00 [---] pick_jobs_to_run() start 02:09:00 [---] pick_jobs_to_run() end 02:09:00 [---] [rr_sim] end 02:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:09:00 [---] [cpu_sched_debug] preliminary job list: 02:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:09:00 [---] [cpu_sched_debug] final job list: 02:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:09:00 [---] [cpu_sched_debug] enforce_run_list: end 02:09:00 [---] choose_project(): 7740.000000 02:09:00 [iThena.Measurements] doing RPC to get work 02:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427859.09 cpu 2170832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:09:00 [---] Got 0 tasks 02:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:10:00 [---] [rr_sim] doing sim: CPU sched 02:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998206626.85G/439.78G) 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269792.06 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 226979.21 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226979.206231 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042812.86 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 204281.29 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431260.491839 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838531.57 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 183853.16 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615113.648886 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654678.41 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 165467.84 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780581.490228 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489210.57 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 148921.06 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929502.547436 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340289.51 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 134028.95 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063531.498923 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206260.56 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 120626.06 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184157.555262 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085634.51 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 108563.45 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292721.005966 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977071.06 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 97707.11 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390428.111601 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879363.95 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 87936.40 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478364.506671 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791427.56 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 79142.76 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557507.262235 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712284.80 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 71228.48 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628735.742242 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641056.32 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 64105.63 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692841.374249 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576950.69 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 57695.07 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750536.443055 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519255.62 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 51925.56 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802462.004980 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467330.06 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 46733.01 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849195.010713 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420597.05 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 42059.71 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891254.715873 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378537.35 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 37853.73 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929108.450516 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340683.61 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 34068.36 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963176.811695 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306615.25 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 30661.53 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993838.336757 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275953.73 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 27595.37 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021433.709312 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248358.35 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 24835.84 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046269.544612 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223522.52 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 22352.25 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068621.796381 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201170.27 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 20117.03 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088738.822974 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181053.24 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 18105.32 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106844.146907 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162947.92 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 16294.79 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123138.938448 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146653.12 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 14665.31 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137804.250834 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131987.81 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 13198.78 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151003.031981 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118789.03 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 11878.90 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162881.935014 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106910.13 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 10691.01 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173572.947743 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96219.11 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 9621.91 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183194.859200 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86597.20 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 8659.72 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191854.579511 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77937.48 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 7793.75 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199648.327790 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70143.73 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 7014.37 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206662.701242 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63129.36 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 6312.94 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212975.637349 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56816.42 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 5681.64 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218657.279845 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51134.78 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 5113.48 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223770.758091 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46021.30 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 4602.13 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228372.888513 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41419.17 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 4141.92 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232514.805892 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37277.26 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3727.73 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236242.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33549.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239842.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29949.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243442.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26349.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247042.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22749.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250642.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19149.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254242.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15549.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257842.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11949.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261442.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8349.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265042.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4749.53 sec) 02:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268642.531534 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1149.53 sec) 02:10:00 [GPUGRID] [rr_sim] 2269792.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998206626.85G/439.78G) 02:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850632.98 02:10:00 [---] pick_jobs_to_run() start 02:10:00 [---] pick_jobs_to_run() end 02:10:00 [---] [rr_sim] end 02:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:10:00 [---] [cpu_sched_debug] preliminary job list: 02:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:10:00 [---] [cpu_sched_debug] final job list: 02:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:10:00 [---] [cpu_sched_debug] enforce_run_list: end 02:10:00 [---] choose_project(): 7800.000000 02:10:00 [iThena.Measurements] doing RPC to get work 02:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427799.09 cpu 2170772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:10:00 [---] Got 0 tasks 02:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:11:00 [---] [rr_sim] doing sim: CPU sched 02:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998179036.49G/439.78G) 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269729.33 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 226972.93 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226972.932543 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042756.39 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 204275.64 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431248.571831 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838480.75 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 183848.08 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615096.647191 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654632.68 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 165463.27 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780559.915014 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489169.41 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 148916.94 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929476.856055 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340252.47 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 134025.25 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063502.102992 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206227.22 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 120622.72 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184124.825236 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085604.50 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 108560.45 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292685.275255 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977044.05 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 97704.41 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390389.680272 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879339.65 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 87933.96 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478323.644787 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791405.68 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 79140.57 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557464.212851 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712265.11 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 71226.51 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628690.724109 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641038.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 64103.86 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692794.584241 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576934.74 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 57693.47 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750488.058359 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519241.27 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 51924.13 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802412.185066 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467317.14 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 46731.71 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849143.899102 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420585.43 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 42058.54 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891202.441734 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378526.88 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 37852.69 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929055.130104 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340674.20 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 34067.42 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963122.549636 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306606.78 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 30660.68 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993783.227215 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275946.10 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 27594.61 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021377.837036 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248351.49 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 24835.15 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046212.985875 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223516.34 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 22351.63 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068564.619830 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201164.71 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 20116.47 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088681.090390 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181048.24 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 18104.82 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106785.913893 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162943.41 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 16294.34 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123080.255047 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146649.07 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 14664.91 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137745.162085 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131984.16 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 13198.42 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150943.578419 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118785.75 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 11878.57 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162822.153120 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106907.17 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 10690.72 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173512.870350 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96216.46 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 9621.65 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183134.515858 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86594.81 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 8659.48 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191793.996815 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77935.33 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 7793.53 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199587.529676 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70141.80 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 7014.18 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206601.709251 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63127.62 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 6312.76 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212914.470868 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56814.85 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 5681.49 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218595.956324 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51133.37 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 5113.34 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223709.293234 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46020.03 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 4602.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228311.296454 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41418.03 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 4141.80 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232453.099351 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37276.23 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3727.62 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236180.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33548.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239780.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29948.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243380.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26348.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246980.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22748.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250580.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19148.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254180.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15548.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257780.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11948.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261380.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8348.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264980.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4748.60 sec) 02:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268580.721958 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1148.60 sec) 02:11:00 [GPUGRID] [rr_sim] 2269729.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998179036.49G/439.78G) 02:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850630.24 02:11:00 [---] pick_jobs_to_run() start 02:11:00 [---] pick_jobs_to_run() end 02:11:00 [---] [rr_sim] end 02:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:11:00 [---] [cpu_sched_debug] preliminary job list: 02:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:11:00 [---] [cpu_sched_debug] final job list: 02:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:11:00 [---] [cpu_sched_debug] enforce_run_list: end 02:11:00 [---] choose_project(): 7860.000000 02:11:00 [iThena.Measurements] doing RPC to get work 02:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427739.09 cpu 2170712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:11:00 [---] Got 0 tasks 02:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:12:00 [---] [rr_sim] doing sim: CPU sched 02:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998151446.13G/439.78G) 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269666.59 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 226966.66 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226966.658854 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042699.93 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 204269.99 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431236.651823 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838429.94 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 183842.99 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615079.645495 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654586.94 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 165458.69 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780538.339800 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489128.25 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 148912.82 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929451.164675 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340215.42 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 134021.54 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063472.707062 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206193.88 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 120619.39 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184092.095210 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085574.49 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 108557.45 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292649.544543 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977017.04 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 97701.70 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390351.248943 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879315.34 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 87931.53 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478282.782903 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791383.81 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 79138.38 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557421.163468 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712245.43 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 71224.54 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628645.705975 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641020.88 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 64102.09 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692747.794232 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576918.79 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 57691.88 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750439.673663 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519226.91 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 51922.69 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802362.365151 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467304.22 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 46730.42 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849092.787491 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420573.80 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 42057.38 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891150.167596 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378516.42 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 37851.64 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929001.809691 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340664.78 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 34066.48 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963068.287576 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306598.30 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 30659.83 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993728.117673 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275938.47 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 27593.85 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021321.964760 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248344.62 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 24834.46 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046156.427138 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223510.16 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 22351.02 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068507.443279 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201159.15 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 20115.91 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088623.357806 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181043.23 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 18104.32 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106727.680879 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162938.91 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 16293.89 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123021.571646 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146645.02 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 14664.50 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137686.073336 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131980.52 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 13198.05 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150884.124857 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118782.46 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 11878.25 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162762.371225 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106904.22 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 10690.42 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173452.792957 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96213.80 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 9621.38 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183074.172516 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86592.42 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 8659.24 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191733.414119 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77933.17 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 7793.32 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199526.731561 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70139.86 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 7013.99 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206540.717260 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63125.87 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 6312.59 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212853.304388 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56813.28 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 5681.33 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218534.632804 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51131.96 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 5113.20 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223647.828378 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46018.76 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 4601.88 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228249.704394 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41416.88 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 4141.69 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232391.392809 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37275.20 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3727.52 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236118.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33547.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239718.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29947.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243318.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26347.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246918.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22747.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250518.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19147.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254118.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15547.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257718.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11947.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261318.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8347.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264918.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4747.68 sec) 02:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268518.912383 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1147.68 sec) 02:12:00 [GPUGRID] [rr_sim] 2269666.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998151446.13G/439.78G) 02:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850627.50 02:12:00 [---] pick_jobs_to_run() start 02:12:00 [---] pick_jobs_to_run() end 02:12:00 [---] [rr_sim] end 02:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:12:00 [---] [cpu_sched_debug] preliminary job list: 02:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:12:00 [---] [cpu_sched_debug] final job list: 02:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:12:00 [---] [cpu_sched_debug] enforce_run_list: end 02:12:00 [---] choose_project(): 7920.000000 02:12:00 [iThena.Measurements] doing RPC to get work 02:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427679.09 cpu 2170652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:12:00 [---] Got 0 tasks 02:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:13:00 [---] [rr_sim] doing sim: CPU sched 02:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998123855.78G/439.78G) 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269603.85 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 226960.39 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226960.385166 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042643.47 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 204264.35 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431224.731816 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838379.12 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 183837.91 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615062.643800 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654541.21 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 165454.12 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780516.764587 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489087.09 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 148908.71 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929425.473294 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340178.38 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 134017.84 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063443.311131 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206160.54 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 120616.05 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184059.365184 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085544.49 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 108554.45 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292613.813832 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976990.04 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 97699.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390312.817615 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879291.03 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 87929.10 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478241.921020 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791361.93 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 79136.19 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557378.114084 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712225.74 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 71222.57 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628600.687842 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641003.16 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 64100.32 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692701.004224 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576902.85 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 57690.28 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750391.288967 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519212.56 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 51921.26 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802312.545237 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467291.31 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 46729.13 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849041.675879 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420562.18 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 42056.22 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891097.893458 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378505.96 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 37850.60 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928948.489278 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340655.36 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 34065.54 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963014.025516 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306589.83 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 30658.98 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993673.008131 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275930.84 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 27593.08 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021266.092484 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248337.76 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 24833.78 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046099.868402 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223503.98 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 22350.40 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068450.266728 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201153.58 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 20115.36 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088565.625221 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181038.23 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 18103.82 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106669.447865 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162934.40 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 16293.44 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122962.888245 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146640.96 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 14664.10 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137626.984587 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131976.87 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 13197.69 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150824.671294 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118779.18 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 11877.92 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162702.589331 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106901.26 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 10690.13 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173392.715564 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96211.14 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 9621.11 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183013.829174 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86590.02 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 8659.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191672.831423 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77931.02 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 7793.10 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199465.933447 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70137.92 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 7013.79 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206479.725268 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63124.13 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 6312.41 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212792.137908 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56811.71 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 5681.17 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218473.309283 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51130.54 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 5113.05 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223586.363521 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46017.49 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 4601.75 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228188.112335 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41415.74 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 4141.57 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232329.686268 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37274.17 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3727.42 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236057.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33546.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239657.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29946.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243257.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26346.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246857.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22746.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250457.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19146.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254057.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15546.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257657.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11946.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261257.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8346.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264857.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4746.75 sec) 02:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268457.102807 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1146.75 sec) 02:13:00 [GPUGRID] [rr_sim] 2269603.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998123855.78G/439.78G) 02:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850624.76 02:13:00 [---] pick_jobs_to_run() start 02:13:00 [---] pick_jobs_to_run() end 02:13:00 [---] [rr_sim] end 02:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:13:00 [---] [cpu_sched_debug] preliminary job list: 02:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:13:00 [---] [cpu_sched_debug] final job list: 02:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:13:00 [---] [cpu_sched_debug] enforce_run_list: end 02:13:00 [---] choose_project(): 7980.000000 02:13:00 [iThena.Measurements] doing RPC to get work 02:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427619.09 cpu 2170592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:13:00 [---] Got 0 tasks 02:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:14:00 [---] [rr_sim] doing sim: CPU sched 02:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998096265.42G/439.78G) 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269541.11 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 226954.11 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226954.111478 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042587.00 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 204258.70 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431212.811808 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838328.30 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 183832.83 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615045.642105 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654495.47 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 165449.55 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780495.189373 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489045.93 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 148904.59 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929399.781913 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340141.33 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 134014.13 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063413.915200 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206127.20 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 120612.72 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1184026.635158 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085514.48 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 108551.45 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292578.083120 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976963.03 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 97696.30 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390274.386286 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879266.73 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 87926.67 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478201.059136 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791340.06 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 79134.01 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557335.064700 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712206.05 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 71220.61 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628555.669708 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640985.45 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 64098.54 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692654.214215 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576886.90 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 57688.69 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750342.904272 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519198.21 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 51919.82 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802262.725322 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467278.39 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 46727.84 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848990.564268 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420550.55 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 42055.06 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891045.619319 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378495.50 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 37849.55 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928895.168865 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340645.95 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 34064.59 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962959.763457 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306581.35 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 30658.14 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993617.898589 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275923.22 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 27592.32 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021210.220208 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248330.89 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 24833.09 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046043.309665 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223497.81 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 22349.78 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068393.090177 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201148.02 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 20114.80 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088507.892637 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181033.22 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 18103.32 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106611.214851 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162929.90 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 16292.99 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122904.204844 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146636.91 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 14663.69 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137567.895838 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131973.22 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 13197.32 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150765.217732 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118775.90 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 11877.59 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162642.807437 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106898.31 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 10689.83 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173332.638171 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96208.48 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 9620.85 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182953.485832 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86587.63 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 8658.76 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191612.248727 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77928.87 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 7792.89 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199405.135332 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70135.98 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 7013.60 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206418.733277 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63122.38 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 6312.24 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212730.971427 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56810.14 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 5681.01 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218411.985762 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51129.13 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 5112.91 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223524.898664 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46016.22 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 4601.62 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228126.520276 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41414.59 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 4141.46 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232267.979726 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37273.14 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3727.31 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235995.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33545.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239595.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29945.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243195.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26345.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246795.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22745.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250395.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19145.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253995.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15545.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257595.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11945.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261195.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8345.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264795.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4745.82 sec) 02:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268395.293231 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1145.82 sec) 02:14:00 [GPUGRID] [rr_sim] 2269541.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998096265.42G/439.78G) 02:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850622.03 02:14:00 [---] pick_jobs_to_run() start 02:14:00 [---] pick_jobs_to_run() end 02:14:00 [---] [rr_sim] end 02:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:14:00 [---] [cpu_sched_debug] preliminary job list: 02:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:14:00 [---] [cpu_sched_debug] final job list: 02:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:14:00 [---] [cpu_sched_debug] enforce_run_list: end 02:14:00 [---] choose_project(): 8040.000000 02:14:00 [iThena.Measurements] doing RPC to get work 02:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427559.09 cpu 2170532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:14:00 [---] Got 0 tasks 02:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:15:00 [---] [rr_sim] doing sim: CPU sched 02:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998068675.07G/439.78G) 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269478.38 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 226947.84 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226947.837790 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042530.54 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 204253.05 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431200.891801 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838277.49 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 183827.75 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615028.640410 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654449.74 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 165444.97 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780473.614159 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1489004.76 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 148900.48 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929374.090533 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340104.29 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 134010.43 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063384.519269 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206093.86 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 120609.39 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183993.905132 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085484.47 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 108548.45 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292542.352409 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976936.03 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 97693.60 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390235.954958 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879242.42 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 87924.24 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478160.197252 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791318.18 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 79131.82 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557292.015316 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712186.36 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 71218.64 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628510.651574 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640967.73 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 64096.77 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692607.424207 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576870.95 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 57687.10 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750294.519576 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519183.86 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 51918.39 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802212.905408 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467265.47 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 46726.55 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848939.452657 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420538.93 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 42053.89 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890993.345181 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378485.03 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 37848.50 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928841.848453 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340636.53 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 34063.65 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962905.501397 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306572.88 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 30657.29 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993562.789047 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275915.59 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 27591.56 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021154.347932 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248324.03 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 24832.40 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045986.750929 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223491.63 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 22349.16 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068335.913626 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201142.46 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 20114.25 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088450.160053 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181028.22 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 18102.82 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106552.981837 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162925.40 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 16292.54 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122845.521443 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146632.86 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 14663.29 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137508.807089 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131969.57 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 13196.96 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150705.764170 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118772.61 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 11877.26 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162583.025542 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106895.35 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 10689.54 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173272.560778 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96205.82 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 9620.58 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182893.142490 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86585.24 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 8658.52 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191551.666031 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77926.71 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 7792.67 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199344.337217 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70134.04 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 7013.40 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206357.741285 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63120.64 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 6312.06 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212669.804947 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56808.57 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 5680.86 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218350.662242 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51127.72 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 5112.77 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223463.433807 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46014.94 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 4601.49 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228064.928216 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41413.45 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 4141.34 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232206.273184 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37272.10 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3727.21 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235933.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33544.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239533.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29944.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243133.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26344.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246733.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22744.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250333.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19144.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253933.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15544.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257533.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11944.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261133.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8344.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264733.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4744.89 sec) 02:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268333.483656 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1144.89 sec) 02:15:00 [GPUGRID] [rr_sim] 2269478.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998068675.07G/439.78G) 02:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850619.29 02:15:00 [---] pick_jobs_to_run() start 02:15:00 [---] pick_jobs_to_run() end 02:15:00 [---] [rr_sim] end 02:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:15:00 [---] [cpu_sched_debug] preliminary job list: 02:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:15:00 [---] [cpu_sched_debug] final job list: 02:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:15:00 [---] [cpu_sched_debug] enforce_run_list: end 02:15:00 [---] choose_project(): 8100.000000 02:15:00 [iThena.Measurements] doing RPC to get work 02:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427499.09 cpu 2170472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:15:00 [---] Got 0 tasks 02:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:16:00 [---] [rr_sim] doing sim: CPU sched 02:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998041084.71G/439.78G) 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269415.64 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 226941.56 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226941.564102 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042474.08 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 204247.41 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431188.971793 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838226.67 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 183822.67 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 615011.638715 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654404.00 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 165440.40 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780452.038945 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488963.60 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 148896.36 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929348.399152 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340067.24 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 134006.72 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063355.123339 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206060.52 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 120606.05 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183961.175106 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085454.47 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 108545.45 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292506.621697 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976909.02 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 97690.90 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390197.523629 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879218.12 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 87921.81 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478119.335368 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791296.31 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 79129.63 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557248.965932 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712166.68 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 71216.67 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628465.633441 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640950.01 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 64095.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692560.634198 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576855.01 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 57685.50 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750246.134880 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519169.51 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 51916.95 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802163.085493 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467252.56 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 46725.26 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848888.341046 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420527.30 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 42052.73 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890941.071043 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378474.57 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 37847.46 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928788.528040 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340627.11 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 34062.71 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962851.239337 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306564.40 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 30656.44 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993507.679505 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275907.96 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 27590.80 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021098.475656 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248317.17 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 24831.72 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045930.192192 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223485.45 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 22348.54 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068278.737075 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201136.90 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 20113.69 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088392.427469 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181023.21 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 18102.32 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106494.748823 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162920.89 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 16292.09 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122786.838043 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146628.80 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 14662.88 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137449.718340 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131965.92 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 13196.59 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150646.310607 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118769.33 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 11876.93 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162523.243648 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106892.40 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 10689.24 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173212.483385 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96203.16 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 9620.32 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182832.799148 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86582.84 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 8658.28 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191491.083335 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77924.56 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 7792.46 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199283.539103 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70132.10 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 7013.21 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206296.749294 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63118.89 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 6311.89 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212608.638466 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56807.00 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 5680.70 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218289.338721 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51126.30 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 5112.63 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223401.968951 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46013.67 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 4601.37 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228003.336157 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41412.30 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 4141.23 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232144.566643 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37271.07 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3727.11 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235871.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33543.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239471.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29943.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243071.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26343.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246671.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22743.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250271.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19143.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253871.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15543.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257471.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11943.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261071.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8343.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264671.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4743.97 sec) 02:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268271.674080 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1143.97 sec) 02:16:00 [GPUGRID] [rr_sim] 2269415.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998041084.71G/439.78G) 02:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850616.55 02:16:00 [---] pick_jobs_to_run() start 02:16:00 [---] pick_jobs_to_run() end 02:16:00 [---] [rr_sim] end 02:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:16:00 [---] [cpu_sched_debug] preliminary job list: 02:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:16:00 [---] [cpu_sched_debug] final job list: 02:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:16:00 [---] [cpu_sched_debug] enforce_run_list: end 02:16:00 [---] choose_project(): 8160.000000 02:16:00 [iThena.Measurements] doing RPC to get work 02:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427439.09 cpu 2170412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:16:00 [---] Got 0 tasks 02:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:17:00 [---] [rr_sim] doing sim: CPU sched 02:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (998013494.35G/439.78G) 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269352.90 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 226935.29 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226935.290413 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042417.61 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 204241.76 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431177.051785 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838175.85 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 183817.59 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614994.637020 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654358.27 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 165435.83 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780430.463731 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488922.44 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 148892.24 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929322.707772 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1340030.20 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 134003.02 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063325.727408 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1206027.18 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 120602.72 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183928.445080 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085424.46 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 108542.45 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292470.890986 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976882.01 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 97688.20 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390159.092300 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879193.81 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 87919.38 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478078.473484 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791274.43 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 79127.44 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557205.916549 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712146.99 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 71214.70 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628420.615307 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640932.29 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 64093.23 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692513.844190 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576839.06 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 57683.91 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750197.750184 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519155.15 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 51915.52 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802113.265579 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467239.64 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 46723.96 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848837.229434 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420515.67 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 42051.57 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890888.796904 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378464.11 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 37846.41 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928735.207627 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340617.70 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 34061.77 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962796.977278 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306555.93 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 30655.59 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993452.569963 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275900.33 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 27590.03 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021042.603380 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248310.30 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 24831.03 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045873.633456 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223479.27 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 22347.93 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068221.560523 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201131.34 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 20113.13 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088334.694884 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181018.21 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 18101.82 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106436.515809 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162916.39 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 16291.64 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122728.154642 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146624.75 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 14662.47 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137390.629591 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131962.27 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 13196.23 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150586.857045 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118766.05 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 11876.60 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162463.461754 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106889.44 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 10688.94 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173152.405992 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96200.50 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 9620.05 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182772.455806 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86580.45 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 8658.04 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191430.500639 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77922.40 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 7792.24 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199222.740988 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70130.16 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 7013.02 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206235.757303 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63117.15 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 6311.71 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212547.471986 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56805.43 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 5680.54 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218228.015201 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51124.89 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 5112.49 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223340.504094 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46012.40 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 4601.24 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227941.744098 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41411.16 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 4141.12 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232082.860101 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37270.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3727.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235809.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33543.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239409.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29943.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243009.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26343.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246609.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22743.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250209.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19143.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253809.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15543.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257409.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11943.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261009.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8343.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264609.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4743.04 sec) 02:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268209.864504 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1143.04 sec) 02:17:00 [GPUGRID] [rr_sim] 2269352.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (998013494.35G/439.78G) 02:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850613.82 02:17:00 [---] pick_jobs_to_run() start 02:17:00 [---] pick_jobs_to_run() end 02:17:00 [---] [rr_sim] end 02:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:17:00 [---] [cpu_sched_debug] preliminary job list: 02:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:17:00 [---] [cpu_sched_debug] final job list: 02:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:17:00 [---] [cpu_sched_debug] enforce_run_list: end 02:17:00 [---] choose_project(): 8220.000000 02:17:00 [iThena.Measurements] doing RPC to get work 02:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427379.09 cpu 2170352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:17:00 [---] Got 0 tasks 02:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:18:00 [---] [rr_sim] doing sim: CPU sched 02:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997985904.00G/439.78G) 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269290.17 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 226929.02 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226929.016725 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042361.15 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 204236.12 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431165.131778 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838125.04 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 183812.50 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614977.635325 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654312.53 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 165431.25 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780408.888518 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488881.28 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 148888.13 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929297.016391 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339993.15 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 133999.32 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063296.331477 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205993.84 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 120599.38 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183895.715054 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085394.45 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 108539.45 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292435.160274 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976855.01 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 97685.50 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390120.660972 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879169.51 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 87916.95 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1478037.611600 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791252.56 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 79125.26 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557162.867165 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712127.30 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 71212.73 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628375.597173 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640914.57 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 64091.46 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692467.054181 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576823.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 57682.31 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750149.365488 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519140.80 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 51914.08 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802063.445665 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467226.72 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 46722.67 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848786.117823 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420504.05 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 42050.40 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890836.522766 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378453.64 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 37845.36 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928681.887214 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340608.28 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 34060.83 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962742.715218 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306547.45 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 30654.75 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993397.460421 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275892.71 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 27589.27 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020986.731104 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248303.44 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 24830.34 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045817.074719 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223473.09 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 22347.31 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068164.383972 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201125.78 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 20112.58 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088276.962300 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181013.20 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 18101.32 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106378.282795 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162911.88 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 16291.19 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122669.471241 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146620.70 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 14662.07 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137331.540842 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131958.63 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 13195.86 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150527.403483 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118762.76 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 11876.28 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162403.679860 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106886.49 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 10688.65 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173092.328599 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96197.84 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 9619.78 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182712.112464 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86578.05 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 8657.81 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191369.917943 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77920.25 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 7792.02 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199161.942874 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70128.22 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 7012.82 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206174.765311 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63115.40 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 6311.54 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212486.305505 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56803.86 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 5680.39 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218166.691680 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51123.48 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 5112.35 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223279.039237 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46011.13 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 4601.11 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227880.152038 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41410.02 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 4141.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232021.153560 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37269.01 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3726.90 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235748.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33542.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239348.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29942.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242948.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26342.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246548.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22742.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250148.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19142.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253748.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15542.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257348.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11942.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260948.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8342.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264548.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4742.11 sec) 02:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268148.054929 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1142.11 sec) 02:18:00 [GPUGRID] [rr_sim] 2269290.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997985904.00G/439.78G) 02:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850611.08 02:18:00 [---] pick_jobs_to_run() start 02:18:00 [---] pick_jobs_to_run() end 02:18:00 [---] [rr_sim] end 02:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:18:00 [---] [cpu_sched_debug] preliminary job list: 02:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:18:00 [---] [cpu_sched_debug] final job list: 02:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:18:00 [---] [cpu_sched_debug] enforce_run_list: end 02:18:00 [---] choose_project(): 8280.000000 02:18:00 [iThena.Measurements] doing RPC to get work 02:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427319.09 cpu 2170292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:18:00 [---] Got 0 tasks 02:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:19:00 [---] [rr_sim] doing sim: CPU sched 02:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997958313.64G/439.78G) 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269227.43 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 226922.74 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226922.743037 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042304.69 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 204230.47 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431153.211770 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838074.22 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 183807.42 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614960.633630 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654266.80 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 165426.68 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780387.313304 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488840.12 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 148884.01 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929271.325010 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339956.11 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 133995.61 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063266.935546 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205960.49 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 120596.05 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183862.985029 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085364.45 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 108536.44 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292399.429563 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976828.00 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 97682.80 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390082.229643 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879145.20 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 87914.52 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477996.749716 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791230.68 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 79123.07 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557119.817781 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712107.61 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 71210.76 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628330.579040 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640896.85 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 64089.69 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692420.264173 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576807.17 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 57680.72 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750100.980792 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519126.45 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 51912.64 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802013.625750 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467213.80 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 46721.38 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848735.006212 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420492.42 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 42049.24 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890784.248628 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378443.18 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 37844.32 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928628.566802 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340598.86 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 34059.89 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962688.453158 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306538.98 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 30653.90 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993342.350880 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275885.08 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 27588.51 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020930.858828 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248296.57 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 24829.66 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045760.515983 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223466.91 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 22346.69 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068107.207421 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201120.22 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 20112.02 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088219.229716 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181008.20 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 18100.82 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106320.049781 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162907.38 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 16290.74 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122610.787840 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146616.64 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 14661.66 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137272.452093 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131954.98 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 13195.50 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150467.949921 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118759.48 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 11875.95 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162343.897965 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106883.53 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 10688.35 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173032.251206 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96195.18 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 9619.52 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182651.769122 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86575.66 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 8657.57 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191309.335247 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77918.10 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 7791.81 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199101.144759 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70126.29 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 7012.63 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206113.773320 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63113.66 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 6311.37 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212425.139025 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56802.29 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 5680.23 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218105.368159 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51122.06 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 5112.21 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223217.574380 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46009.86 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 4600.99 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227818.559979 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41408.87 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 4140.89 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231959.447018 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37267.98 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3726.80 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235686.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33541.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239286.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29941.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242886.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26341.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246486.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22741.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250086.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19141.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253686.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15541.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257286.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11941.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260886.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8341.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264486.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4741.19 sec) 02:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268086.245353 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1141.19 sec) 02:19:00 [GPUGRID] [rr_sim] 2269227.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997958313.64G/439.78G) 02:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850608.34 02:19:00 [---] pick_jobs_to_run() start 02:19:00 [---] pick_jobs_to_run() end 02:19:00 [---] [rr_sim] end 02:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:19:00 [---] [cpu_sched_debug] preliminary job list: 02:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:19:00 [---] [cpu_sched_debug] final job list: 02:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:19:00 [---] [cpu_sched_debug] enforce_run_list: end 02:19:00 [---] choose_project(): 8340.000000 02:19:00 [iThena.Measurements] doing RPC to get work 02:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 600059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427259.09 cpu 2170232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:19:00 [---] Got 0 tasks 02:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:20:00 [---] [rr_sim] doing sim: CPU sched 02:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997930723.28G/439.78G) 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269164.69 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 226916.47 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226916.469349 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042248.22 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 204224.82 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431141.291762 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1838023.40 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 183802.34 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614943.631935 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654221.06 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 165422.11 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780365.738090 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488798.96 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 148879.90 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929245.633630 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339919.06 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 133991.91 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063237.539615 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205927.15 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 120592.72 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183830.255003 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085334.44 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 108533.44 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292363.698851 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976800.99 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 97680.10 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390043.798315 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879120.90 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 87912.09 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477955.887832 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791208.81 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 79120.88 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557076.768397 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712087.93 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 71208.79 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628285.560906 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640879.13 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 64087.91 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692373.474164 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576791.22 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 57679.12 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750052.596097 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519112.10 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 51911.21 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801963.805836 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467200.89 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 46720.09 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848683.894601 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420480.80 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 42048.08 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890731.974489 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378432.72 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 37843.27 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928575.246389 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340589.45 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 34058.94 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962634.191099 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306530.50 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 30653.05 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993287.241338 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275877.45 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 27587.75 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020874.986553 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248289.71 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 24828.97 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045703.957246 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223460.74 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 22346.07 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068050.030870 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201114.66 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 20111.47 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088161.497132 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181003.20 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 18100.32 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106261.816767 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162902.88 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 16290.29 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122552.104439 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146612.59 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 14661.26 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137213.363344 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131951.33 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 13195.13 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150408.496358 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118756.20 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 11875.62 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162284.116071 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106880.58 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 10688.06 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172972.173813 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96192.52 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 9619.25 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182591.425780 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86573.27 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 8657.33 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191248.752551 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77915.94 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 7791.59 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199040.346644 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70124.35 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 7012.43 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206052.781329 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63111.91 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 6311.19 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212363.972544 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56800.72 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 5680.07 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218044.044639 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51120.65 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 5112.06 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223156.109523 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46008.58 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 4600.86 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227756.967920 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41407.73 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 4140.77 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231897.740476 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37266.95 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3726.70 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235624.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33540.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239224.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29940.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242824.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26340.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246424.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22740.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250024.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19140.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253624.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15540.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257224.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11940.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260824.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8340.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264424.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4740.26 sec) 02:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2268024.435778 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1140.26 sec) 02:20:00 [GPUGRID] [rr_sim] 2269164.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997930723.28G/439.78G) 02:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850605.61 02:20:00 [---] pick_jobs_to_run() start 02:20:00 [---] pick_jobs_to_run() end 02:20:00 [---] [rr_sim] end 02:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:20:00 [---] [cpu_sched_debug] preliminary job list: 02:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:20:00 [---] [cpu_sched_debug] final job list: 02:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:20:00 [---] [cpu_sched_debug] enforce_run_list: end 02:20:00 [---] choose_project(): 8400.000000 02:20:00 [iThena.Measurements] doing RPC to get work 02:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427199.09 cpu 2170172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:20:00 [---] Got 0 tasks 02:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:21:00 [---] [rr_sim] doing sim: CPU sched 02:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997903132.93G/439.78G) 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269101.96 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 226910.20 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226910.195660 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042191.76 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 204219.18 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431129.371755 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837972.58 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 183797.26 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614926.630240 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654175.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 165417.53 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780344.162876 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488757.79 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 148875.78 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929219.942249 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339882.01 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 133988.20 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063208.143685 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205893.81 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 120589.38 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183797.524977 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085304.43 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 108530.44 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292327.968139 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976773.99 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 97677.40 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1390005.366986 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879096.59 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 87909.66 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477915.025948 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791186.93 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 79118.69 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1557033.719014 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712068.24 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 71206.82 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628240.542773 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640861.41 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 64086.14 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692326.684156 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576775.27 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 57677.53 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750004.211401 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519097.75 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 51909.77 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801913.985921 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467187.97 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 46718.80 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848632.782989 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420469.17 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 42046.92 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890679.700351 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378422.26 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 37842.23 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928521.925976 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340580.03 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 34058.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962579.929039 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306522.03 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 30652.20 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993232.131796 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275869.82 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 27586.98 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020819.114277 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248282.84 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 24828.28 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045647.398509 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223454.56 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 22345.46 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067992.854319 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201109.10 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 20110.91 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088103.764548 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180998.19 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 18099.82 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106203.583753 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162898.37 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 16289.84 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122493.421038 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146608.54 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 14660.85 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137154.274595 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131947.68 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 13194.77 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150349.042796 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118752.91 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 11875.29 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162224.334177 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106877.62 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 10687.76 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172912.096420 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96189.86 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 9618.99 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182531.082438 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86570.87 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 8657.09 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191188.169855 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77913.79 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 7791.38 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198979.548530 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70122.41 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 7012.24 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205991.789337 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63110.17 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 6311.02 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212302.806064 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56799.15 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 5679.92 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217982.721118 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51119.24 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 5111.92 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223094.644667 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46007.31 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 4600.73 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227695.375860 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41406.58 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 4140.66 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231836.033935 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37265.92 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3726.59 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235562.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33539.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239162.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29939.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242762.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26339.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246362.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22739.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249962.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19139.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253562.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15539.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257162.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11939.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260762.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8339.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264362.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4739.33 sec) 02:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267962.626202 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1139.33 sec) 02:21:00 [GPUGRID] [rr_sim] 2269101.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997903132.93G/439.78G) 02:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850602.87 02:21:00 [---] pick_jobs_to_run() start 02:21:00 [---] pick_jobs_to_run() end 02:21:00 [---] [rr_sim] end 02:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:21:00 [---] [cpu_sched_debug] preliminary job list: 02:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:21:00 [---] [cpu_sched_debug] final job list: 02:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:21:00 [---] [cpu_sched_debug] enforce_run_list: end 02:21:00 [---] choose_project(): 8460.000000 02:21:00 [iThena.Measurements] doing RPC to get work 02:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427139.09 cpu 2170112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:21:00 [---] Got 0 tasks 02:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:22:00 [---] [rr_sim] doing sim: CPU sched 02:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997875542.57G/439.78G) 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2269039.22 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 226903.92 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226903.921972 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042135.30 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 204213.53 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431117.451747 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837921.77 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 183792.18 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614909.628545 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654129.59 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 165412.96 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780322.587663 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488716.63 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 148871.66 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929194.250869 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339844.97 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 133984.50 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063178.747754 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205860.47 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 120586.05 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183764.794951 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085274.42 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 108527.44 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292292.237428 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976746.98 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 97674.70 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389966.935657 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879072.28 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 87907.23 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477874.164064 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791165.06 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 79116.51 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556990.669630 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712048.55 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 71204.86 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628195.524639 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640843.70 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 64084.37 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692279.894147 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576759.33 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 57675.93 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749955.826705 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519083.39 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 51908.34 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801864.166007 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467175.05 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 46717.51 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848581.671378 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420457.55 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 42045.75 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890627.426213 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378411.79 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 37841.18 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928468.605564 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340570.61 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 34057.06 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962525.666980 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306513.55 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 30651.36 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993177.022254 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275862.20 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 27586.22 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020763.242001 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248275.98 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 24827.60 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045590.839773 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223448.38 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 22344.84 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067935.677768 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201103.54 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 20110.35 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088046.031963 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180993.19 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 18099.32 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106145.350739 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162893.87 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 16289.39 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122434.737638 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146604.48 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 14660.45 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137095.185846 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131944.03 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 13194.40 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150289.589234 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118749.63 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 11874.96 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162164.552283 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106874.67 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 10687.47 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172852.019027 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96187.20 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 9618.72 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182470.739096 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86568.48 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 8656.85 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191127.587159 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77911.63 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 7791.16 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198918.750415 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70120.47 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 7012.05 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205930.797346 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63108.42 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 6310.84 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212241.639584 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56797.58 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 5679.76 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217921.397597 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51117.82 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 5111.78 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223033.179810 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46006.04 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 4600.60 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227633.783801 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41405.44 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 4140.54 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231774.327393 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37264.89 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3726.49 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235500.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33538.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239100.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29938.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242700.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26338.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246300.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22738.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249900.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19138.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253500.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15538.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257100.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11938.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260700.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8338.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264300.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4738.40 sec) 02:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267900.816626 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1138.40 sec) 02:22:00 [GPUGRID] [rr_sim] 2269039.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997875542.57G/439.78G) 02:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850600.13 02:22:00 [---] pick_jobs_to_run() start 02:22:00 [---] pick_jobs_to_run() end 02:22:00 [---] [rr_sim] end 02:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:22:00 [---] [cpu_sched_debug] preliminary job list: 02:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:22:00 [---] [cpu_sched_debug] final job list: 02:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:22:00 [---] [cpu_sched_debug] enforce_run_list: end 02:22:00 [---] choose_project(): 8520.000000 02:22:00 [iThena.Measurements] doing RPC to get work 02:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427079.09 cpu 2170052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2170052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2170052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:22:00 [---] Got 0 tasks 02:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:23:00 [---] [rr_sim] doing sim: CPU sched 02:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997847952.22G/439.78G) 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268976.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 226897.65 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226897.648284 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042078.83 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 204207.88 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431105.531740 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837870.95 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 183787.10 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614892.626850 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654083.86 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 165408.39 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780301.012449 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488675.47 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 148867.55 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929168.559488 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339807.92 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 133980.79 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063149.351823 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205827.13 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 120582.71 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183732.064925 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085244.42 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 108524.44 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292256.506716 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976719.98 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 97672.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389928.504329 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879047.98 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 87904.80 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477833.302180 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791143.18 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 79114.32 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556947.620246 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712028.86 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 71202.89 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628150.506505 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640825.98 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 64082.60 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692233.104139 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576743.38 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 57674.34 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749907.442009 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519069.04 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 51906.90 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801814.346092 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467162.14 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 46716.21 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848530.559767 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420445.92 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 42044.59 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890575.152074 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378401.33 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 37840.13 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928415.285151 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340561.20 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 34056.12 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962471.404920 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306505.08 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 30650.51 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993121.912712 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275854.57 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 27585.46 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020707.369725 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248269.11 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 24826.91 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045534.281036 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223442.20 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 22344.22 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067878.501217 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201097.98 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 20109.80 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087988.299379 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180988.18 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 18098.82 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106087.117725 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162889.37 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 16288.94 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122376.054237 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146600.43 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 14660.04 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137036.097097 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131940.39 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 13194.04 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150230.135671 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118746.35 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 11874.63 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162104.770388 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106871.71 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 10687.17 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172791.941633 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96184.54 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 9618.45 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182410.395754 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86566.09 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 8656.61 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191067.004463 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77909.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 7790.95 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198857.952301 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70118.53 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 7011.85 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205869.805354 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63106.68 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 6310.67 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212180.473103 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56796.01 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 5679.60 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217860.074077 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51116.41 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 5111.64 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222971.714953 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46004.77 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 4600.48 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227572.191742 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41404.29 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 4140.43 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231712.620852 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37263.86 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3726.39 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235439.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33537.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239039.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29937.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242639.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26337.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246239.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22737.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249839.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19137.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253439.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15537.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257039.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11937.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260639.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8337.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264239.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4737.48 sec) 02:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267839.007051 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1137.48 sec) 02:23:00 [GPUGRID] [rr_sim] 2268976.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997847952.22G/439.78G) 02:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850597.40 02:23:00 [---] pick_jobs_to_run() start 02:23:00 [---] pick_jobs_to_run() end 02:23:00 [---] [rr_sim] end 02:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:23:00 [---] [cpu_sched_debug] preliminary job list: 02:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:23:00 [---] [cpu_sched_debug] final job list: 02:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:23:00 [---] [cpu_sched_debug] enforce_run_list: end 02:23:00 [---] choose_project(): 8580.000000 02:23:00 [iThena.Measurements] doing RPC to get work 02:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 427019.09 cpu 2169992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:23:00 [---] Got 0 tasks 02:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:24:00 [---] [rr_sim] doing sim: CPU sched 02:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997820361.86G/439.78G) 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268913.75 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 226891.37 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226891.374596 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2042022.37 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 204202.24 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431093.611732 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837820.13 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 183782.01 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614875.625155 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1654038.12 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 165403.81 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780279.437235 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488634.31 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 148863.43 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929142.868107 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339770.88 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 133977.09 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063119.955892 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205793.79 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 120579.38 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183699.334899 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085214.41 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 108521.44 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292220.776005 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976692.97 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 97669.30 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389890.073000 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879023.67 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 87902.37 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477792.440296 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791121.31 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 79112.13 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556904.570862 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712009.18 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 71200.92 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628105.488372 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640808.26 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 64080.83 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692186.314130 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576727.43 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 57672.74 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749859.057313 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519054.69 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 51905.47 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801764.526178 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467149.22 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 46714.92 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848479.448156 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420434.30 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 42043.43 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890522.877936 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378390.87 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 37839.09 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928361.964738 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340551.78 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 34055.18 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962417.142860 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306496.60 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 30649.66 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993066.803170 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275846.94 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 27584.69 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020651.497449 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248262.25 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 24826.22 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045477.722300 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223436.02 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 22343.60 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067821.324666 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201092.42 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 20109.24 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087930.566795 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180983.18 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 18098.32 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106028.884711 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162884.86 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 16288.49 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122317.370836 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146596.38 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 14659.64 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136977.008348 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131936.74 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 13193.67 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150170.682109 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118743.06 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 11874.31 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162044.988494 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106868.76 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 10686.88 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172731.864240 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96181.88 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 9618.19 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182350.052412 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86563.69 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 8656.37 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191006.421767 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77907.32 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 7790.73 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198797.154186 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70116.59 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 7011.66 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205808.813363 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63104.93 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 6310.49 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212119.306623 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56794.44 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 5679.44 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217798.750556 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51115.00 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 5111.50 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222910.250096 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46003.50 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 4600.35 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227510.599683 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41403.15 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 4140.31 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231650.914310 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37262.83 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3726.28 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235377.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33536.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238977.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29936.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242577.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26336.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246177.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22736.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249777.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19136.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253377.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15536.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256977.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11936.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260577.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8336.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264177.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4736.55 sec) 02:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267777.197475 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1136.55 sec) 02:24:00 [GPUGRID] [rr_sim] 2268913.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997820361.86G/439.78G) 02:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850594.66 02:24:00 [---] pick_jobs_to_run() start 02:24:00 [---] pick_jobs_to_run() end 02:24:00 [---] [rr_sim] end 02:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:24:00 [---] [cpu_sched_debug] preliminary job list: 02:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:24:00 [---] [cpu_sched_debug] final job list: 02:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:24:00 [---] [cpu_sched_debug] enforce_run_list: end 02:24:00 [---] choose_project(): 8640.000000 02:24:00 [iThena.Measurements] doing RPC to get work 02:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426959.09 cpu 2169932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:24:00 [---] Got 0 tasks 02:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:25:00 [---] [rr_sim] doing sim: CPU sched 02:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997792771.50G/439.78G) 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268851.01 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 226885.10 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226885.100908 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041965.91 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 204196.59 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431081.691724 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837769.32 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 183776.93 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614858.623460 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653992.39 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 165399.24 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780257.862021 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488593.15 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 148859.31 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929117.176727 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339733.83 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 133973.38 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063090.559962 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205760.45 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 120576.04 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183666.604873 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085184.40 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 108518.44 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292185.045293 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976665.96 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 97666.60 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389851.641672 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878999.37 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 87899.94 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477751.578412 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791099.43 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 79109.94 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556861.521478 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711989.49 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 71198.95 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628060.470238 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640790.54 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 64079.05 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692139.524122 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576711.48 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 57671.15 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749810.672617 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519040.34 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 51904.03 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801714.706263 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467136.30 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 46713.63 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848428.336545 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420422.67 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 42042.27 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890470.603798 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378380.41 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 37838.04 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928308.644325 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340542.36 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 34054.24 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962362.880801 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306488.13 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 30648.81 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993011.693628 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275839.32 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 27583.93 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020595.625173 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248255.38 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 24825.54 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045421.163563 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223429.85 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 22342.98 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067764.148114 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201086.86 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 20108.69 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087872.834211 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180978.17 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 18097.82 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105970.651697 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162880.36 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 16288.04 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122258.687435 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146592.32 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 14659.23 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136917.919599 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131933.09 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 13193.31 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150111.228547 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118739.78 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 11873.98 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161985.206600 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106865.80 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 10686.58 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172671.786847 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96179.22 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 9617.92 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182289.709070 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86561.30 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 8656.13 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190945.839071 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77905.17 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 7790.52 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198736.356071 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70114.65 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 7011.47 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205747.821372 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63103.19 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 6310.32 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212058.140142 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56792.87 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 5679.29 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217737.427036 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51113.58 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 5111.36 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222848.785240 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46002.22 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 4600.22 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227449.007623 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41402.00 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 4140.20 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231589.207769 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37261.80 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3726.18 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235315.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33535.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238915.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29935.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242515.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26335.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246115.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22735.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249715.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19135.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253315.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15535.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256915.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11935.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260515.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8335.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264115.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4735.62 sec) 02:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267715.387899 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1135.62 sec) 02:25:00 [GPUGRID] [rr_sim] 2268851.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997792771.50G/439.78G) 02:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850591.92 02:25:00 [---] pick_jobs_to_run() start 02:25:00 [---] pick_jobs_to_run() end 02:25:00 [---] [rr_sim] end 02:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:25:00 [---] [cpu_sched_debug] preliminary job list: 02:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:25:00 [---] [cpu_sched_debug] final job list: 02:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:25:00 [---] [cpu_sched_debug] enforce_run_list: end 02:25:00 [---] choose_project(): 8700.000000 02:25:00 [iThena.Measurements] doing RPC to get work 02:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426899.09 cpu 2169872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:25:00 [---] Got 0 tasks 02:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:26:00 [---] [rr_sim] doing sim: CPU sched 02:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997765181.15G/439.78G) 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268788.27 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 226878.83 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226878.827219 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041909.44 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 204190.94 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431069.771717 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837718.50 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 183771.85 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614841.621765 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653946.65 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 165394.67 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780236.286807 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488551.99 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 148855.20 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929091.485346 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339696.79 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 133969.68 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063061.164031 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205727.11 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 120572.71 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183633.874847 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085154.40 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 108515.44 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292149.314582 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976638.96 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 97663.90 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389813.210343 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878975.06 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 87897.51 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477710.716528 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791077.56 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 79107.76 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556818.472095 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711969.80 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 71196.98 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1628015.452105 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640772.82 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 64077.28 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692092.734114 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576695.54 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 57669.55 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749762.287922 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519025.98 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 51902.60 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801664.886349 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467123.39 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 46712.34 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848377.224933 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420411.05 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 42041.10 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890418.329659 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378369.94 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 37836.99 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928255.323913 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340532.95 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 34053.29 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962308.618741 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306479.65 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 30647.97 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992956.584086 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275831.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 27583.17 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020539.752897 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248248.52 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 24824.85 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045364.604827 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223423.67 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 22342.37 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067706.971563 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201081.30 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 20108.13 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087815.101626 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180973.17 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 18097.32 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105912.418683 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162875.85 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 16287.59 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122200.004034 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146588.27 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 14658.83 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136858.830850 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131929.44 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 13192.94 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150051.774985 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118736.50 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 11873.65 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161925.424705 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106862.85 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 10686.28 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172611.709454 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96176.56 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 9617.66 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182229.365728 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86558.91 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 8655.89 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190885.256375 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77903.02 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 7790.30 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198675.557957 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70112.71 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 7011.27 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205686.829380 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63101.44 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 6310.14 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211996.973662 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56791.30 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 5679.13 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217676.103515 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51112.17 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 5111.22 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222787.320383 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46000.95 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 4600.10 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227387.415564 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41400.86 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 4140.09 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231527.501227 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37260.77 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3726.08 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235253.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33534.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238853.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29934.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242453.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26334.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246053.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22734.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249653.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19134.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253253.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15534.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256853.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11934.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260453.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8334.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264053.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4734.69 sec) 02:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267653.578324 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1134.69 sec) 02:26:00 [GPUGRID] [rr_sim] 2268788.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997765181.15G/439.78G) 02:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850589.18 02:26:00 [---] pick_jobs_to_run() start 02:26:00 [---] pick_jobs_to_run() end 02:26:00 [---] [rr_sim] end 02:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:26:00 [---] [cpu_sched_debug] preliminary job list: 02:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:26:00 [---] [cpu_sched_debug] final job list: 02:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:26:00 [---] [cpu_sched_debug] enforce_run_list: end 02:26:00 [---] choose_project(): 8760.000000 02:26:00 [iThena.Measurements] doing RPC to get work 02:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426839.09 cpu 2169812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:26:00 [---] Got 0 tasks 02:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:27:00 [---] [rr_sim] doing sim: CPU sched 02:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997737590.79G/439.78G) 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268725.54 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 226872.55 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226872.553531 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041852.98 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 204185.30 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431057.851709 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837667.68 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 183766.77 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614824.620069 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653900.92 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 165390.09 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780214.711594 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488510.82 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 148851.08 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929065.793965 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339659.74 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 133965.97 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063031.768100 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205693.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 120569.38 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183601.144821 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085124.39 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 108512.44 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292113.583870 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976611.95 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 97661.20 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389774.779014 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878950.76 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 87895.08 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477669.854644 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791055.68 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 79105.57 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556775.422711 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711950.11 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 71195.01 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627970.433971 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640755.10 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 64075.51 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692045.944105 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576679.59 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 57667.96 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749713.903226 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519011.63 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 51901.16 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801615.066434 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467110.47 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 46711.05 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848326.113322 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420399.42 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 42039.94 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890366.055521 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378359.48 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 37835.95 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928202.003500 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340523.53 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 34052.35 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962254.356681 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306471.18 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 30647.12 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992901.474544 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275824.06 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 27582.41 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020483.880621 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248241.65 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 24824.17 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045308.046090 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223417.49 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 22341.75 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067649.795012 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201075.74 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 20107.57 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087757.369042 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180968.17 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 18096.82 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105854.185669 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162871.35 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 16287.13 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122141.320633 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146584.21 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 14658.42 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136799.742101 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131925.79 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 13192.58 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149992.321422 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118733.21 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 11873.32 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161865.642811 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106859.89 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 10685.99 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172551.632061 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96173.90 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 9617.39 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182169.022386 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86556.51 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 8655.65 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190824.673679 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77900.86 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 7790.09 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198614.759842 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70110.78 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 7011.08 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205625.837389 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63099.70 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 6309.97 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211935.807181 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56789.73 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 5678.97 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217614.779994 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51110.76 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 5111.08 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222725.855526 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45999.68 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 4599.97 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227325.823505 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41399.71 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 4139.97 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231465.794685 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37259.74 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3725.97 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235191.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33533.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238791.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29933.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242391.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26333.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245991.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22733.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249591.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19133.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253191.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15533.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256791.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11933.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260391.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8333.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263991.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4733.77 sec) 02:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267591.768748 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1133.77 sec) 02:27:00 [GPUGRID] [rr_sim] 2268725.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997737590.79G/439.78G) 02:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850586.45 02:27:00 [---] pick_jobs_to_run() start 02:27:00 [---] pick_jobs_to_run() end 02:27:00 [---] [rr_sim] end 02:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:27:00 [---] [cpu_sched_debug] preliminary job list: 02:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:27:00 [---] [cpu_sched_debug] final job list: 02:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:27:00 [---] [cpu_sched_debug] enforce_run_list: end 02:27:00 [---] choose_project(): 8820.000000 02:27:00 [iThena.Measurements] doing RPC to get work 02:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426779.09 cpu 2169752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:27:00 [---] Got 0 tasks 02:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:28:00 [---] [rr_sim] doing sim: CPU sched 02:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997710000.43G/439.78G) 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268662.80 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 226866.28 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226866.279843 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041796.52 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 204179.65 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431045.931702 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837616.87 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 183761.69 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614807.618374 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653855.18 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 165385.52 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780193.136380 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488469.66 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 148846.97 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929040.102585 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339622.70 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 133962.27 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1063002.372169 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205660.43 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 120566.04 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183568.414795 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085094.38 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 108509.44 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292077.853159 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976584.95 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 97658.49 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389736.347686 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878926.45 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 87892.65 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477628.992760 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791033.81 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 79103.38 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556732.373327 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711930.43 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 71193.04 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627925.415837 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640737.38 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 64073.74 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691999.154097 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576663.64 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 57666.36 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749665.518530 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518997.28 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 51899.73 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801565.246520 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467097.55 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 46709.76 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848275.001711 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420387.80 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 42038.78 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890313.781383 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378349.02 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 37834.90 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928148.683087 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340514.12 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 34051.41 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962200.094622 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306462.70 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 30646.27 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992846.365002 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275816.43 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 27581.64 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020428.008345 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248234.79 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 24823.48 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045251.487353 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223411.31 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 22341.13 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067592.618461 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201070.18 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 20107.02 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087699.636458 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180963.16 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 18096.32 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105795.952655 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162866.85 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 16286.68 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122082.637233 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146580.16 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 14658.02 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136740.653352 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131922.15 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 13192.21 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149932.867860 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118729.93 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 11872.99 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161805.860917 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106856.94 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 10685.69 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172491.554668 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96171.24 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 9617.12 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182108.679044 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86554.12 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 8655.41 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190764.090983 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77898.71 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 7789.87 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198553.961727 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70108.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 7010.88 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205564.845398 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63097.95 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 6309.80 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211874.640701 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56788.16 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 5678.82 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217553.456474 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51109.34 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 5110.93 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222664.390669 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45998.41 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 4599.84 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227264.231445 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41398.57 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 4139.86 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231404.088144 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37258.71 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3725.87 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235129.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33532.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238729.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29932.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242329.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26332.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245929.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22732.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249529.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19132.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253129.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15532.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256729.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11932.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260329.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8332.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263929.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4732.84 sec) 02:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267529.959172 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1132.84 sec) 02:28:00 [GPUGRID] [rr_sim] 2268662.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997710000.43G/439.78G) 02:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850583.71 02:28:00 [---] pick_jobs_to_run() start 02:28:00 [---] pick_jobs_to_run() end 02:28:00 [---] [rr_sim] end 02:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:28:00 [---] [cpu_sched_debug] preliminary job list: 02:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:28:00 [---] [cpu_sched_debug] final job list: 02:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:28:00 [---] [cpu_sched_debug] enforce_run_list: end 02:28:00 [---] choose_project(): 8880.000000 02:28:00 [iThena.Measurements] doing RPC to get work 02:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426719.09 cpu 2169692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:28:00 [---] Got 0 tasks 02:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:29:00 [---] [rr_sim] doing sim: CPU sched 02:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997682410.08G/439.78G) 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268600.06 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 226860.01 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226860.006155 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041740.06 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 204174.01 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431034.011694 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837566.05 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 183756.60 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614790.616679 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653809.44 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 165380.94 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780171.561166 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488428.50 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 148842.85 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929014.411204 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339585.65 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 133958.57 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062972.976239 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205627.09 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 120562.71 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183535.684769 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085064.38 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 108506.44 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292042.122447 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976557.94 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 97655.79 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389697.916357 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878902.15 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 87890.21 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477588.130876 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791011.93 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 79101.19 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556689.323943 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711910.74 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 71191.07 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627880.397704 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640719.66 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 64071.97 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691952.364088 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576647.70 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 57664.77 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749617.133834 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518982.93 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 51898.29 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801515.426605 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467084.63 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 46708.46 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848223.890100 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420376.17 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 42037.62 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890261.507244 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378338.55 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 37833.86 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928095.362675 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340504.70 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 34050.47 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962145.832562 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306454.23 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 30645.42 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992791.255460 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275808.81 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 27580.88 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020372.136069 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248227.93 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 24822.79 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045194.928617 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223405.13 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 22340.51 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067535.441910 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201064.62 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 20106.46 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087641.903874 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180958.16 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 18095.82 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105737.719641 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162862.34 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 16286.23 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122023.953832 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146576.11 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 14657.61 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136681.564603 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131918.50 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 13191.85 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149873.414298 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118726.65 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 11872.66 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161746.079023 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106853.98 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 10685.40 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172431.477275 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96168.58 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 9616.86 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182048.335702 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86551.73 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 8655.17 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190703.508287 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77896.55 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 7789.66 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198493.163613 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70106.90 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 7010.69 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205503.853406 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63096.21 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 6309.62 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211813.474220 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56786.59 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 5678.66 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217492.132953 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51107.93 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 5110.79 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222602.925813 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45997.14 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 4599.71 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227202.639386 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41397.42 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 4139.74 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231342.381602 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37257.68 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3725.77 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235068.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33531.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238668.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29931.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242268.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26331.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245868.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22731.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249468.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19131.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253068.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15531.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256668.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11931.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260268.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8331.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263868.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4731.91 sec) 02:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267468.149597 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1131.91 sec) 02:29:00 [GPUGRID] [rr_sim] 2268600.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997682410.08G/439.78G) 02:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850580.97 02:29:00 [---] pick_jobs_to_run() start 02:29:00 [---] pick_jobs_to_run() end 02:29:00 [---] [rr_sim] end 02:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:29:00 [---] [cpu_sched_debug] preliminary job list: 02:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:29:00 [---] [cpu_sched_debug] final job list: 02:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:29:00 [---] [cpu_sched_debug] enforce_run_list: end 02:29:00 [---] choose_project(): 8940.000000 02:29:00 [iThena.Measurements] doing RPC to get work 02:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426659.09 cpu 2169632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:29:00 [---] Got 0 tasks 02:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:36:00 [---] [rr_sim] doing sim: CPU sched 02:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997682410.08G/439.78G) 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268600.06 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 226860.01 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226860.006155 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041740.06 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 204174.01 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431034.011694 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837566.05 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 183756.60 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614790.616679 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653809.44 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 165380.94 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780171.561166 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488428.50 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 148842.85 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 929014.411204 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339585.65 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 133958.57 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062972.976239 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205627.09 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 120562.71 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183535.684769 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085064.38 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 108506.44 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292042.122447 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976557.94 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 97655.79 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389697.916357 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878902.15 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 87890.21 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477588.130876 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791011.93 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 79101.19 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556689.323943 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711910.74 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 71191.07 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627880.397704 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640719.66 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 64071.97 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691952.364088 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576647.70 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 57664.77 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749617.133834 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518982.93 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 51898.29 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801515.426605 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467084.63 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 46708.46 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848223.890100 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420376.17 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 42037.62 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890261.507244 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378338.55 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 37833.86 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928095.362675 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340504.70 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 34050.47 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962145.832562 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306454.23 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 30645.42 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992791.255460 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275808.81 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 27580.88 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020372.136069 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248227.93 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 24822.79 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045194.928617 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223405.13 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 22340.51 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067535.441910 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201064.62 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 20106.46 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087641.903874 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180958.16 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 18095.82 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105737.719641 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162862.34 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 16286.23 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122023.953832 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146576.11 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 14657.61 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136681.564603 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131918.50 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 13191.85 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149873.414298 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118726.65 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 11872.66 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161746.079023 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106853.98 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 10685.40 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172431.477275 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96168.58 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 9616.86 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182048.335702 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86551.73 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 8655.17 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190703.508287 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77896.55 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 7789.66 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198493.163613 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70106.90 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 7010.69 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205503.853406 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63096.21 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 6309.62 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211813.474220 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56786.59 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 5678.66 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217492.132953 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51107.93 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 5110.79 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222602.925813 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45997.14 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 4599.71 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227202.639386 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41397.42 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 4139.74 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231342.381602 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37257.68 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3725.77 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235068.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33531.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238668.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29931.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242268.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26331.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245868.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22731.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249468.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19131.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253068.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15531.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256668.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11931.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260268.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8331.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263868.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4731.91 sec) 02:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267468.149597 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1131.91 sec) 02:36:00 [GPUGRID] [rr_sim] 2268600.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997682410.08G/439.78G) 02:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851000.97 02:36:00 [---] pick_jobs_to_run() start 02:36:00 [---] pick_jobs_to_run() end 02:36:00 [---] [rr_sim] end 02:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:36:00 [---] [cpu_sched_debug] preliminary job list: 02:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:36:00 [---] [cpu_sched_debug] final job list: 02:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:36:00 [---] [cpu_sched_debug] enforce_run_list: end 02:36:00 [---] choose_project(): 9360.000000 02:36:00 [iThena.Measurements] doing RPC to get work 02:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 599039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426239.09 cpu 2169632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:36:00 [---] Got 0 tasks 02:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:37:00 [---] [rr_sim] doing sim: CPU sched 02:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997654819.72G/439.78G) 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268537.32 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 226853.73 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226853.732467 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041683.59 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 204168.36 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431022.091686 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837515.23 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 183751.52 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614773.614984 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653763.71 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 165376.37 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780149.985952 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488387.34 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 148838.73 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928988.719824 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339548.60 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 133954.86 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062943.580308 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205593.74 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 120559.37 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183502.954744 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085034.37 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 108503.44 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292006.391736 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976530.93 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 97653.09 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389659.485029 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878877.84 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 87887.78 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477547.268992 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790990.06 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 79099.01 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556646.274560 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711891.05 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 71189.11 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627835.379570 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640701.95 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 64070.19 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691905.574080 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576631.75 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 57663.18 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749568.749138 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518968.58 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 51896.86 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801465.606691 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467071.72 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 46707.17 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848172.778488 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420364.55 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 42036.45 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890209.233106 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378328.09 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 37832.81 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928042.042262 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340495.28 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 34049.53 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962091.570502 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306445.75 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 30644.58 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992736.145919 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275801.18 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 27580.12 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020316.263793 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248221.06 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 24822.11 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045138.369880 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223398.95 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 22339.90 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067478.265359 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201059.06 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 20105.91 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087584.171289 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180953.15 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 18095.32 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105679.486627 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162857.84 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 16285.78 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121965.270431 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146572.05 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 14657.21 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136622.475854 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131914.85 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 13191.48 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149813.960735 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118723.36 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 11872.34 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161686.297128 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106851.03 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 10685.10 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172371.399882 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96165.92 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 9616.59 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181987.992360 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86549.33 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 8654.93 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190642.925591 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77894.40 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 7789.44 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198432.365498 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70104.96 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 7010.50 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205442.861415 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63094.46 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 6309.45 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211752.307740 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56785.02 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 5678.50 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217430.809432 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51106.52 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 5110.65 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222541.460956 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45995.86 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 4599.59 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227141.047327 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41396.28 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 4139.63 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231280.675061 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37256.65 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3725.66 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235006.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33530.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238606.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29930.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242206.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26330.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245806.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22730.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249406.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19130.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253006.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15530.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256606.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11930.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260206.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8330.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263806.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4730.98 sec) 02:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267406.340021 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1130.98 sec) 02:37:00 [GPUGRID] [rr_sim] 2268537.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997654819.72G/439.78G) 02:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1850998.24 02:37:00 [---] pick_jobs_to_run() start 02:37:00 [---] pick_jobs_to_run() end 02:37:00 [---] [rr_sim] end 02:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:37:00 [---] [cpu_sched_debug] preliminary job list: 02:37:00 [---] [cpu_sched_debug] final job list: 02:37:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 02:37:00 [---] [work_fetch] Request work fetch: CPUs idle 02:37:00 [---] [cpu_sched_debug] enforce_run_list: end 02:37:00 [---] choose_project(): 9420.000000 02:37:00 [iThena.Measurements] doing RPC to get work 02:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426179.09 cpu 2169572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:37:00 [---] Got 0 tasks 02:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:38:00 [---] [rr_sim] doing sim: CPU sched 02:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997654819.72G/439.78G) 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268537.32 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 226853.73 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226853.732467 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041683.59 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 204168.36 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431022.091686 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837515.23 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 183751.52 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614773.614984 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653763.71 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 165376.37 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780149.985952 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488387.34 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 148838.73 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928988.719824 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339548.60 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 133954.86 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062943.580308 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205593.74 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 120559.37 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183502.954744 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085034.37 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 108503.44 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1292006.391736 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976530.93 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 97653.09 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389659.485029 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878877.84 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 87887.78 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477547.268992 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790990.06 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 79099.01 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556646.274560 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711891.05 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 71189.11 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627835.379570 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640701.95 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 64070.19 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691905.574080 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576631.75 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 57663.18 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749568.749138 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518968.58 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 51896.86 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801465.606691 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467071.72 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 46707.17 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848172.778488 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420364.55 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 42036.45 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890209.233106 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378328.09 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 37832.81 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928042.042262 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340495.28 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 34049.53 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962091.570502 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306445.75 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 30644.58 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992736.145919 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275801.18 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 27580.12 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020316.263793 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248221.06 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 24822.11 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045138.369880 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223398.95 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 22339.90 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067478.265359 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201059.06 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 20105.91 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087584.171289 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180953.15 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 18095.32 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105679.486627 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162857.84 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 16285.78 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121965.270431 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146572.05 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 14657.21 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136622.475854 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131914.85 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 13191.48 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149813.960735 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118723.36 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 11872.34 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161686.297128 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106851.03 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 10685.10 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172371.399882 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96165.92 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 9616.59 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181987.992360 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86549.33 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 8654.93 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190642.925591 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77894.40 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 7789.44 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198432.365498 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70104.96 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 7010.50 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205442.861415 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63094.46 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 6309.45 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211752.307740 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56785.02 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 5678.50 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217430.809432 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51106.52 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 5110.65 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222541.460956 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45995.86 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 4599.59 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227141.047327 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41396.28 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 4139.63 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231280.675061 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37256.65 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3725.66 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235006.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33530.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238606.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29930.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242206.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26330.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245806.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22730.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249406.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19130.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253006.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15530.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256606.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11930.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260206.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8330.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263806.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4730.98 sec) 02:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267406.340021 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1130.98 sec) 02:38:00 [GPUGRID] [rr_sim] 2268537.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997654819.72G/439.78G) 02:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851058.24 02:38:00 [---] pick_jobs_to_run() start 02:38:00 [---] pick_jobs_to_run() end 02:38:00 [---] [rr_sim] end 02:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:38:00 [---] [cpu_sched_debug] preliminary job list: 02:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:38:00 [---] [cpu_sched_debug] final job list: 02:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 02:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:38:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 997654819.72G 02:38:00 [---] [cpu_sched_debug] enforce_run_list: end 02:38:00 [---] choose_project(): 9480.000000 02:38:00 [iThena.Measurements] doing RPC to get work 02:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426119.09 cpu 2169572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:38:00 [---] Got 0 tasks 02:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:39:00 [---] [rr_sim] doing sim: CPU sched 02:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997627229.37G/439.78G) 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268474.59 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 226847.46 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226847.458778 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041627.13 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 204162.71 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 431010.171679 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837464.42 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 183746.44 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614756.613289 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653717.97 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 165371.80 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780128.410739 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488346.18 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 148834.62 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928963.028443 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339511.56 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 133951.16 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062914.184377 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205560.40 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 120556.04 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183470.224718 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085004.36 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 108500.44 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291970.661024 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976503.93 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 97650.39 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389621.053700 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878853.53 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 87885.35 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477506.407108 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790968.18 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 79096.82 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556603.225176 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711871.36 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 71187.14 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627790.361437 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640684.23 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 64068.42 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691858.784071 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576615.80 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 57661.58 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749520.364442 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518954.22 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 51895.42 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801415.786776 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467058.80 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 46705.88 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848121.666877 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420352.92 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 42035.29 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890156.958968 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378317.63 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 37831.76 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927988.721849 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340485.87 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 34048.59 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962037.308443 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306437.28 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 30643.73 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992681.036377 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275793.55 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 27579.36 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020260.391517 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248214.20 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 24821.42 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045081.811144 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223392.78 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 22339.28 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067421.088808 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201053.50 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 20105.35 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087526.438705 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180948.15 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 18094.81 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105621.253613 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162853.33 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 16285.33 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121906.587030 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146568.00 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 14656.80 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136563.387105 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131911.20 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 13191.12 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149754.507173 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118720.08 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 11872.01 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161626.515234 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106848.07 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 10684.81 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172311.322489 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96163.27 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 9616.33 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181927.649018 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86546.94 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 8654.69 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190582.342895 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77892.24 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 7789.22 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198371.567384 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70103.02 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 7010.30 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205381.869424 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63092.72 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 6309.27 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211691.141260 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56783.45 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 5678.34 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217369.485912 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51105.10 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 5110.51 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222479.996099 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45994.59 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 4599.46 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227079.455267 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41395.13 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 4139.51 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231218.968519 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37255.62 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3725.56 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234944.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33530.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238544.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29930.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242144.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26330.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245744.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22730.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249344.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19130.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252944.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15530.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256544.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11930.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260144.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8330.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263744.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4730.06 sec) 02:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267344.530445 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1130.06 sec) 02:39:00 [GPUGRID] [rr_sim] 2268474.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997627229.37G/439.78G) 02:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851055.50 02:39:00 [---] pick_jobs_to_run() start 02:39:00 [---] pick_jobs_to_run() end 02:39:00 [---] [rr_sim] end 02:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:39:00 [---] [cpu_sched_debug] preliminary job list: 02:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 02:39:00 [---] [cpu_sched_debug] final job list: 02:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:39:00 [---] [cpu_sched_debug] enforce_run_list: end 02:39:00 [---] choose_project(): 9540.000000 02:39:00 [iThena.Measurements] doing RPC to get work 02:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 426059.09 cpu 2169512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:39:00 [---] Got 0 tasks 02:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:40:00 [---] [rr_sim] doing sim: CPU sched 02:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997599639.01G/439.78G) 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268411.85 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 226841.19 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226841.185090 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041570.67 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 204157.07 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430998.251671 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837413.60 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 183741.36 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614739.611594 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653672.24 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 165367.22 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780106.835525 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488305.02 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 148830.50 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928937.337062 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339474.51 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 133947.45 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062884.788446 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205527.06 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 120552.71 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183437.494692 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084974.36 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 108497.44 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291934.930313 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976476.92 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 97647.69 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389582.622371 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878829.23 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 87882.92 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477465.545224 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790946.31 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 79094.63 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556560.175792 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711851.68 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 71185.17 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627745.343303 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640666.51 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 64066.65 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691811.994063 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576599.86 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 57659.99 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749471.979746 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518939.87 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 51893.99 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801365.966862 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467045.88 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 46704.59 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848070.555266 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420341.30 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 42034.13 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890104.684829 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378307.17 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 37830.72 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927935.401436 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340476.45 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 34047.64 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961983.046383 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306428.80 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 30642.88 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992625.926835 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275785.92 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 27578.59 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020204.519241 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248207.33 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 24820.73 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045025.252407 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223386.60 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 22338.66 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067363.912257 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201047.94 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 20104.79 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087468.706121 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180943.14 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 18094.31 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105563.020599 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162848.83 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 16284.88 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121847.903629 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146563.95 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 14656.39 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136504.298356 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131907.55 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 13190.76 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149695.053611 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118716.80 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 11871.68 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161566.733340 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106845.12 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 10684.51 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172251.245096 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96160.61 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 9616.06 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181867.305676 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86544.55 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 8654.45 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190521.760199 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77890.09 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 7789.01 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198310.769269 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70101.08 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 7010.11 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205320.877432 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63090.97 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 6309.10 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211629.974779 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56781.88 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 5678.19 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217308.162391 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51103.69 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 5110.37 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222418.531242 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45993.32 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 4599.33 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227017.863208 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41393.99 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 4139.40 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231157.261977 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37254.59 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3725.46 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234882.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33529.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238482.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29929.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242082.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26329.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245682.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22729.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249282.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19129.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252882.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15529.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256482.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11929.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260082.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8329.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263682.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4729.13 sec) 02:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267282.720870 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1129.13 sec) 02:40:00 [GPUGRID] [rr_sim] 2268411.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997599639.01G/439.78G) 02:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851052.76 02:40:00 [---] pick_jobs_to_run() start 02:40:00 [---] pick_jobs_to_run() end 02:40:00 [---] [rr_sim] end 02:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:40:00 [---] [cpu_sched_debug] preliminary job list: 02:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:40:00 [---] [cpu_sched_debug] final job list: 02:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:40:00 [---] [cpu_sched_debug] enforce_run_list: end 02:40:00 [---] choose_project(): 9600.000000 02:40:00 [iThena.Measurements] doing RPC to get work 02:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425999.09 cpu 2169452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:40:00 [---] Got 0 tasks 02:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:41:00 [---] [rr_sim] doing sim: CPU sched 02:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997572048.65G/439.78G) 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268349.11 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 226834.91 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226834.911402 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041514.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 204151.42 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430986.331664 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837362.78 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 183736.28 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614722.609899 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653626.50 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 165362.65 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780085.260311 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488263.85 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 148826.39 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928911.645682 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339437.47 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 133943.75 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062855.392515 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205493.72 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 120549.37 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183404.764666 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084944.35 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 108494.43 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291899.199601 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976449.91 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 97644.99 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389544.191043 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878804.92 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 87880.49 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477424.683340 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790924.43 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 79092.44 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556517.126408 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711831.99 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 71183.20 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627700.325169 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640648.79 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 64064.88 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691765.204054 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576583.91 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 57658.39 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749423.595051 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518925.52 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 51892.55 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801316.146947 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467032.97 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 46703.30 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848019.443655 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420329.67 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 42032.97 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890052.410691 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378296.70 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 37829.67 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927882.081024 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340467.03 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 34046.70 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961928.784323 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306420.33 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 30642.03 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992570.817293 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275778.30 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 27577.83 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020148.646965 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248200.47 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 24820.05 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044968.693671 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223380.42 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 22338.04 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067306.735705 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201042.38 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 20104.24 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087410.973537 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180938.14 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 18093.81 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105504.787585 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162844.33 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 16284.43 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121789.220228 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146559.89 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 14655.99 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136445.209607 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131903.90 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 13190.39 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149635.600049 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118713.51 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 11871.35 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161506.951446 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106842.16 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 10684.22 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172191.167703 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96157.95 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 9615.79 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181806.962334 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86542.15 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 8654.22 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190461.177503 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77887.94 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 7788.79 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198249.971154 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70099.14 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 7009.91 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205259.885441 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63089.23 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 6308.92 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211568.808299 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56780.31 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 5678.03 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217246.838871 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51102.28 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 5110.23 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222357.066385 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45992.05 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 4599.20 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226956.271149 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41392.84 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 4139.28 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231095.555436 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37253.56 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3725.36 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234820.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33528.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238420.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29928.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242020.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26328.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245620.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22728.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249220.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19128.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252820.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15528.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256420.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11928.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260020.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8328.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263620.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4728.20 sec) 02:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267220.911294 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1128.20 sec) 02:41:00 [GPUGRID] [rr_sim] 2268349.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997572048.65G/439.78G) 02:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851050.03 02:41:00 [---] pick_jobs_to_run() start 02:41:00 [---] pick_jobs_to_run() end 02:41:00 [---] [rr_sim] end 02:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:41:00 [---] [cpu_sched_debug] preliminary job list: 02:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:41:00 [---] [cpu_sched_debug] final job list: 02:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:41:00 [---] [cpu_sched_debug] enforce_run_list: end 02:41:00 [---] choose_project(): 9660.000000 02:41:00 [iThena.Measurements] doing RPC to get work 02:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425939.09 cpu 2169392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:41:00 [---] Got 0 tasks 02:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:42:00 [---] [rr_sim] doing sim: CPU sched 02:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997544458.30G/439.78G) 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268286.38 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 226828.64 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226828.637714 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041457.74 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 204145.77 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430974.411656 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837311.97 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 183731.20 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614705.608204 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653580.77 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 165358.08 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780063.685097 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488222.69 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 148822.27 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928885.954301 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339400.42 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 133940.04 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062825.996585 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205460.38 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 120546.04 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183372.034640 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084914.34 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 108491.43 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291863.468890 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976422.91 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 97642.29 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389505.759714 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878780.62 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 87878.06 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477383.821456 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790902.56 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 79090.26 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556474.077024 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711812.30 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 71181.23 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627655.307036 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640631.07 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 64063.11 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691718.414046 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576567.96 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 57656.80 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749375.210355 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518911.17 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 51891.12 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801266.327033 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467020.05 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 46702.01 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847968.332043 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420318.05 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 42031.80 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890000.136553 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378286.24 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 37828.62 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927828.760611 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340457.62 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 34045.76 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961874.522264 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306411.85 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 30641.19 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992515.707751 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275770.67 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 27577.07 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020092.774689 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248193.60 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 24819.36 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044912.134934 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223374.24 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 22337.42 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067249.559154 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201036.82 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 20103.68 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087353.240953 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180933.14 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 18093.31 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105446.554571 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162839.82 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 16283.98 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121730.536828 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146555.84 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 14655.58 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136386.120858 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131900.26 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 13190.03 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149576.146486 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118710.23 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 11871.02 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161447.169551 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106839.21 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 10683.92 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172131.090310 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96155.29 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 9615.53 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181746.618992 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86539.76 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 8653.98 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190400.594807 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77885.78 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 7788.58 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198189.173040 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70097.20 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 7009.72 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205198.893449 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63087.48 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 6308.75 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211507.641818 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56778.74 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 5677.87 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217185.515350 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51100.86 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 5110.09 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222295.601529 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45990.78 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 4599.08 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226894.679089 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41391.70 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 4139.17 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231033.848894 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37252.53 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3725.25 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234759.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33527.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238359.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29927.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241959.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26327.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245559.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22727.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249159.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19127.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252759.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15527.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256359.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11927.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259959.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8327.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263559.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4727.28 sec) 02:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267159.101718 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1127.28 sec) 02:42:00 [GPUGRID] [rr_sim] 2268286.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997544458.30G/439.78G) 02:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851047.29 02:42:00 [---] pick_jobs_to_run() start 02:42:00 [---] pick_jobs_to_run() end 02:42:00 [---] [rr_sim] end 02:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:42:00 [---] [cpu_sched_debug] preliminary job list: 02:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:42:00 [---] [cpu_sched_debug] final job list: 02:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:42:00 [---] [cpu_sched_debug] enforce_run_list: end 02:42:00 [---] choose_project(): 9720.000000 02:42:00 [iThena.Measurements] doing RPC to get work 02:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425879.09 cpu 2169332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:42:00 [---] Got 0 tasks 02:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:43:00 [---] [rr_sim] doing sim: CPU sched 02:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997516867.94G/439.78G) 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268223.64 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 226822.36 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226822.364025 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041401.28 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 204140.13 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430962.491648 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837261.15 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 183726.11 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614688.606509 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653535.03 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 165353.50 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780042.109883 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488181.53 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 148818.15 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928860.262921 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339363.38 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 133936.34 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062796.600654 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205427.04 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 120542.70 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183339.304614 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084884.34 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 108488.43 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291827.738178 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976395.90 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 97639.59 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389467.328386 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878756.31 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 87875.63 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477342.959573 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790880.68 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 79088.07 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556431.027641 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711792.61 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 71179.26 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627610.288902 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640613.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 64061.34 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691671.624037 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576552.02 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 57655.20 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749326.825659 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518896.81 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 51889.68 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801216.507119 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467007.13 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 46700.71 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847917.220432 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420306.42 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 42030.64 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889947.862414 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378275.78 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 37827.58 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927775.440198 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340448.20 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 34044.82 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961820.260204 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306403.38 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 30640.34 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992460.598209 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275763.04 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 27576.30 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020036.902413 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248186.74 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 24818.67 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044855.576198 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223368.06 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 22336.81 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067192.382603 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201031.26 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 20103.13 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087295.508368 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180928.13 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 18092.81 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105388.321557 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162835.32 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 16283.53 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121671.853427 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146551.79 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 14655.18 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136327.032109 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131896.61 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 13189.66 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149516.692924 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118706.95 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 11870.69 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161387.387657 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106836.25 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 10683.63 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172071.012917 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96152.63 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 9615.26 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181686.275650 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86537.36 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 8653.74 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190340.012111 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77883.63 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 7788.36 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198128.374925 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70095.27 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 7009.53 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205137.901458 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63085.74 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 6308.57 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211446.475338 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56777.16 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 5677.72 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217124.191829 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51099.45 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 5109.94 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222234.136672 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45989.50 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 4598.95 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226833.087030 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41390.55 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 4139.06 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230972.142353 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37251.50 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3725.15 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234697.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33526.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238297.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29926.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241897.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26326.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245497.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22726.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249097.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19126.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252697.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15526.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256297.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11926.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259897.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8326.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263497.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4726.35 sec) 02:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267097.292143 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1126.35 sec) 02:43:00 [GPUGRID] [rr_sim] 2268223.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997516867.94G/439.78G) 02:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851044.55 02:43:00 [---] pick_jobs_to_run() start 02:43:00 [---] pick_jobs_to_run() end 02:43:00 [---] [rr_sim] end 02:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:43:00 [---] [cpu_sched_debug] preliminary job list: 02:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:43:00 [---] [cpu_sched_debug] final job list: 02:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:43:00 [---] [cpu_sched_debug] enforce_run_list: end 02:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:44:00 [---] [rr_sim] doing sim: CPU sched 02:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997489277.58G/439.78G) 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268160.90 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 226816.09 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226816.090337 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041344.81 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 204134.48 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430950.571641 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837210.33 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 183721.03 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614671.604814 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653489.30 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 165348.93 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 780020.534670 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488140.37 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 148814.04 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928834.571540 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339326.33 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 133932.63 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062767.204723 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205393.70 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 120539.37 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183306.574588 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084854.33 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 108485.43 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291792.007466 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976368.90 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 97636.89 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389428.897057 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878732.01 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 87873.20 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477302.097689 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790858.81 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 79085.88 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556387.978257 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711772.93 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 71177.29 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627565.270768 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640595.63 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 64059.56 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691624.834029 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576536.07 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 57653.61 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749278.440963 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518882.46 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 51888.25 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801166.687204 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466994.22 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 46699.42 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847866.108821 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420294.79 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 42029.48 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889895.588276 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378265.32 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 37826.53 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927722.119786 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340438.78 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 34043.88 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961765.998144 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306394.91 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 30639.49 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992405.488667 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275755.41 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 27575.54 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019981.030138 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248179.87 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 24817.99 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044799.017461 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223361.89 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 22336.19 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067135.206052 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201025.70 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 20102.57 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087237.775784 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180923.13 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 18092.31 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105330.088543 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162830.81 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 16283.08 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121613.170026 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146547.73 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 14654.77 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136267.943360 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131892.96 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 13189.30 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149457.239362 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118703.66 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 11870.37 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161327.605763 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106833.30 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 10683.33 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172010.935524 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96149.97 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 9615.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181625.932309 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86534.97 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 8653.50 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190279.429415 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77881.47 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 7788.15 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198067.576811 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70093.33 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 7009.33 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205076.909467 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63083.99 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 6308.40 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211385.308857 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56775.59 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 5677.56 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217062.868309 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51098.04 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 5109.80 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222172.671815 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45988.23 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 4598.82 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226771.494971 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41389.41 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 4138.94 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230910.435811 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37250.47 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3725.05 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234635.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33525.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238235.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29925.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241835.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26325.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245435.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22725.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249035.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19125.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252635.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15525.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256235.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11925.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259835.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8325.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263435.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4725.42 sec) 02:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2267035.482567 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1125.42 sec) 02:44:00 [GPUGRID] [rr_sim] 2268160.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997489277.58G/439.78G) 02:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851041.82 02:44:00 [---] pick_jobs_to_run() start 02:44:00 [---] pick_jobs_to_run() end 02:44:00 [---] [rr_sim] end 02:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:44:00 [---] [cpu_sched_debug] preliminary job list: 02:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:44:00 [---] [cpu_sched_debug] final job list: 02:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:44:00 [---] [cpu_sched_debug] enforce_run_list: end 02:44:00 [---] choose_project(): 9840.000000 02:44:00 [iThena.Measurements] doing RPC to get work 02:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425759.09 cpu 2169212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:44:00 [---] Got 0 tasks 02:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:45:00 [---] [rr_sim] doing sim: CPU sched 02:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997461687.23G/439.78G) 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268098.17 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 226809.82 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226809.816649 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041288.35 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 204128.83 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430938.651633 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837159.51 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 183715.95 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614654.603119 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653443.56 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 165344.36 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779998.959456 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488099.21 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 148809.92 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928808.880159 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339289.29 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 133928.93 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062737.808792 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205360.36 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 120536.04 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183273.844562 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084824.32 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 108482.43 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291756.276755 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976341.89 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 97634.19 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389390.465728 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878707.70 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 87870.77 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477261.235805 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790836.93 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 79083.69 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556344.928873 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711753.24 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 71175.32 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627520.252635 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640577.91 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 64057.79 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691578.044020 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576520.12 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 57652.01 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749230.056267 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518868.11 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 51886.81 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801116.867290 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466981.30 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 46698.13 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847814.997210 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420283.17 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 42028.32 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889843.314138 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378254.85 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 37825.49 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927668.799373 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340429.37 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 34042.94 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961711.736085 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306386.43 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 30638.64 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992350.379125 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275747.79 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 27574.78 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019925.157862 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248173.01 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 24817.30 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044742.458724 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223355.71 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 22335.57 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067078.029501 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201020.14 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 20102.01 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087180.043200 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180918.12 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 18091.81 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105271.855529 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162826.31 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 16282.63 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121554.486625 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146543.68 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 14654.37 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136208.854612 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131889.31 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 13188.93 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149397.785799 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118700.38 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 11870.04 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161267.823868 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106830.34 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 10683.03 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171950.858131 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96147.31 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 9614.73 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181565.588967 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86532.58 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 8653.26 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190218.846719 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77879.32 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 7787.93 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198006.778696 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70091.39 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 7009.14 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205015.917475 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63082.25 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 6308.22 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211324.142377 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56774.02 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 5677.40 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217001.544788 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51096.62 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 5109.66 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222111.206958 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45986.96 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 4598.70 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226709.902912 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41388.26 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 4138.83 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230848.729269 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37249.44 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3724.94 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234573.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33524.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238173.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29924.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241773.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26324.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245373.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22724.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248973.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19124.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252573.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15524.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256173.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11924.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259773.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8324.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263373.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4724.49 sec) 02:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266973.672991 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1124.49 sec) 02:45:00 [GPUGRID] [rr_sim] 2268098.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997461687.23G/439.78G) 02:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851039.08 02:45:00 [---] pick_jobs_to_run() start 02:45:00 [---] pick_jobs_to_run() end 02:45:00 [---] [rr_sim] end 02:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:45:00 [---] [cpu_sched_debug] preliminary job list: 02:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:45:00 [---] [cpu_sched_debug] final job list: 02:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:45:00 [---] [cpu_sched_debug] enforce_run_list: end 02:45:00 [---] choose_project(): 9900.000000 02:45:00 [iThena.Measurements] doing RPC to get work 02:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425699.09 cpu 2169152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:45:00 [---] Got 0 tasks 02:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:46:00 [---] [rr_sim] doing sim: CPU sched 02:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997434096.87G/439.78G) 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2268035.43 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 226803.54 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226803.542961 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041231.89 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 204123.19 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430926.731625 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837108.70 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 183710.87 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614637.601424 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653397.83 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 165339.78 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779977.384242 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488058.05 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 148805.80 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928783.188779 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339252.24 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 133925.22 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062708.412862 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205327.02 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 120532.70 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183241.114536 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084794.32 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 108479.43 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291720.546043 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976314.88 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 97631.49 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389352.034400 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878683.40 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 87868.34 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477220.373921 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790815.06 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 79081.51 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556301.879489 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711733.55 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 71173.36 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627475.234501 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640560.20 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 64056.02 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691531.254012 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576504.18 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 57650.42 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749181.671571 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518853.76 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 51885.38 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801067.047375 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466968.38 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 46696.84 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847763.885598 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420271.54 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 42027.15 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889791.039999 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378244.39 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 37824.44 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927615.478960 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340419.95 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 34042.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961657.474025 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306377.96 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 30637.80 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992295.269583 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275740.16 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 27574.02 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019869.285586 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248166.14 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 24816.61 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044685.899988 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223349.53 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 22334.95 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067020.852950 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201014.58 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 20101.46 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087122.310616 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180913.12 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 18091.31 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105213.622515 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162821.81 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 16282.18 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121495.803224 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146539.63 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 14653.96 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136149.765863 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131885.66 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 13188.57 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149338.332237 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118697.10 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 11869.71 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161208.041974 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106827.39 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 10682.74 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171890.780738 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96144.65 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 9614.46 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181505.245625 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86530.18 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 8653.02 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190158.264023 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77877.17 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 7787.72 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197945.980581 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70089.45 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 7008.94 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204954.925484 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63080.50 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 6308.05 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211262.975896 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56772.45 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 5677.25 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216940.221268 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51095.21 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 5109.52 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222049.742102 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45985.69 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 4598.57 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226648.310852 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41387.12 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 4138.71 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230787.022728 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37248.41 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3724.84 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234511.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33523.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238111.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29923.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241711.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26323.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245311.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22723.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248911.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19123.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252511.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15523.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256111.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11923.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259711.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8323.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263311.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4723.57 sec) 02:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266911.863416 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1123.57 sec) 02:46:00 [GPUGRID] [rr_sim] 2268035.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997434096.87G/439.78G) 02:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851036.34 02:46:00 [---] pick_jobs_to_run() start 02:46:00 [---] pick_jobs_to_run() end 02:46:00 [---] [rr_sim] end 02:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:46:00 [---] [cpu_sched_debug] preliminary job list: 02:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:46:00 [---] [cpu_sched_debug] final job list: 02:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:46:00 [---] [cpu_sched_debug] enforce_run_list: end 02:46:00 [---] choose_project(): 9960.000000 02:46:00 [iThena.Measurements] doing RPC to get work 02:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425639.09 cpu 2169092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:46:00 [---] Got 0 tasks 02:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:47:00 [---] [rr_sim] doing sim: CPU sched 02:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997406506.52G/439.78G) 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267972.69 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 226797.27 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226797.269273 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041175.42 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 204117.54 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430914.811618 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837057.88 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 183705.79 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614620.599729 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653352.09 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 165335.21 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779955.809028 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1488016.88 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 148801.69 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928757.497398 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339215.20 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 133921.52 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062679.016931 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205293.68 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 120529.37 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183208.384510 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084764.31 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 108476.43 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291684.815332 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976287.88 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 97628.79 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389313.603071 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878659.09 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 87865.91 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477179.512037 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790793.18 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 79079.32 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556258.830106 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711713.86 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 71171.39 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627430.216368 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640542.48 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 64054.25 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691484.464003 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576488.23 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 57648.82 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749133.286876 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518839.41 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 51883.94 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801017.227461 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466955.47 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 46695.55 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847712.773987 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420259.92 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 42025.99 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889738.765861 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378233.93 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 37823.39 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927562.158547 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340410.53 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 34041.05 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961603.211965 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306369.48 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 30636.95 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992240.160041 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275732.53 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 27573.25 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019813.413310 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248159.28 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 24815.93 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044629.341251 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223343.35 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 22334.34 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066963.676399 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201009.02 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 20100.90 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087064.578031 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180908.11 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 18090.81 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105155.389501 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162817.30 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 16281.73 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121437.119823 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146535.57 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 14653.56 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136090.677114 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131882.02 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 13188.20 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149278.878675 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118693.81 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 11869.38 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161148.260080 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106824.43 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 10682.44 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171830.703344 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96141.99 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 9614.20 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181444.902283 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86527.79 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 8652.78 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190097.681327 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77875.01 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 7787.50 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197885.182467 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70087.51 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 7008.75 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204893.933493 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63078.76 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 6307.88 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211201.809416 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56770.88 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 5677.09 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216878.897747 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51093.79 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 5109.38 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221988.277245 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45984.42 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 4598.44 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226586.718793 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41385.97 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 4138.60 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230725.316186 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37247.38 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3724.74 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234450.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33522.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238050.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29922.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241650.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26322.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245250.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22722.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248850.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19122.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252450.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15522.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256050.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11922.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259650.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8322.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263250.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4722.64 sec) 02:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266850.053840 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1122.64 sec) 02:47:00 [GPUGRID] [rr_sim] 2267972.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997406506.52G/439.78G) 02:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851033.61 02:47:00 [---] pick_jobs_to_run() start 02:47:00 [---] pick_jobs_to_run() end 02:47:00 [---] [rr_sim] end 02:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:47:00 [---] [cpu_sched_debug] preliminary job list: 02:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:47:00 [---] [cpu_sched_debug] final job list: 02:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:47:00 [---] [cpu_sched_debug] enforce_run_list: end 02:47:00 [---] choose_project(): 10020.000000 02:47:00 [iThena.Measurements] doing RPC to get work 02:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425579.09 cpu 2169032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2169032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2169032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:47:00 [---] Got 0 tasks 02:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:48:00 [---] [rr_sim] doing sim: CPU sched 02:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997378916.16G/439.78G) 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267909.96 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 226791.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226790.995584 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041118.96 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 204111.90 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430902.891610 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1837007.06 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 183700.71 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614603.598034 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653306.36 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 165330.64 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779934.233815 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487975.72 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 148797.57 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928731.806017 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339178.15 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 133917.81 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062649.621000 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205260.33 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 120526.03 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183175.654484 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084734.30 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 108473.43 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291649.084620 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976260.87 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 97626.09 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389275.171743 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878634.78 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 87863.48 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477138.650153 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790771.31 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 79077.13 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556215.780722 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711694.18 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 71169.42 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627385.198234 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640524.76 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 64052.48 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691437.673995 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576472.28 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 57647.23 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749084.902180 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518825.05 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 51882.51 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800967.407546 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466942.55 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 46694.25 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847661.662376 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420248.29 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 42024.83 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889686.491723 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378223.46 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 37822.35 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927508.838135 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340401.12 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 34040.11 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961548.949906 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306361.01 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 30636.10 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992185.050499 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275724.91 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 27572.49 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019757.541034 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248152.41 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 24815.24 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044572.782515 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223337.17 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 22333.72 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066906.499848 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201003.46 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 20100.35 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087006.845447 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180903.11 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 18090.31 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105097.156487 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162812.80 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 16281.28 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121378.436423 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146531.52 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 14653.15 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136031.588365 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131878.37 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 13187.84 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149219.425113 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118690.53 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 11869.05 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161088.478186 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106821.48 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 10682.15 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171770.625951 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96139.33 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 9613.93 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181384.558941 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86525.40 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 8652.54 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190037.098631 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77872.86 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 7787.29 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197824.384352 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70085.57 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 7008.56 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204832.941501 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63077.01 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 6307.70 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211140.642936 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56769.31 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 5676.93 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216817.574226 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51092.38 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 5109.24 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221926.812388 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45983.14 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 4598.31 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226525.126734 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41384.83 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 4138.48 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230663.609645 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37246.35 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3724.63 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234388.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33521.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237988.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29921.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241588.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26321.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245188.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22721.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248788.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19121.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252388.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15521.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255988.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11921.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259588.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8321.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263188.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4721.71 sec) 02:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266788.244264 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1121.71 sec) 02:48:00 [GPUGRID] [rr_sim] 2267909.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997378916.16G/439.78G) 02:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851030.87 02:48:00 [---] pick_jobs_to_run() start 02:48:00 [---] pick_jobs_to_run() end 02:48:00 [---] [rr_sim] end 02:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:48:00 [---] [cpu_sched_debug] preliminary job list: 02:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:48:00 [---] [cpu_sched_debug] final job list: 02:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:48:00 [---] [cpu_sched_debug] enforce_run_list: end 02:48:00 [---] choose_project(): 10080.000000 02:48:00 [iThena.Measurements] doing RPC to get work 02:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425519.09 cpu 2168972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:48:00 [---] Got 0 tasks 02:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:49:00 [---] [rr_sim] doing sim: CPU sched 02:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997351325.80G/439.78G) 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267847.22 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 226784.72 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226784.721896 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041062.50 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 204106.25 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430890.971603 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836956.25 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 183695.62 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614586.596339 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653260.62 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 165326.06 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779912.658601 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487934.56 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 148793.46 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928706.114637 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339141.10 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 133914.11 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062620.225069 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205226.99 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 120522.70 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183142.924459 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084704.29 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 108470.43 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291613.353909 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976233.87 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 97623.39 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389236.740414 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878610.48 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 87861.05 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477097.788269 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790749.43 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 79074.94 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556172.731338 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711674.49 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 71167.45 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627340.180100 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640507.04 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 64050.70 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691390.883986 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576456.33 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 57645.63 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749036.517484 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518810.70 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 51881.07 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800917.587632 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466929.63 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 46692.96 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847610.550765 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420236.67 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 42023.67 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889634.217584 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378213.00 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 37821.30 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927455.517722 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340391.70 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 34039.17 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961494.687846 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306352.53 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 30635.25 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992129.940957 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275717.28 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 27571.73 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019701.668758 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248145.55 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 24814.56 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044516.223778 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223331.00 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 22333.10 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066849.323297 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200997.90 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 20099.79 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086949.112863 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180898.11 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 18089.81 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105038.923473 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162808.30 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 16280.83 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121319.753022 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146527.47 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 14652.75 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135972.499616 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131874.72 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 13187.47 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149159.971550 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118687.25 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 11868.72 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161028.696291 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106818.52 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 10681.85 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171710.548558 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96136.67 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 9613.67 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181324.215599 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86523.00 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 8652.30 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189976.515935 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77870.70 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 7787.07 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197763.586238 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70083.63 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 7008.36 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204771.949510 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63075.27 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 6307.53 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211079.476455 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56767.74 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 5676.77 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216756.250706 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51090.97 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 5109.10 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221865.347531 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45981.87 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 4598.19 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226463.534674 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41383.68 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 4138.37 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230601.903103 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37245.32 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3724.53 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234326.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33520.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237926.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29920.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241526.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26320.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245126.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22720.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248726.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19120.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252326.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15520.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255926.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11920.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259526.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8320.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263126.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4720.78 sec) 02:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266726.434689 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1120.78 sec) 02:49:00 [GPUGRID] [rr_sim] 2267847.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997351325.80G/439.78G) 02:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851028.13 02:49:00 [---] pick_jobs_to_run() start 02:49:00 [---] pick_jobs_to_run() end 02:49:00 [---] [rr_sim] end 02:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:49:00 [---] [cpu_sched_debug] preliminary job list: 02:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:49:00 [---] [cpu_sched_debug] final job list: 02:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:49:00 [---] [cpu_sched_debug] enforce_run_list: end 02:49:00 [---] choose_project(): 10140.000000 02:49:00 [iThena.Measurements] doing RPC to get work 02:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425459.09 cpu 2168912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:49:00 [---] Got 0 tasks 02:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:50:00 [---] [rr_sim] doing sim: CPU sched 02:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997323735.45G/439.78G) 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267784.48 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 226778.45 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226778.448208 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2041006.03 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 204100.60 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430879.051595 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836905.43 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 183690.54 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614569.594643 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653214.89 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 165321.49 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779891.083387 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487893.40 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 148789.34 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928680.423256 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339104.06 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 133910.41 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062590.829139 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205193.65 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 120519.37 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183110.194433 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084674.29 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 108467.43 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291577.623197 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976206.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 97620.69 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389198.309085 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878586.17 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 87858.62 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477056.926385 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790727.56 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 79072.76 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556129.681954 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711654.80 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 71165.48 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627295.161967 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640489.32 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 64048.93 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691344.093978 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576440.39 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 57644.04 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748988.132788 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518796.35 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 51879.63 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800867.767717 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466916.71 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 46691.67 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847559.439153 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420225.04 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 42022.50 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889581.943446 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378202.54 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 37820.25 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927402.197309 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340382.28 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 34038.23 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961440.425786 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306344.06 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 30634.41 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992074.831416 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275709.65 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 27570.97 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019645.796482 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248138.69 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 24813.87 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044459.665042 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223324.82 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 22332.48 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066792.146745 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200992.34 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 20099.23 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086891.380279 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180893.10 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 18089.31 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104980.690459 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162803.79 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 16280.38 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121261.069621 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146523.41 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 14652.34 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135913.410867 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131871.07 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 13187.11 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149100.517988 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118683.96 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 11868.40 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160968.914397 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106815.57 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 10681.56 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171650.471165 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96134.01 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 9613.40 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181263.872257 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86520.61 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 8652.06 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189915.933239 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77868.55 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 7786.85 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197702.788123 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70081.69 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 7008.17 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204710.957519 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63073.52 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 6307.35 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211018.309975 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56766.17 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 5676.62 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216694.927185 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51089.55 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 5108.96 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221803.882674 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45980.60 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 4598.06 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226401.942615 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41382.54 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 4138.25 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230540.196561 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37244.29 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3724.43 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234264.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33519.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237864.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29919.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241464.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26319.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245064.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22719.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248664.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19119.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252264.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15519.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255864.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11919.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259464.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8319.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263064.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4719.86 sec) 02:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266664.625113 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1119.86 sec) 02:50:00 [GPUGRID] [rr_sim] 2267784.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997323735.45G/439.78G) 02:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851025.39 02:50:00 [---] pick_jobs_to_run() start 02:50:00 [---] pick_jobs_to_run() end 02:50:00 [---] [rr_sim] end 02:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:50:00 [---] [cpu_sched_debug] preliminary job list: 02:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:50:00 [---] [cpu_sched_debug] final job list: 02:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:50:00 [---] [cpu_sched_debug] enforce_run_list: end 02:50:00 [---] choose_project(): 10200.000000 02:50:00 [iThena.Measurements] doing RPC to get work 02:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425399.09 cpu 2168852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:50:00 [---] Got 0 tasks 02:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:51:00 [---] [rr_sim] doing sim: CPU sched 02:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997296145.09G/439.78G) 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267721.75 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 226772.17 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226772.174520 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040949.57 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 204094.96 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430867.131587 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836854.61 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 183685.46 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614552.592948 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653169.15 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 165316.92 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779869.508173 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487852.24 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 148785.22 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928654.731876 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339067.01 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 133906.70 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062561.433208 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205160.31 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 120516.03 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183077.464407 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084644.28 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 108464.43 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291541.892486 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976179.85 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 97617.99 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389159.877757 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878561.87 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 87856.19 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477016.064501 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790705.68 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 79070.57 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556086.632570 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711635.11 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 71163.51 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627250.143833 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640471.60 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 64047.16 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691297.303970 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576424.44 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 57642.44 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748939.748092 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518782.00 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 51878.20 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800817.947803 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466903.80 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 46690.38 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847508.327542 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420213.42 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 42021.34 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889529.669308 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378192.08 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 37819.21 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927348.876897 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340372.87 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 34037.29 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961386.163727 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306335.58 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 30633.56 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992019.721874 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275702.02 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 27570.20 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019589.924206 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248131.82 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 24813.18 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044403.106305 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223318.64 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 22331.86 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066734.970194 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200986.78 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 20098.68 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086833.647695 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180888.10 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 18088.81 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104922.457445 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162799.29 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 16279.93 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121202.386220 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146519.36 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 14651.94 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135854.322118 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131867.42 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 13186.74 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149041.064426 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118680.68 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 11868.07 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160909.132503 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106812.61 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 10681.26 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171590.393772 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96131.35 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 9613.14 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181203.528915 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86518.22 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 8651.82 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189855.350543 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77866.39 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 7786.64 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197641.990008 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70079.76 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 7007.98 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204649.965527 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63071.78 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 6307.18 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210957.143494 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56764.60 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 5676.46 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216633.603664 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51088.14 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 5108.81 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221742.417818 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45979.33 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 4597.93 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226340.350556 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41381.39 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 4138.14 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230478.490020 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37243.26 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3724.33 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234202.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33518.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237802.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29918.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241402.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26318.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245002.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22718.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248602.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19118.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252202.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15518.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255802.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11918.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259402.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8318.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263002.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4718.93 sec) 02:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266602.815538 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1118.93 sec) 02:51:00 [GPUGRID] [rr_sim] 2267721.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997296145.09G/439.78G) 02:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851022.66 02:51:00 [---] pick_jobs_to_run() start 02:51:00 [---] pick_jobs_to_run() end 02:51:00 [---] [rr_sim] end 02:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:51:00 [---] [cpu_sched_debug] preliminary job list: 02:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:51:00 [---] [cpu_sched_debug] final job list: 02:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:51:00 [---] [cpu_sched_debug] enforce_run_list: end 02:51:00 [---] choose_project(): 10260.000000 02:51:00 [iThena.Measurements] doing RPC to get work 02:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425339.09 cpu 2168792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:51:00 [---] Got 0 tasks 02:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:52:00 [---] [rr_sim] doing sim: CPU sched 02:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997296145.09G/439.78G) 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267721.75 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 226772.17 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226772.174520 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040949.57 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 204094.96 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430867.131587 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836854.61 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 183685.46 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614552.592948 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653169.15 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 165316.92 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779869.508173 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487852.24 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 148785.22 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928654.731876 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339067.01 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 133906.70 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062561.433208 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205160.31 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 120516.03 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183077.464407 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084644.28 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 108464.43 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291541.892486 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976179.85 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 97617.99 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389159.877757 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878561.87 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 87856.19 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1477016.064501 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790705.68 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 79070.57 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556086.632570 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711635.11 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 71163.51 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627250.143833 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640471.60 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 64047.16 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691297.303970 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576424.44 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 57642.44 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748939.748092 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518782.00 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 51878.20 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800817.947803 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466903.80 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 46690.38 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847508.327542 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420213.42 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 42021.34 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889529.669308 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378192.08 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 37819.21 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927348.876897 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340372.87 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 34037.29 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961386.163727 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306335.58 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 30633.56 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992019.721874 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275702.02 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 27570.20 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019589.924206 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248131.82 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 24813.18 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044403.106305 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223318.64 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 22331.86 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066734.970194 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200986.78 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 20098.68 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086833.647695 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180888.10 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 18088.81 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104922.457445 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162799.29 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 16279.93 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121202.386220 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146519.36 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 14651.94 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135854.322118 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131867.42 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 13186.74 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149041.064426 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118680.68 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 11868.07 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160909.132503 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106812.61 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 10681.26 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171590.393772 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96131.35 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 9613.14 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181203.528915 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86518.22 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 8651.82 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189855.350543 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77866.39 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 7786.64 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197641.990008 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70079.76 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 7007.98 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204649.965527 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63071.78 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 6307.18 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210957.143494 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56764.60 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 5676.46 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216633.603664 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51088.14 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 5108.81 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221742.417818 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45979.33 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 4597.93 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226340.350556 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41381.39 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 4138.14 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230478.490020 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37243.26 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3724.33 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234202.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33518.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237802.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29918.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241402.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26318.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245002.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22718.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248602.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19118.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252202.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15518.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255802.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11918.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259402.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8318.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263002.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4718.93 sec) 02:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266602.815538 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1118.93 sec) 02:52:00 [GPUGRID] [rr_sim] 2267721.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997296145.09G/439.78G) 02:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851082.66 02:52:00 [---] pick_jobs_to_run() start 02:52:00 [---] pick_jobs_to_run() end 02:52:00 [---] [rr_sim] end 02:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:52:00 [---] [cpu_sched_debug] preliminary job list: 02:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:52:00 [---] [cpu_sched_debug] final job list: 02:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:52:00 [---] [cpu_sched_debug] enforce_run_list: end 02:52:00 [---] choose_project(): 10320.000000 02:52:00 [iThena.Measurements] doing RPC to get work 02:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425279.09 cpu 2168792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:52:00 [---] Got 0 tasks 02:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:53:00 [---] [rr_sim] doing sim: CPU sched 02:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997268554.74G/439.78G) 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267659.01 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 226765.90 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226765.900831 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040893.11 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 204089.31 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430855.211580 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836803.80 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 183680.38 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614535.591253 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653123.42 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 165312.34 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779847.932959 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487811.08 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 148781.11 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928629.040495 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339029.97 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 133903.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062532.037277 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205126.97 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 120512.70 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183044.734381 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084614.27 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 108461.43 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291506.161774 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976152.85 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 97615.28 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389121.446428 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878537.56 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 87853.76 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476975.202617 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790683.81 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 79068.38 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556043.583187 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711615.43 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 71161.54 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627205.125700 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640453.88 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 64045.39 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691250.513961 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576408.49 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 57640.85 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748891.363396 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518767.64 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 51876.76 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800768.127888 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466890.88 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 46689.09 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847457.215931 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420201.79 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 42020.18 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889477.395169 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378181.61 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 37818.16 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927295.556484 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340363.45 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 34036.35 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961331.901667 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306327.11 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 30632.71 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991964.612332 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275694.40 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 27569.44 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019534.051930 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248124.96 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 24812.50 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044346.547569 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223312.46 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 22331.25 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066677.793643 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200981.21 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 20098.12 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086775.915110 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180883.09 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 18088.31 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104864.224431 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162794.78 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 16279.48 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121143.702819 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146515.31 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 14651.53 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135795.233369 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131863.77 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 13186.38 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148981.610863 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118677.40 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 11867.74 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160849.350609 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106809.66 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 10680.97 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171530.316379 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96128.69 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 9612.87 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181143.185573 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86515.82 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 8651.58 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189794.767847 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77864.24 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 7786.42 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197581.191894 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70077.82 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 7007.78 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204588.973536 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63070.03 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 6307.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210895.977014 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56763.03 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 5676.30 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216572.280144 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51086.73 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 5108.67 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221680.952961 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45978.06 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 4597.81 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226278.758496 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41380.25 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 4138.02 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230416.783478 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37242.22 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3724.22 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234141.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33518.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237741.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29918.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241341.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26318.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244941.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22718.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248541.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19118.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252141.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15518.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255741.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11918.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259341.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8318.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262941.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4718.00 sec) 02:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266541.005962 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1118.00 sec) 02:53:00 [GPUGRID] [rr_sim] 2267659.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997268554.74G/439.78G) 02:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851079.92 02:53:00 [---] pick_jobs_to_run() start 02:53:00 [---] pick_jobs_to_run() end 02:53:00 [---] [rr_sim] end 02:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:53:00 [---] [cpu_sched_debug] preliminary job list: 02:53:00 [---] [cpu_sched_debug] final job list: 02:53:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 02:53:00 [---] [work_fetch] Request work fetch: CPUs idle 02:53:00 [---] [cpu_sched_debug] enforce_run_list: end 02:53:00 [---] choose_project(): 10380.000000 02:53:00 [iThena.Measurements] doing RPC to get work 02:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 598019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425219.09 cpu 2168732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:53:00 [---] Got 0 tasks 02:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:54:00 [---] [rr_sim] doing sim: CPU sched 02:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997268554.74G/439.78G) 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267659.01 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 226765.90 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226765.900831 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040893.11 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 204089.31 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430855.211580 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836803.80 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 183680.38 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614535.591253 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653123.42 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 165312.34 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779847.932959 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487811.08 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 148781.11 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928629.040495 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1339029.97 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 133903.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062532.037277 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205126.97 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 120512.70 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183044.734381 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084614.27 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 108461.43 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291506.161774 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976152.85 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 97615.28 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389121.446428 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878537.56 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 87853.76 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476975.202617 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790683.81 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 79068.38 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556043.583187 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711615.43 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 71161.54 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627205.125700 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640453.88 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 64045.39 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691250.513961 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576408.49 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 57640.85 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748891.363396 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518767.64 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 51876.76 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800768.127888 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466890.88 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 46689.09 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847457.215931 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420201.79 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 42020.18 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889477.395169 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378181.61 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 37818.16 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927295.556484 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340363.45 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 34036.35 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961331.901667 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306327.11 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 30632.71 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991964.612332 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275694.40 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 27569.44 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019534.051930 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248124.96 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 24812.50 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044346.547569 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223312.46 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 22331.25 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066677.793643 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200981.21 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 20098.12 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086775.915110 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180883.09 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 18088.31 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104864.224431 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162794.78 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 16279.48 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121143.702819 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146515.31 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 14651.53 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135795.233369 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131863.77 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 13186.38 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148981.610863 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118677.40 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 11867.74 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160849.350609 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106809.66 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 10680.97 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171530.316379 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96128.69 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 9612.87 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181143.185573 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86515.82 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 8651.58 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189794.767847 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77864.24 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 7786.42 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197581.191894 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70077.82 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 7007.78 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204588.973536 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63070.03 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 6307.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210895.977014 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56763.03 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 5676.30 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216572.280144 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51086.73 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 5108.67 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221680.952961 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45978.06 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 4597.81 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226278.758496 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41380.25 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 4138.02 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230416.783478 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37242.22 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3724.22 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234141.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33518.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237741.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29918.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241341.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26318.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244941.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22718.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248541.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19118.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252141.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15518.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255741.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11918.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259341.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8318.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262941.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4718.00 sec) 02:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266541.005962 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1118.00 sec) 02:54:00 [GPUGRID] [rr_sim] 2267659.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997268554.74G/439.78G) 02:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851139.92 02:54:00 [---] pick_jobs_to_run() start 02:54:00 [---] pick_jobs_to_run() end 02:54:00 [---] [rr_sim] end 02:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:54:00 [---] [cpu_sched_debug] preliminary job list: 02:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:54:00 [---] [cpu_sched_debug] final job list: 02:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 02:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:54:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 997268554.74G 02:54:00 [---] [cpu_sched_debug] enforce_run_list: end 02:54:00 [---] choose_project(): 10440.000000 02:54:00 [iThena.Measurements] doing RPC to get work 02:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425159.09 cpu 2168732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:54:00 [---] Got 0 tasks 02:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:55:00 [---] [rr_sim] doing sim: CPU sched 02:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997240964.38G/439.78G) 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267596.27 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 226759.63 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226759.627143 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040836.64 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 204083.66 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430843.291572 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836752.98 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 183675.30 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614518.589558 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653077.68 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 165307.77 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779826.357746 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487769.91 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 148776.99 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928603.349114 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338992.92 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 133899.29 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062502.641346 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205093.63 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 120509.36 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1183012.004355 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084584.27 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 108458.43 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291470.431063 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976125.84 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 97612.58 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389083.015100 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878513.26 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 87851.33 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476934.340733 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790661.93 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 79066.19 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1556000.533803 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711595.74 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 71159.57 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627160.107566 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640436.16 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 64043.62 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691203.723953 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576392.55 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 57639.25 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748842.978701 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518753.29 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 51875.33 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800718.307974 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466877.96 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 46687.80 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847406.104320 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420190.17 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 42019.02 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889425.121031 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378171.15 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 37817.12 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927242.236071 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340354.04 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 34035.40 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961277.639607 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306318.63 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 30631.86 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991909.502790 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275686.77 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 27568.68 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019478.179654 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248118.09 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 24811.81 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044289.988832 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223306.28 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 22330.63 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066620.617092 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200975.65 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 20097.57 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086718.182526 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180878.09 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 18087.81 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104805.991417 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162790.28 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 16279.03 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121085.019418 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146511.25 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 14651.13 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135736.144620 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131860.13 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 13186.01 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148922.157301 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118674.11 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 11867.41 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160789.568714 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106806.70 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 10680.67 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171470.238986 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96126.03 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 9612.60 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181082.842231 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86513.43 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 8651.34 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189734.185151 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77862.09 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 7786.21 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197520.393779 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70075.88 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 7007.59 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204527.981544 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63068.29 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 6306.83 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210834.810533 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56761.46 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 5676.15 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216510.956623 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51085.31 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 5108.53 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221619.488104 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45976.78 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 4597.68 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226217.166437 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41379.10 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 4137.91 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230355.076937 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37241.19 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3724.12 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234079.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33517.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237679.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29917.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241279.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26317.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244879.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22717.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248479.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19117.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252079.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15517.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255679.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11917.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259279.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8317.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262879.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4717.08 sec) 02:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266479.196386 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1117.08 sec) 02:55:00 [GPUGRID] [rr_sim] 2267596.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997240964.38G/439.78G) 02:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851137.18 02:55:00 [---] pick_jobs_to_run() start 02:55:00 [---] pick_jobs_to_run() end 02:55:00 [---] [rr_sim] end 02:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:55:00 [---] [cpu_sched_debug] preliminary job list: 02:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 02:55:00 [---] [cpu_sched_debug] final job list: 02:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:55:00 [---] [cpu_sched_debug] enforce_run_list: end 02:55:00 [---] choose_project(): 10500.000000 02:55:00 [iThena.Measurements] doing RPC to get work 02:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425099.09 cpu 2168672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:55:00 [---] Got 0 tasks 02:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:56:00 [---] [rr_sim] doing sim: CPU sched 02:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997213374.02G/439.78G) 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267533.53 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 226753.35 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226753.353455 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040780.18 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 204078.02 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430831.371565 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836702.16 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 183670.22 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614501.587863 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1653031.95 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 165303.19 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779804.782532 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487728.75 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 148772.88 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928577.657734 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338955.88 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 133895.59 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062473.245415 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205060.29 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 120506.03 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182979.274329 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084554.26 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 108455.43 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291434.700351 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976098.83 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 97609.88 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389044.583771 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878488.95 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 87848.90 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476893.478849 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790640.06 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 79064.01 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555957.484419 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711576.05 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 71157.61 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627115.089432 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640418.45 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 64041.84 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691156.933944 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576376.60 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 57637.66 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748794.594005 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518738.94 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 51873.89 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800668.488059 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466865.05 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 46686.50 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847354.992708 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420178.54 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 42017.85 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889372.846893 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378160.69 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 37816.07 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927188.915658 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340344.62 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 34034.46 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961223.377548 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306310.16 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 30631.02 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991854.393248 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275679.14 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 27567.91 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019422.307378 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248111.23 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 24811.12 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044233.430095 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223300.10 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 22330.01 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066563.440541 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200970.09 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 20097.01 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086660.449942 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180873.08 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 18087.31 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104747.758403 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162785.78 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 16278.58 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121026.336018 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146507.20 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 14650.72 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135677.055871 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131856.48 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 13185.65 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148862.703739 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118670.83 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 11867.08 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160729.786820 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106803.75 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 10680.37 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171410.161593 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96123.37 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 9612.34 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181022.498889 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86511.04 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 8651.10 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189673.602455 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77859.93 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 7785.99 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197459.595664 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70073.94 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 7007.39 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204466.989553 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63066.54 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 6306.65 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210773.644053 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56759.89 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 5675.99 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216449.633103 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51083.90 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 5108.39 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221558.023247 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45975.51 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 4597.55 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226155.574378 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41377.96 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 4137.80 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230293.370395 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37240.16 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3724.02 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234017.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33516.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237617.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29916.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241217.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26316.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244817.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22716.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248417.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19116.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252017.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15516.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255617.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11916.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259217.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8316.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262817.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4716.15 sec) 02:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266417.386811 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1116.15 sec) 02:56:00 [GPUGRID] [rr_sim] 2267533.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997213374.02G/439.78G) 02:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851134.45 02:56:00 [---] pick_jobs_to_run() start 02:56:00 [---] pick_jobs_to_run() end 02:56:00 [---] [rr_sim] end 02:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:56:00 [---] [cpu_sched_debug] preliminary job list: 02:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:56:00 [---] [cpu_sched_debug] final job list: 02:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:56:00 [---] [cpu_sched_debug] enforce_run_list: end 02:56:00 [---] choose_project(): 10560.000000 02:56:00 [iThena.Measurements] doing RPC to get work 02:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 425039.09 cpu 2168612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:56:00 [---] Got 0 tasks 02:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:57:00 [---] [rr_sim] doing sim: CPU sched 02:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997185783.67G/439.78G) 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267470.80 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 226747.08 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226747.079767 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040723.72 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 204072.37 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430819.451557 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836651.35 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 183665.13 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614484.586168 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652986.21 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 165298.62 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779783.207318 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487687.59 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 148768.76 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928551.966353 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338918.83 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 133891.88 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062443.849485 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1205026.95 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 120502.69 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182946.544303 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084524.25 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 108452.43 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291398.969640 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976071.83 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 97607.18 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1389006.152442 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878464.65 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 87846.46 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476852.616965 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790618.18 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 79061.82 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555914.435035 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711556.36 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 71155.64 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627070.071299 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640400.73 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 64040.07 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691110.143936 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576360.65 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 57636.07 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748746.209309 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518724.59 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 51872.46 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800618.668145 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466852.13 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 46685.21 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847303.881097 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420166.92 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 42016.69 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889320.572754 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378150.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 37815.02 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927135.595246 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340335.20 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 34033.52 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961169.115488 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306301.68 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 30630.17 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991799.283706 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275671.51 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 27567.15 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019366.435102 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248104.36 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 24810.44 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044176.871359 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223293.93 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 22329.39 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066506.263990 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200964.53 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 20096.45 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086602.717358 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180868.08 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 18086.81 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104689.525389 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162781.27 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 16278.13 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120967.652617 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146503.15 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 14650.31 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135617.967122 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131852.83 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 13185.28 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148803.250177 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118667.55 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 11866.75 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160670.004926 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106800.79 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 10680.08 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171350.084200 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96120.71 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 9612.07 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180962.155547 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86508.64 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 8650.86 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189613.019759 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77857.78 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 7785.78 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197398.797550 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70072.00 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 7007.20 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204405.997562 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63064.80 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 6306.48 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210712.477572 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56758.32 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 5675.83 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216388.309582 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51082.49 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 5108.25 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221496.558391 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45974.24 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 4597.42 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226093.982318 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41376.82 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 4137.68 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230231.663853 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37239.13 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3723.91 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233955.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33515.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237555.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29915.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241155.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26315.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244755.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22715.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248355.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19115.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251955.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15515.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255555.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11915.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259155.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8315.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262755.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4715.22 sec) 02:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266355.577235 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1115.22 sec) 02:57:00 [GPUGRID] [rr_sim] 2267470.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997185783.67G/439.78G) 02:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851131.71 02:57:00 [---] pick_jobs_to_run() start 02:57:00 [---] pick_jobs_to_run() end 02:57:00 [---] [rr_sim] end 02:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:57:00 [---] [cpu_sched_debug] preliminary job list: 02:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:57:00 [---] [cpu_sched_debug] final job list: 02:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:57:00 [---] [cpu_sched_debug] enforce_run_list: end 02:57:00 [---] choose_project(): 10620.000000 02:57:00 [iThena.Measurements] doing RPC to get work 02:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424979.09 cpu 2168552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:57:00 [---] Got 0 tasks 02:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:58:00 [---] [rr_sim] doing sim: CPU sched 02:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997158193.31G/439.78G) 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267408.06 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 226740.81 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226740.806079 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040667.25 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 204066.73 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430807.531549 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836600.53 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 183660.05 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614467.584473 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652940.48 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 165294.05 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779761.632104 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487646.43 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 148764.64 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928526.274973 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338881.79 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 133888.18 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062414.453554 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204993.61 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 120499.36 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182913.814277 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084494.25 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 108449.42 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291363.238928 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976044.82 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 97604.48 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388967.721114 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878440.34 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 87844.03 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476811.755081 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790596.31 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 79059.63 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555871.385652 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711536.68 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 71153.67 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1627025.053165 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640383.01 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 64038.30 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691063.353927 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576344.71 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 57634.47 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748697.824613 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518710.24 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 51871.02 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800568.848230 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466839.21 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 46683.92 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847252.769486 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420155.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 42015.53 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889268.298616 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378139.76 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 37813.98 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927082.274833 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340325.79 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 34032.58 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961114.853428 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306293.21 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 30629.32 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991744.174164 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275663.89 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 27566.39 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019310.562826 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248097.50 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 24809.75 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044120.312622 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223287.75 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 22328.77 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066449.087439 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200958.97 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 20095.90 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086544.984773 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180863.08 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 18086.31 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104631.292375 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162776.77 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 16277.68 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120908.969216 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146499.09 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 14649.91 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135558.878373 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131849.18 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 13184.92 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148743.796614 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118664.26 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 11866.43 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160610.223031 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106797.84 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 10679.78 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171290.006807 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96118.05 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 9611.81 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180901.812205 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86506.25 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 8650.62 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189552.437063 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77855.62 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 7785.56 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197337.999435 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70070.06 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 7007.01 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204345.005570 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63063.06 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 6306.31 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210651.311092 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56756.75 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 5675.67 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216326.986061 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51081.07 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 5108.11 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221435.093534 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45972.97 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 4597.30 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226032.390259 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41375.67 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 4137.57 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230169.957312 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37238.10 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3723.81 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233893.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33514.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237493.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29914.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241093.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26314.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244693.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22714.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248293.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19114.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251893.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15514.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255493.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11914.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259093.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8314.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262693.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4714.29 sec) 02:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266293.767659 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1114.29 sec) 02:58:00 [GPUGRID] [rr_sim] 2267408.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997158193.31G/439.78G) 02:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851128.97 02:58:00 [---] pick_jobs_to_run() start 02:58:00 [---] pick_jobs_to_run() end 02:58:00 [---] [rr_sim] end 02:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:58:00 [---] [cpu_sched_debug] preliminary job list: 02:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:58:00 [---] [cpu_sched_debug] final job list: 02:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:58:00 [---] [cpu_sched_debug] enforce_run_list: end 02:58:00 [---] choose_project(): 10680.000000 02:58:00 [iThena.Measurements] doing RPC to get work 02:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424919.09 cpu 2168492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:58:00 [---] Got 0 tasks 02:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 02:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 02:59:00 [---] [rr_sim] doing sim: CPU sched 02:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997130602.95G/439.78G) 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267345.32 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 226734.53 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226734.532390 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040610.79 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 204061.08 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430795.611542 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836549.71 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 183654.97 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614450.582778 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652894.74 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 165289.47 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779740.056891 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487605.27 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 148760.53 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928500.583592 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338844.74 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 133884.47 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062385.057623 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204960.27 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 120496.03 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182881.084251 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084464.24 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 108446.42 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291327.508217 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976017.82 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 97601.78 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388929.289785 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878416.03 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 87841.60 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476770.893197 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790574.43 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 79057.44 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555828.336268 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711516.99 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 71151.70 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626980.035031 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640365.29 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 64036.53 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691016.563919 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576328.76 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 57632.88 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748649.439917 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518695.88 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 51869.59 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800519.028316 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466826.30 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 46682.63 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847201.657875 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420143.67 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 42014.37 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889216.024478 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378129.30 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 37812.93 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927028.954420 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340316.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 34031.64 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961060.591369 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306284.73 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 30628.47 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991689.064622 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275656.26 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 27565.63 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019254.690550 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248090.63 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 24809.06 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044063.753886 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223281.57 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 22328.16 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066391.910888 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200953.41 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 20095.34 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086487.252189 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180858.07 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 18085.81 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104573.059361 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162772.26 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 16277.23 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120850.285815 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146495.04 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 14649.50 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135499.789624 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131845.53 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 13184.55 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148684.343052 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118660.98 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 11866.10 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160550.441137 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106794.88 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 10679.49 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171229.929414 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96115.39 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 9611.54 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180841.468863 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86503.86 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 8650.39 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189491.854367 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77853.47 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 7785.35 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197277.201321 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70068.12 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 7006.81 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204284.013579 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63061.31 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 6306.13 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210590.144612 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56755.18 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 5675.52 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216265.662541 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51079.66 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 5107.97 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221373.628677 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45971.70 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 4597.17 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225970.798200 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41374.53 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 4137.45 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230108.250770 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37237.07 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3723.71 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233831.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33513.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237431.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29913.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241031.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26313.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244631.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22713.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248231.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19113.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251831.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15513.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255431.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11913.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259031.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8313.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262631.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4713.37 sec) 02:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266231.958084 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1113.37 sec) 02:59:00 [GPUGRID] [rr_sim] 2267345.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997130602.95G/439.78G) 02:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851126.24 02:59:00 [---] pick_jobs_to_run() start 02:59:00 [---] pick_jobs_to_run() end 02:59:00 [---] [rr_sim] end 02:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 02:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 02:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 02:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 02:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 02:59:00 [---] [cpu_sched_debug] preliminary job list: 02:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:59:00 [---] [cpu_sched_debug] final job list: 02:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 02:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 02:59:00 [---] [cpu_sched_debug] enforce_run_list: end 02:59:00 [---] choose_project(): 10740.000000 02:59:00 [iThena.Measurements] doing RPC to get work 02:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424859.09 cpu 2168432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 02:59:00 [---] Got 0 tasks 03:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:00:00 [---] [rr_sim] doing sim: CPU sched 03:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997103012.60G/439.78G) 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267282.59 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 226728.26 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226728.258702 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040554.33 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 204055.43 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430783.691534 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836498.90 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 183649.89 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614433.581083 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652849.01 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 165284.90 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779718.481677 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487564.11 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 148756.41 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928474.892211 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338807.69 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 133880.77 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062355.661692 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204926.93 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 120492.69 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182848.354225 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084434.23 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 108443.42 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291291.777505 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975990.81 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 97599.08 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388890.858457 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878391.73 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 87839.17 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476730.031313 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790552.56 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 79055.26 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555785.286884 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711497.30 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 71149.73 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626935.016898 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640347.57 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 64034.76 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690969.773910 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576312.81 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 57631.28 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748601.055221 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518681.53 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 51868.15 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800469.208401 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466813.38 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 46681.34 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847150.546263 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420132.04 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 42013.20 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889163.750339 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378118.84 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 37811.88 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926975.634008 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340306.95 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 34030.70 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961006.329309 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306276.26 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 30627.63 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991633.955080 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275648.63 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 27564.86 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019198.818274 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248083.77 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 24808.38 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044007.195149 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223275.39 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 22327.54 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066334.734336 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200947.85 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 20094.79 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086429.519605 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180853.07 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 18085.31 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104514.826347 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162767.76 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 16276.78 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120791.602414 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146490.98 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 14649.10 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135440.700875 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131841.89 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 13184.19 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148624.889490 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118657.70 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 11865.77 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160490.659243 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106791.93 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 10679.19 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171169.852021 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96112.74 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 9611.27 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180781.125521 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86501.46 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 8650.15 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189431.271671 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77851.32 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 7785.13 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197216.403206 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70066.18 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 7006.62 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204223.021588 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63059.57 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 6305.96 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210528.978131 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56753.61 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 5675.36 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216204.339020 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51078.25 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 5107.82 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221312.163820 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45970.42 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 4597.04 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225909.206140 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41373.38 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 4137.34 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230046.544229 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37236.04 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3723.60 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233770.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33512.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237370.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29912.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240970.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26312.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244570.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22712.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248170.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19112.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251770.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15512.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255370.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11912.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258970.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8312.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262570.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4712.44 sec) 03:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266170.148508 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1112.44 sec) 03:00:00 [GPUGRID] [rr_sim] 2267282.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997103012.60G/439.78G) 03:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851123.50 03:00:00 [---] pick_jobs_to_run() start 03:00:00 [---] pick_jobs_to_run() end 03:00:00 [---] [rr_sim] end 03:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:00:00 [---] [cpu_sched_debug] preliminary job list: 03:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:00:00 [---] [cpu_sched_debug] final job list: 03:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:00:00 [---] [cpu_sched_debug] enforce_run_list: end 03:00:00 [---] choose_project(): 10800.000000 03:00:00 [iThena.Measurements] doing RPC to get work 03:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424799.09 cpu 2168372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:00:00 [---] Got 0 tasks 03:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:01:00 [---] [rr_sim] doing sim: CPU sched 03:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997075422.24G/439.78G) 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267219.85 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 226721.99 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226721.985014 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040497.87 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 204049.79 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430771.771527 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836448.08 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 183644.81 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614416.579388 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652803.27 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 165280.33 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779696.906463 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487522.94 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 148752.29 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928449.200831 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338770.65 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 133877.06 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062326.265762 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204893.58 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 120489.36 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182815.624199 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084404.23 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 108440.42 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291256.046793 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975963.80 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 97596.38 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388852.427128 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878367.42 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 87836.74 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476689.169429 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790530.68 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 79053.07 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555742.237500 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711477.61 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 71147.76 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626889.998764 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640329.85 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 64032.99 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690922.983902 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576296.87 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 57629.69 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748552.670526 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518667.18 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 51866.72 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800419.388487 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466800.46 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 46680.05 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847099.434652 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420120.42 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 42012.04 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889111.476201 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378108.37 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 37810.84 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926922.313595 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340297.54 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 34029.75 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960952.067249 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306267.78 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 30626.78 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991578.845538 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275641.00 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 27564.10 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019142.945999 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248076.90 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 24807.69 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043950.636413 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223269.21 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 22326.92 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066277.557785 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200942.29 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 20094.23 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086371.787021 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180848.06 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 18084.81 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104456.593333 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162763.26 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 16276.33 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120732.919013 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146486.93 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 14648.69 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135381.612126 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131838.24 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 13183.82 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148565.435927 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118654.41 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 11865.44 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160430.877349 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106788.97 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 10678.90 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171109.774628 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96110.08 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 9611.01 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180720.782179 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86499.07 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 8649.91 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189370.688975 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77849.16 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 7784.92 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197155.605091 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70064.25 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 7006.42 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204162.029596 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63057.82 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 6305.78 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210467.811651 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56752.04 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 5675.20 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216143.015500 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51076.83 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 5107.68 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221250.698964 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45969.15 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 4596.92 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225847.614081 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41372.24 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 4137.22 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229984.837687 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37235.01 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3723.50 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233708.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33511.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237308.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29911.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240908.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26311.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244508.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22711.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248108.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19111.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251708.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15511.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255308.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11911.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258908.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8311.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262508.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4711.51 sec) 03:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266108.338932 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1111.51 sec) 03:01:00 [GPUGRID] [rr_sim] 2267219.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997075422.24G/439.78G) 03:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851120.76 03:01:00 [---] pick_jobs_to_run() start 03:01:00 [---] pick_jobs_to_run() end 03:01:00 [---] [rr_sim] end 03:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:01:00 [---] [cpu_sched_debug] preliminary job list: 03:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:01:00 [---] [cpu_sched_debug] final job list: 03:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:01:00 [---] [cpu_sched_debug] enforce_run_list: end 03:01:00 [---] choose_project(): 10860.000000 03:01:00 [iThena.Measurements] doing RPC to get work 03:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424739.09 cpu 2168312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:01:00 [---] Got 0 tasks 03:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:02:00 [---] [rr_sim] doing sim: CPU sched 03:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997047831.89G/439.78G) 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267157.11 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 226715.71 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226715.711326 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040441.40 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 204044.14 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430759.851519 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836397.26 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 183639.73 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614399.577693 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652757.54 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 165275.75 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779675.331249 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487481.78 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 148748.18 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928423.509450 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338733.60 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 133873.36 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062296.869831 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204860.24 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 120486.02 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182782.894174 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084374.22 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 108437.42 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291220.316082 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975936.80 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 97593.68 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388813.995799 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878343.12 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 87834.31 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476648.307545 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790508.81 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 79050.88 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555699.188116 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711457.93 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 71145.79 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626844.980631 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640312.13 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 64031.21 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690876.193893 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576280.92 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 57628.09 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748504.285830 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518652.83 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 51865.28 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800369.568572 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466787.54 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 46678.75 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847048.323041 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420108.79 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 42010.88 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889059.202063 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378097.91 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 37809.79 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926868.993182 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340288.12 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 34028.81 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960897.805190 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306259.31 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 30625.93 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991523.735996 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275633.38 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 27563.34 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019087.073723 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248070.04 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 24807.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043894.077676 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223263.04 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 22326.30 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066220.381234 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200936.73 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 20093.67 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086314.054437 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180843.06 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 18084.31 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104398.360319 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162758.75 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 16275.88 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120674.235613 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146482.88 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 14648.29 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135322.523377 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131834.59 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 13183.46 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148505.982365 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118651.13 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 11865.11 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160371.095454 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106786.02 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 10678.60 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171049.697235 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96107.42 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 9610.74 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180660.438837 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86496.67 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 8649.67 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189310.106279 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77847.01 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 7784.70 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197094.806977 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70062.31 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 7006.23 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204101.037605 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63056.08 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 6305.61 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210406.645170 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56750.47 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 5675.05 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216081.691979 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51075.42 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 5107.54 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221189.234107 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45967.88 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 4596.79 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225786.022022 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41371.09 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 4137.11 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229923.131145 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37233.98 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3723.40 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233646.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33510.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237246.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29910.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240846.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26310.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244446.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22710.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248046.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19110.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251646.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15510.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255246.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11910.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258846.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8310.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262446.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4710.58 sec) 03:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2266046.529357 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1110.58 sec) 03:02:00 [GPUGRID] [rr_sim] 2267157.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997047831.89G/439.78G) 03:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851118.03 03:02:00 [---] pick_jobs_to_run() start 03:02:00 [---] pick_jobs_to_run() end 03:02:00 [---] [rr_sim] end 03:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:02:00 [---] [cpu_sched_debug] preliminary job list: 03:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:02:00 [---] [cpu_sched_debug] final job list: 03:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:02:00 [---] [cpu_sched_debug] enforce_run_list: end 03:02:00 [---] choose_project(): 10920.000000 03:02:00 [iThena.Measurements] doing RPC to get work 03:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424679.09 cpu 2168252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:02:00 [---] Got 0 tasks 03:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:03:00 [---] [rr_sim] doing sim: CPU sched 03:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (997020241.53G/439.78G) 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267094.38 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 226709.44 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226709.437638 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040384.94 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 204038.49 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430747.931511 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836346.44 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 183634.64 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614382.575998 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652711.80 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 165271.18 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779653.756035 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487440.62 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 148744.06 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928397.818069 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338696.56 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 133869.66 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062267.473900 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204826.90 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 120482.69 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182750.164148 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084344.21 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 108434.42 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291184.585370 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975909.79 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 97590.98 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388775.564471 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878318.81 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 87831.88 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476607.445661 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790486.93 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 79048.69 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555656.138733 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711438.24 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 71143.82 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626799.962497 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640294.41 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 64029.44 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690829.403885 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576264.97 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 57626.50 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748455.901134 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518638.48 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 51863.85 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800319.748658 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466774.63 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 46677.46 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846997.211430 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420097.16 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 42009.72 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889006.927924 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378087.45 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 37808.74 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926815.672769 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340278.70 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 34027.87 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960843.543130 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306250.83 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 30625.08 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991468.626455 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275625.75 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 27562.57 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019031.201447 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248063.17 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 24806.32 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043837.518940 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223256.86 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 22325.69 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066163.204683 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200931.17 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 20093.12 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086256.321852 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180838.05 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 18083.81 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104340.127305 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162754.25 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 16275.42 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120615.552212 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146478.82 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 14647.88 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135263.434628 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131830.94 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 13183.09 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148446.528803 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118647.85 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 11864.78 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160311.313560 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106783.06 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 10678.31 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170989.619842 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96104.76 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 9610.48 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180600.095495 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86494.28 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 8649.43 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189249.523583 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77844.85 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 7784.49 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197034.008862 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70060.37 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 7006.04 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204040.045614 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63054.33 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 6305.43 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210345.478690 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56748.90 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 5674.89 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216020.368458 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51074.01 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 5107.40 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221127.769250 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45966.61 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 4596.66 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225724.429963 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41369.95 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 4136.99 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229861.424604 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37232.95 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3723.30 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233584.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33509.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237184.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29909.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240784.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26309.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244384.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22709.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247984.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19109.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251584.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15509.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255184.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11909.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258784.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8309.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262384.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4709.66 sec) 03:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265984.719781 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1109.66 sec) 03:03:00 [GPUGRID] [rr_sim] 2267094.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (997020241.53G/439.78G) 03:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851115.29 03:03:00 [---] pick_jobs_to_run() start 03:03:00 [---] pick_jobs_to_run() end 03:03:00 [---] [rr_sim] end 03:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:03:00 [---] [cpu_sched_debug] preliminary job list: 03:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:03:00 [---] [cpu_sched_debug] final job list: 03:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:03:00 [---] [cpu_sched_debug] enforce_run_list: end 03:03:00 [---] choose_project(): 10980.000000 03:03:00 [iThena.Measurements] doing RPC to get work 03:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424619.09 cpu 2168192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:03:00 [---] Got 0 tasks 03:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:04:00 [---] [rr_sim] doing sim: CPU sched 03:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996992651.17G/439.78G) 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2267031.64 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 226703.16 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226703.163949 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040328.48 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 204032.85 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430736.011504 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836295.63 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 183629.56 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614365.574303 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652666.07 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 165266.61 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779632.180822 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487399.46 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 148739.95 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928372.126689 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338659.51 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 133865.95 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062238.077969 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204793.56 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 120479.36 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182717.434122 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084314.21 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 108431.42 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291148.854659 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975882.78 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 97588.28 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388737.133142 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878294.51 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 87829.45 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476566.583777 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790465.06 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 79046.51 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555613.089349 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711418.55 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 71141.86 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626754.944363 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640276.70 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 64027.67 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690782.613876 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576249.03 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 57624.90 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748407.516438 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518624.12 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 51862.41 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800269.928744 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466761.71 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 46676.17 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846946.099819 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420085.54 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 42008.55 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888954.653786 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378076.99 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 37807.70 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926762.352357 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340269.29 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 34026.93 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960789.281070 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306242.36 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 30624.24 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991413.516913 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275618.12 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 27561.81 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018975.329171 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248056.31 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 24805.63 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043780.960203 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223250.68 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 22325.07 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066106.028132 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200925.61 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 20092.56 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086198.589268 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180833.05 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 18083.31 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104281.894291 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162749.75 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 16274.97 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120556.868811 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146474.77 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 14647.48 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135204.345879 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131827.29 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 13182.73 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148387.075241 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118644.56 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 11864.46 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160251.531666 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106780.11 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 10678.01 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170929.542449 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96102.10 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 9610.21 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180539.752153 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86491.89 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 8649.19 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189188.940887 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77842.70 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 7784.27 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196973.210748 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70058.43 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 7005.84 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203979.053622 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63052.59 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 6305.26 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210284.312209 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56747.33 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 5674.73 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215959.044938 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51072.59 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 5107.26 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221066.304393 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45965.34 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 4596.53 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225662.837903 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41368.80 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 4136.88 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229799.718062 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37231.92 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3723.19 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233522.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33508.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237122.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29908.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240722.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26308.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244322.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22708.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247922.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19108.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251522.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15508.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255122.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11908.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258722.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8308.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262322.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4708.73 sec) 03:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265922.910205 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1108.73 sec) 03:04:00 [GPUGRID] [rr_sim] 2267031.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996992651.17G/439.78G) 03:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851112.55 03:04:00 [---] pick_jobs_to_run() start 03:04:00 [---] pick_jobs_to_run() end 03:04:00 [---] [rr_sim] end 03:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:04:00 [---] [cpu_sched_debug] preliminary job list: 03:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:04:00 [---] [cpu_sched_debug] final job list: 03:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:04:00 [---] [cpu_sched_debug] enforce_run_list: end 03:04:00 [---] choose_project(): 11040.000000 03:04:00 [iThena.Measurements] doing RPC to get work 03:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424559.09 cpu 2168132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:04:00 [---] Got 0 tasks 03:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:05:00 [---] [rr_sim] doing sim: CPU sched 03:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996965060.82G/439.78G) 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266968.90 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 226696.89 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226696.890261 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040272.01 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 204027.20 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430724.091496 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836244.81 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 183624.48 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614348.572608 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652620.33 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 165262.03 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779610.605608 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487358.30 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 148735.83 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928346.435308 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338622.47 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 133862.25 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062208.682039 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204760.22 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 120476.02 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182684.704096 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084284.20 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 108428.42 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291113.123947 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975855.78 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 97585.58 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388698.701814 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878270.20 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 87827.02 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476525.721893 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790443.18 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 79044.32 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555570.039965 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711398.86 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 71139.89 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626709.926230 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640258.98 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 64025.90 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690735.823868 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576233.08 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 57623.31 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748359.131742 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518609.77 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 51860.98 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800220.108829 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466748.79 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 46674.88 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846894.988207 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420073.91 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 42007.39 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888902.379648 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378066.52 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 37806.65 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926709.031944 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340259.87 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 34025.99 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960735.019011 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306233.88 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 30623.39 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991358.407371 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275610.50 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 27561.05 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018919.456895 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248049.45 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 24804.94 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043724.401466 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223244.50 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 22324.45 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066048.851581 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200920.05 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 20092.01 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086140.856684 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180828.05 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 18082.80 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104223.661277 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162745.24 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 16274.52 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120498.185410 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146470.72 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 14647.07 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135145.257130 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131823.65 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 13182.36 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148327.621678 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118641.28 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 11864.13 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160191.749771 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106777.15 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 10677.72 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170869.465055 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96099.44 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 9609.94 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180479.408811 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86489.49 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 8648.95 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189128.358191 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77840.54 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 7784.05 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196912.412633 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70056.49 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 7005.65 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203918.061631 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63050.84 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 6305.08 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210223.145729 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56745.76 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 5674.58 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215897.721417 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51071.18 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 5107.12 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221004.839536 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45964.06 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 4596.41 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225601.245844 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41367.66 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 4136.77 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229738.011521 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37230.89 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3723.09 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233461.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33507.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237061.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29907.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240661.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26307.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244261.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22707.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247861.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19107.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251461.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15507.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255061.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11907.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258661.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8307.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262261.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4707.80 sec) 03:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265861.100630 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1107.80 sec) 03:05:00 [GPUGRID] [rr_sim] 2266968.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996965060.82G/439.78G) 03:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851109.82 03:05:00 [---] pick_jobs_to_run() start 03:05:00 [---] pick_jobs_to_run() end 03:05:00 [---] [rr_sim] end 03:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:05:00 [---] [cpu_sched_debug] preliminary job list: 03:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:05:00 [---] [cpu_sched_debug] final job list: 03:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:05:00 [---] [cpu_sched_debug] enforce_run_list: end 03:05:00 [---] choose_project(): 11100.000000 03:05:00 [iThena.Measurements] doing RPC to get work 03:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424499.09 cpu 2168072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:05:00 [---] Got 0 tasks 03:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:06:00 [---] [rr_sim] doing sim: CPU sched 03:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996937470.46G/439.78G) 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266906.17 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 226690.62 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226690.616573 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040215.55 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 204021.55 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430712.171488 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836193.99 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 183619.40 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614331.570913 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652574.59 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 165257.46 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779589.030394 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487317.14 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 148731.71 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928320.743928 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338585.42 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 133858.54 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062179.286108 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204726.88 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 120472.69 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182651.974070 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084254.19 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 108425.42 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291077.393236 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975828.77 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 97582.88 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388660.270485 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878245.90 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 87824.59 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476484.860009 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790421.31 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 79042.13 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555526.990581 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711379.18 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 71137.92 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626664.908096 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640241.26 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 64024.13 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690689.033859 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576217.13 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 57621.71 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748310.747046 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518595.42 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 51859.54 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800170.288915 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466735.88 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 46673.59 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846843.876596 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420062.29 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 42006.23 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888850.105509 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378056.06 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 37805.61 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926655.711531 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340250.45 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 34025.05 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960680.756951 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306225.41 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 30622.54 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991303.297829 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275602.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 27560.29 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018863.584619 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248042.58 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 24804.26 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043667.842730 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223238.32 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 22323.83 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065991.675030 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200914.49 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 20091.45 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086083.124100 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180823.04 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 18082.30 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104165.428263 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162740.74 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 16274.07 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120439.502009 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146466.66 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 14646.67 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135086.168381 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131820.00 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 13182.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148268.168116 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118638.00 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 11863.80 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160131.967877 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106774.20 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 10677.42 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170809.387662 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96096.78 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 9609.68 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180419.065469 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86487.10 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 8648.71 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189067.775495 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77838.39 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 7783.84 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196851.614518 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70054.55 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 7005.46 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203857.069639 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63049.10 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 6304.91 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210161.979248 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56744.19 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 5674.42 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215836.397896 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51069.77 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 5106.98 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220943.374680 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45962.79 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 4596.28 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225539.653785 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41366.51 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 4136.65 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229676.304979 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37229.86 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3722.99 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233399.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33506.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236999.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29906.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240599.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26306.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244199.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22706.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247799.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19106.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251399.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15506.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254999.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11906.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258599.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8306.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262199.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4706.87 sec) 03:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265799.291054 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1106.87 sec) 03:06:00 [GPUGRID] [rr_sim] 2266906.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996937470.46G/439.78G) 03:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851107.08 03:06:00 [---] pick_jobs_to_run() start 03:06:00 [---] pick_jobs_to_run() end 03:06:00 [---] [rr_sim] end 03:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:06:00 [---] [cpu_sched_debug] preliminary job list: 03:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:06:00 [---] [cpu_sched_debug] final job list: 03:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:06:00 [---] [cpu_sched_debug] enforce_run_list: end 03:06:00 [---] choose_project(): 11160.000000 03:06:00 [iThena.Measurements] doing RPC to get work 03:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424439.09 cpu 2168012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2168012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2168012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:06:00 [---] Got 0 tasks 03:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:07:00 [---] [rr_sim] doing sim: CPU sched 03:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996909880.10G/439.78G) 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266843.43 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 226684.34 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226684.342885 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040159.09 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 204015.91 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430700.251481 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836143.18 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 183614.32 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614314.569217 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652528.86 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 165252.89 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779567.455180 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487275.97 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 148727.60 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928295.052547 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338548.38 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 133854.84 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062149.890177 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204693.54 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 120469.35 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182619.244044 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084224.18 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 108422.42 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291041.662524 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975801.77 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 97580.18 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388621.839156 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878221.59 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 87822.16 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476443.998125 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790399.43 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 79039.94 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555483.941198 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711359.49 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 71135.95 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626619.889963 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640223.54 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 64022.35 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690642.243851 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576201.18 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 57620.12 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748262.362351 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518581.07 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 51858.11 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800120.469000 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466722.96 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 46672.30 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846792.764985 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420050.66 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 42005.07 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888797.831371 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378045.60 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 37804.56 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926602.391119 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340241.04 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 34024.10 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960626.494891 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306216.93 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 30621.69 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991248.188287 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275595.24 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 27559.52 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018807.712343 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248035.72 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 24803.57 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043611.283993 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223232.14 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 22323.21 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065934.498479 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200908.93 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 20090.89 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086025.391515 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180818.04 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 18081.80 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104107.195249 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162736.23 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 16273.62 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120380.818608 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146462.61 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 14646.26 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135027.079632 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131816.35 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 13181.63 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148208.714554 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118634.71 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 11863.47 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160072.185983 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106771.24 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 10677.12 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170749.310269 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96094.12 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 9609.41 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180358.722127 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86484.71 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 8648.47 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189007.192799 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77836.24 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 7783.62 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196790.816404 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70052.61 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 7005.26 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203796.077648 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63047.35 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 6304.74 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210100.812768 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56742.62 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 5674.26 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215775.074376 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51068.35 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 5106.84 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220881.909823 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45961.52 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 4596.15 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225478.061725 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41365.37 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 4136.54 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229614.598437 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37228.83 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3722.88 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233337.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33505.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236937.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29905.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240537.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26305.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244137.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22705.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247737.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19105.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251337.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15505.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254937.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11905.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258537.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8305.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262137.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4705.95 sec) 03:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265737.481478 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1105.95 sec) 03:07:00 [GPUGRID] [rr_sim] 2266843.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996909880.10G/439.78G) 03:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851104.34 03:07:00 [---] pick_jobs_to_run() start 03:07:00 [---] pick_jobs_to_run() end 03:07:00 [---] [rr_sim] end 03:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:07:00 [---] [cpu_sched_debug] preliminary job list: 03:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:07:00 [---] [cpu_sched_debug] final job list: 03:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:07:00 [---] [cpu_sched_debug] enforce_run_list: end 03:07:00 [---] choose_project(): 11220.000000 03:07:00 [iThena.Measurements] doing RPC to get work 03:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424379.09 cpu 2167952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:07:00 [---] Got 0 tasks 03:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:08:00 [---] [rr_sim] doing sim: CPU sched 03:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996909880.10G/439.78G) 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266843.43 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 226684.34 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226684.342885 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040159.09 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 204015.91 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430700.251481 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836143.18 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 183614.32 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614314.569217 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652528.86 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 165252.89 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779567.455180 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487275.97 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 148727.60 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928295.052547 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338548.38 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 133854.84 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062149.890177 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204693.54 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 120469.35 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182619.244044 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084224.18 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 108422.42 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291041.662524 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975801.77 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 97580.18 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388621.839156 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878221.59 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 87822.16 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476443.998125 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790399.43 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 79039.94 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555483.941198 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711359.49 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 71135.95 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626619.889963 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640223.54 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 64022.35 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690642.243851 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576201.18 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 57620.12 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748262.362351 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518581.07 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 51858.11 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800120.469000 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466722.96 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 46672.30 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846792.764985 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420050.66 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 42005.07 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888797.831371 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378045.60 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 37804.56 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926602.391119 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340241.04 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 34024.10 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960626.494891 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306216.93 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 30621.69 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991248.188287 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275595.24 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 27559.52 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018807.712343 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248035.72 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 24803.57 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043611.283993 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223232.14 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 22323.21 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065934.498479 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200908.93 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 20090.89 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086025.391515 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180818.04 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 18081.80 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104107.195249 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162736.23 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 16273.62 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120380.818608 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146462.61 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 14646.26 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135027.079632 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131816.35 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 13181.63 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148208.714554 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118634.71 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 11863.47 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160072.185983 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106771.24 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 10677.12 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170749.310269 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96094.12 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 9609.41 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180358.722127 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86484.71 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 8648.47 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189007.192799 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77836.24 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 7783.62 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196790.816404 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70052.61 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 7005.26 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203796.077648 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63047.35 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 6304.74 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210100.812768 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56742.62 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 5674.26 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215775.074376 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51068.35 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 5106.84 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220881.909823 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45961.52 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 4596.15 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225478.061725 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41365.37 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 4136.54 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229614.598437 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37228.83 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3722.88 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233337.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33505.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236937.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29905.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240537.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26305.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244137.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22705.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247737.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19105.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251337.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15505.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254937.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11905.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258537.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8305.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262137.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4705.95 sec) 03:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265737.481478 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1105.95 sec) 03:08:00 [GPUGRID] [rr_sim] 2266843.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996909880.10G/439.78G) 03:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851164.34 03:08:00 [---] pick_jobs_to_run() start 03:08:00 [---] pick_jobs_to_run() end 03:08:00 [---] [rr_sim] end 03:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:08:00 [---] [cpu_sched_debug] preliminary job list: 03:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:08:00 [---] [cpu_sched_debug] final job list: 03:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:08:00 [---] [cpu_sched_debug] enforce_run_list: end 03:08:00 [---] choose_project(): 11280.000000 03:08:00 [iThena.Measurements] doing RPC to get work 03:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424319.09 cpu 2167952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:08:00 [---] Got 0 tasks 03:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:09:00 [---] [rr_sim] doing sim: CPU sched 03:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996882289.75G/439.78G) 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266780.69 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 226678.07 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226678.069196 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040102.62 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 204010.26 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430688.331473 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836092.36 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 183609.24 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614297.567522 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652483.12 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 165248.31 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779545.879967 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487234.81 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 148723.48 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928269.361166 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338511.33 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 133851.13 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062120.494246 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204660.20 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 120466.02 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182586.514018 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084194.18 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 108419.42 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291005.931813 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975774.76 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 97577.48 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388583.407828 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878197.28 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 87819.73 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476403.136242 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790377.56 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 79037.76 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555440.891814 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711339.80 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 71133.98 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626574.871829 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640205.82 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 64020.58 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690595.453842 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576185.24 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 57618.52 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748213.977655 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518566.71 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 51856.67 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800070.649086 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466710.04 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 46671.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846741.653374 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420039.04 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 42003.90 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888745.557233 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378035.13 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 37803.51 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926549.070706 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340231.62 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 34023.16 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960572.232832 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306208.46 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 30620.85 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991193.078745 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275587.61 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 27558.76 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018751.840067 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248028.85 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 24802.89 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043554.725257 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223225.97 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 22322.60 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065877.321927 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200903.37 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 20090.34 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085967.658931 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180813.03 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 18081.30 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104048.962235 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162731.73 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 16273.17 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120322.135208 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146458.56 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 14645.86 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134967.990883 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131812.70 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 13181.27 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148149.260991 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118631.43 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 11863.14 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160012.404089 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106768.29 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 10676.83 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170689.232876 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96091.46 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 9609.15 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180298.378785 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86482.31 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 8648.23 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188946.610103 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77834.08 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 7783.41 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196730.018289 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70050.67 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 7005.07 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203735.085657 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63045.61 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 6304.56 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210039.646288 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56741.05 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 5674.10 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215713.750855 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51066.94 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 5106.69 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220820.444966 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45960.25 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 4596.02 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225416.469666 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41364.22 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 4136.42 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229552.891896 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37227.80 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3722.78 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233275.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33505.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236875.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29905.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240475.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26305.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244075.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22705.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247675.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19105.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251275.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15505.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254875.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11905.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258475.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8305.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262075.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4705.02 sec) 03:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265675.671903 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1105.02 sec) 03:09:00 [GPUGRID] [rr_sim] 2266780.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996882289.75G/439.78G) 03:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851161.60 03:09:00 [---] pick_jobs_to_run() start 03:09:00 [---] pick_jobs_to_run() end 03:09:00 [---] [rr_sim] end 03:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:09:00 [---] [cpu_sched_debug] preliminary job list: 03:09:00 [---] [cpu_sched_debug] final job list: 03:09:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 03:09:00 [---] [work_fetch] Request work fetch: CPUs idle 03:09:00 [---] [cpu_sched_debug] enforce_run_list: end 03:09:00 [---] choose_project(): 11340.000000 03:09:00 [iThena.Measurements] doing RPC to get work 03:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 597059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424259.09 cpu 2167892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:09:00 [---] Got 0 tasks 03:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:10:00 [---] [rr_sim] doing sim: CPU sched 03:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996882289.75G/439.78G) 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266780.69 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 226678.07 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226678.069196 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040102.62 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 204010.26 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430688.331473 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836092.36 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 183609.24 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614297.567522 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652483.12 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 165248.31 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779545.879967 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487234.81 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 148723.48 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928269.361166 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338511.33 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 133851.13 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062120.494246 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204660.20 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 120466.02 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182586.514018 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084194.18 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 108419.42 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1291005.931813 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975774.76 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 97577.48 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388583.407828 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878197.28 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 87819.73 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476403.136242 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790377.56 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 79037.76 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555440.891814 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711339.80 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 71133.98 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626574.871829 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640205.82 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 64020.58 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690595.453842 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576185.24 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 57618.52 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748213.977655 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518566.71 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 51856.67 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800070.649086 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466710.04 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 46671.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846741.653374 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420039.04 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 42003.90 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888745.557233 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378035.13 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 37803.51 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926549.070706 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340231.62 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 34023.16 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960572.232832 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306208.46 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 30620.85 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991193.078745 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275587.61 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 27558.76 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018751.840067 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248028.85 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 24802.89 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043554.725257 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223225.97 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 22322.60 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065877.321927 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200903.37 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 20090.34 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085967.658931 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180813.03 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 18081.30 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104048.962235 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162731.73 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 16273.17 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120322.135208 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146458.56 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 14645.86 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134967.990883 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131812.70 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 13181.27 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148149.260991 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118631.43 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 11863.14 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160012.404089 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106768.29 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 10676.83 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170689.232876 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96091.46 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 9609.15 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180298.378785 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86482.31 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 8648.23 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188946.610103 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77834.08 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 7783.41 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196730.018289 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70050.67 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 7005.07 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203735.085657 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63045.61 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 6304.56 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210039.646288 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56741.05 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 5674.10 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215713.750855 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51066.94 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 5106.69 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220820.444966 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45960.25 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 4596.02 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225416.469666 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41364.22 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 4136.42 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229552.891896 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37227.80 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3722.78 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233275.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33505.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236875.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29905.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240475.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26305.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244075.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22705.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247675.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19105.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251275.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15505.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254875.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11905.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258475.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8305.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262075.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4705.02 sec) 03:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265675.671903 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1105.02 sec) 03:10:00 [GPUGRID] [rr_sim] 2266780.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996882289.75G/439.78G) 03:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851221.60 03:10:00 [---] pick_jobs_to_run() start 03:10:00 [---] pick_jobs_to_run() end 03:10:00 [---] [rr_sim] end 03:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:10:00 [---] [cpu_sched_debug] preliminary job list: 03:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:10:00 [---] [cpu_sched_debug] final job list: 03:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 03:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:10:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 996882289.75G 03:10:00 [---] [cpu_sched_debug] enforce_run_list: end 03:10:00 [---] choose_project(): 11400.000000 03:10:00 [iThena.Measurements] doing RPC to get work 03:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424199.09 cpu 2167892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:10:00 [---] Got 0 tasks 03:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:11:00 [---] [rr_sim] doing sim: CPU sched 03:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996854699.39G/439.78G) 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266717.96 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 226671.80 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226671.795508 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2040046.16 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 204004.62 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430676.411466 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1836041.54 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 183604.15 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614280.565827 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652437.39 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 165243.74 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779524.304753 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487193.65 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 148719.37 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928243.669786 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338474.29 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 133847.43 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062091.098315 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204626.86 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 120462.69 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182553.783992 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084164.17 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 108416.42 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290970.201101 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975747.75 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 97574.78 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388544.976499 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878172.98 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 87817.30 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476362.274358 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790355.68 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 79035.57 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555397.842430 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711320.11 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 71132.01 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626529.853695 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640188.10 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 64018.81 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690548.663834 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576169.29 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 57616.93 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748165.592959 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518552.36 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 51855.24 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800020.829171 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466697.13 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 46669.71 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846690.541762 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420027.41 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 42002.74 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888693.283094 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378024.67 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 37802.47 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926495.750293 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340222.20 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 34022.22 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960517.970772 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306199.98 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 30620.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991137.969203 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275579.99 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 27558.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018695.967791 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248021.99 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 24802.20 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043498.166520 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223219.79 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 22321.98 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065820.145376 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200897.81 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 20089.78 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085909.926347 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180808.03 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 18080.80 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103990.729221 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162727.23 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 16272.72 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120263.451807 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146454.50 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 14645.45 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134908.902134 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131809.05 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 13180.91 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148089.807429 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118628.15 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 11862.81 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159952.622194 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106765.33 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 10676.53 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170629.155483 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96088.80 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 9608.88 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180238.035443 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86479.92 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 8647.99 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188886.027407 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77831.93 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 7783.19 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196669.220175 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70048.73 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 7004.87 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203674.093665 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63043.86 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 6304.39 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209978.479807 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56739.48 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 5673.95 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215652.427335 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51065.53 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 5106.55 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220758.980109 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45958.97 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 4595.90 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225354.877607 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41363.08 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 4136.31 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229491.185354 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37226.77 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3722.68 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233213.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33504.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236813.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29904.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240413.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26304.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244013.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22704.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247613.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19104.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251213.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15504.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254813.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11904.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258413.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8304.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262013.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4704.09 sec) 03:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265613.862327 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1104.09 sec) 03:11:00 [GPUGRID] [rr_sim] 2266717.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996854699.39G/439.78G) 03:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851218.87 03:11:00 [---] pick_jobs_to_run() start 03:11:00 [---] pick_jobs_to_run() end 03:11:00 [---] [rr_sim] end 03:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:11:00 [---] [cpu_sched_debug] preliminary job list: 03:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 03:11:00 [---] [cpu_sched_debug] final job list: 03:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:11:00 [---] [cpu_sched_debug] enforce_run_list: end 03:11:00 [---] choose_project(): 11460.000000 03:11:00 [iThena.Measurements] doing RPC to get work 03:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424139.09 cpu 2167832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:11:00 [---] Got 0 tasks 03:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:12:00 [---] [rr_sim] doing sim: CPU sched 03:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996827109.04G/439.78G) 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266655.22 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 226665.52 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226665.521820 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039989.70 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 203998.97 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430664.491458 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835990.73 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 183599.07 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614263.564132 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652391.65 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 165239.17 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779502.729539 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487152.49 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 148715.25 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928217.978405 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338437.24 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 133843.72 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062061.702385 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204593.52 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 120459.35 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182521.053966 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084134.16 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 108413.42 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290934.470390 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975720.75 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 97572.07 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388506.545171 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878148.67 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 87814.87 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476321.412474 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790333.81 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 79033.38 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555354.793046 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711300.43 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 71130.04 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626484.835562 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640170.38 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 64017.04 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690501.873826 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576153.34 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 57615.33 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748117.208263 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518538.01 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 51853.80 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799971.009257 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466684.21 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 46668.42 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846639.430151 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420015.79 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 42001.58 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888641.008956 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378014.21 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 37801.42 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926442.429880 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340212.79 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 34021.28 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960463.708712 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306191.51 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 30619.15 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991082.859661 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275572.36 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 27557.24 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018640.095515 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248015.12 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 24801.51 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043441.607784 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223213.61 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 22321.36 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065762.968825 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200892.25 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 20089.22 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085852.193763 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180803.02 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 18080.30 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103932.496206 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162722.72 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 16272.27 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120204.768406 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146450.45 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 14645.04 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134849.813385 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131805.40 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 13180.54 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148030.353867 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118624.86 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 11862.49 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159892.840300 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106762.38 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 10676.24 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170569.078090 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96086.14 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 9608.61 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180177.692101 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86477.53 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 8647.75 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188825.444711 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77829.77 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 7782.98 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196608.422060 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70046.80 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 7004.68 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203613.101674 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63042.12 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 6304.21 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209917.313327 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56737.90 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 5673.79 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215591.103814 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51064.11 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 5106.41 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220697.515253 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45957.70 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 4595.77 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225293.285547 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41361.93 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 4136.19 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229429.478813 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37225.74 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3722.57 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233152.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33503.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236752.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29903.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240352.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26303.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243952.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22703.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247552.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19103.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251152.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15503.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254752.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11903.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258352.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8303.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261952.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4703.17 sec) 03:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265552.052751 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1103.17 sec) 03:12:00 [GPUGRID] [rr_sim] 2266655.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996827109.04G/439.78G) 03:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851216.13 03:12:00 [---] pick_jobs_to_run() start 03:12:00 [---] pick_jobs_to_run() end 03:12:00 [---] [rr_sim] end 03:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:12:00 [---] [cpu_sched_debug] preliminary job list: 03:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:12:00 [---] [cpu_sched_debug] final job list: 03:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:12:00 [---] [cpu_sched_debug] enforce_run_list: end 03:12:00 [---] choose_project(): 11520.000000 03:12:00 [iThena.Measurements] doing RPC to get work 03:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424079.09 cpu 2167772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:12:00 [---] Got 0 tasks 03:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:13:00 [---] [rr_sim] doing sim: CPU sched 03:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996799518.68G/439.78G) 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266592.48 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 226659.25 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226659.248132 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039933.23 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 203993.32 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430652.571450 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835939.91 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 183593.99 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614246.562437 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652345.92 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 165234.59 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779481.154325 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487111.33 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 148711.13 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928192.287025 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338400.19 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 133840.02 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062032.306454 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204560.17 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 120456.02 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182488.323940 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084104.16 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 108410.42 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290898.739678 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975693.74 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 97569.37 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388468.113842 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878124.37 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 87812.44 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476280.550590 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790311.93 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 79031.19 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555311.743663 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711280.74 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 71128.07 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626439.817428 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640152.66 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 64015.27 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690455.083817 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576137.40 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 57613.74 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748068.823567 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518523.66 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 51852.37 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799921.189342 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466671.29 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 46667.13 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846588.318540 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420004.16 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 42000.42 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888588.734818 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378003.75 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 37800.37 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926389.109468 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340203.37 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 34020.34 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960409.446653 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306183.03 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 30618.30 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991027.750119 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275564.73 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 27556.47 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018584.223239 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248008.26 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 24800.83 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043385.049047 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223207.43 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 22320.74 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065705.792274 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200886.69 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 20088.67 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085794.461179 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180798.02 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 18079.80 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103874.263192 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162718.22 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 16271.82 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120146.085005 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146446.40 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 14644.64 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134790.724636 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131801.76 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 13180.18 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147970.900304 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118621.58 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 11862.16 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159833.058406 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106759.42 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 10675.94 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170509.000697 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96083.48 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 9608.35 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180117.348759 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86475.13 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 8647.51 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188764.862015 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77827.62 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 7782.76 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196547.623945 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70044.86 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 7004.49 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203552.109683 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63040.37 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 6304.04 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209856.146846 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56736.33 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 5673.63 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215529.780293 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51062.70 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 5106.27 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220636.050396 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45956.43 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 4595.64 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225231.693488 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41360.79 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 4136.08 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229367.772271 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37224.71 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3722.47 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233090.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33502.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236690.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29902.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240290.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26302.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243890.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22702.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247490.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19102.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251090.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15502.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254690.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11902.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258290.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8302.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261890.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4702.24 sec) 03:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265490.243176 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1102.24 sec) 03:13:00 [GPUGRID] [rr_sim] 2266592.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996799518.68G/439.78G) 03:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851213.39 03:13:00 [---] pick_jobs_to_run() start 03:13:00 [---] pick_jobs_to_run() end 03:13:00 [---] [rr_sim] end 03:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:13:00 [---] [cpu_sched_debug] preliminary job list: 03:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:13:00 [---] [cpu_sched_debug] final job list: 03:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:13:00 [---] [cpu_sched_debug] enforce_run_list: end 03:13:00 [---] choose_project(): 11580.000000 03:13:00 [iThena.Measurements] doing RPC to get work 03:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 424019.09 cpu 2167712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:13:00 [---] Got 0 tasks 03:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:14:00 [---] [rr_sim] doing sim: CPU sched 03:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996771928.32G/439.78G) 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266529.74 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 226652.97 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226652.974444 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039876.77 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 203987.68 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430640.651443 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835889.09 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 183588.91 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614229.560742 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652300.18 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 165230.02 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779459.579111 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487070.17 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 148707.02 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928166.595644 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338363.15 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 133836.31 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1062002.910523 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204526.83 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 120452.68 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182455.593914 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084074.15 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 108407.42 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290863.008967 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975666.74 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 97566.67 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388429.682513 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878100.06 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 87810.01 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476239.688706 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790290.06 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 79029.01 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555268.694279 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711261.05 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 71126.11 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626394.799294 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640134.95 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 64013.49 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690408.293809 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576121.45 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 57612.15 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748020.438871 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518509.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 51850.93 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799871.369428 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466658.38 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 46665.84 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846537.206929 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419992.54 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 41999.25 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888536.460679 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377993.28 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 37799.33 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926335.789055 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340193.96 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 34019.40 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960355.184593 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306174.56 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 30617.46 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990972.640577 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275557.10 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 27555.71 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018528.350963 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248001.39 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 24800.14 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043328.490310 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223201.25 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 22320.13 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065648.615723 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200881.13 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 20088.11 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085736.728594 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180793.02 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 18079.30 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103816.030178 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162713.71 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 16271.37 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120087.401604 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146442.34 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 14644.23 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134731.635887 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131798.11 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 13179.81 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147911.446742 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118618.30 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 11861.83 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159773.276512 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106756.47 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 10675.65 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170448.923304 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96080.82 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 9608.08 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180057.005417 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86472.74 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 8647.27 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188704.279319 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77825.47 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 7782.55 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196486.825831 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70042.92 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 7004.29 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203491.117691 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63038.63 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 6303.86 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209794.980366 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56734.76 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 5673.48 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215468.456773 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51061.29 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 5106.13 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220574.585539 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45955.16 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 4595.52 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225170.101429 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41359.64 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 4135.96 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229306.065729 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37223.68 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3722.37 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233028.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33501.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236628.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29901.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240228.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26301.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243828.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22701.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247428.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19101.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251028.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15501.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254628.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11901.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258228.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8301.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261828.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4701.31 sec) 03:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265428.433600 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1101.31 sec) 03:14:00 [GPUGRID] [rr_sim] 2266529.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996771928.32G/439.78G) 03:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851210.66 03:14:00 [---] pick_jobs_to_run() start 03:14:00 [---] pick_jobs_to_run() end 03:14:00 [---] [rr_sim] end 03:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:14:00 [---] [cpu_sched_debug] preliminary job list: 03:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:14:00 [---] [cpu_sched_debug] final job list: 03:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:14:00 [---] [cpu_sched_debug] enforce_run_list: end 03:14:00 [---] choose_project(): 11640.000000 03:14:00 [iThena.Measurements] doing RPC to get work 03:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423959.09 cpu 2167652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:14:00 [---] Got 0 tasks 03:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:15:00 [---] [rr_sim] doing sim: CPU sched 03:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996744337.97G/439.78G) 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266467.01 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 226646.70 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226646.700755 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039820.31 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 203982.03 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430628.731435 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835838.28 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 183583.83 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614212.559047 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652254.45 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 165225.44 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779438.003898 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1487029.00 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 148702.90 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928140.904263 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338326.10 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 133832.61 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061973.514592 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204493.49 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 120449.35 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182422.863888 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084044.14 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 108404.41 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290827.278255 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975639.73 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 97563.97 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388391.251185 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878075.76 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 87807.58 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476198.826822 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790268.18 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 79026.82 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555225.644895 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711241.36 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 71124.14 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626349.781161 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640117.23 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 64011.72 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690361.503800 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576105.50 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 57610.55 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747972.054175 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518494.95 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 51849.50 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799821.549513 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466645.46 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 46664.55 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846486.095317 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419980.91 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 41998.09 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888484.186541 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377982.82 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 37798.28 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926282.468642 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340184.54 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 34018.45 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960300.922533 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306166.09 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 30616.61 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990917.531035 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275549.48 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 27554.95 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018472.478687 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247994.53 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 24799.45 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043271.931574 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223195.08 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 22319.51 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065591.439172 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200875.57 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 20087.56 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085678.996010 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180788.01 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 18078.80 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103757.797164 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162709.21 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 16270.92 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120028.718203 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146438.29 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 14643.83 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134672.547138 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131794.46 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 13179.45 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147851.993180 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118615.01 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 11861.50 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159713.494617 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106753.51 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 10675.35 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170388.845911 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96078.16 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 9607.82 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179996.662075 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86470.35 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 8647.03 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188643.696623 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77823.31 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 7782.33 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196426.027716 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70040.98 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 7004.10 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203430.125700 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63036.88 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 6303.69 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209733.813885 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56733.19 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 5673.32 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215407.133252 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51059.87 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 5105.99 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220513.120682 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45953.89 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 4595.39 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225108.509369 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41358.50 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 4135.85 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229244.359188 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37222.65 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3722.26 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232966.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33500.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236566.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29900.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240166.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26300.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243766.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22700.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247366.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19100.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250966.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15500.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254566.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11900.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258166.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8300.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261766.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4700.38 sec) 03:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265366.624024 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1100.38 sec) 03:15:00 [GPUGRID] [rr_sim] 2266467.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996744337.97G/439.78G) 03:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851207.92 03:15:00 [---] pick_jobs_to_run() start 03:15:00 [---] pick_jobs_to_run() end 03:15:00 [---] [rr_sim] end 03:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:15:00 [---] [cpu_sched_debug] preliminary job list: 03:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:15:00 [---] [cpu_sched_debug] final job list: 03:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:15:00 [---] [cpu_sched_debug] enforce_run_list: end 03:15:00 [---] choose_project(): 11700.000000 03:15:00 [iThena.Measurements] doing RPC to get work 03:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423899.09 cpu 2167592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:15:00 [---] Got 0 tasks 03:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:16:00 [---] [rr_sim] doing sim: CPU sched 03:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996716747.61G/439.78G) 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266404.27 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 226640.43 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226640.427067 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039763.84 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 203976.38 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430616.811428 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835787.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 183578.75 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614195.557352 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652208.71 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 165220.87 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779416.428684 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486987.84 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 148698.78 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928115.212883 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338289.06 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 133828.91 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061944.118662 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204460.15 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 120446.02 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182390.133863 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084014.14 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 108401.41 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290791.547543 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975612.72 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 97561.27 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388352.819856 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878051.45 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 87805.15 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476157.964938 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790246.31 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 79024.63 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555182.595511 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711221.68 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 71122.17 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626304.763027 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640099.51 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 64009.95 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690314.713792 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576089.56 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 57608.96 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747923.669480 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518480.60 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 51848.06 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799771.729599 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466632.54 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 46663.25 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846434.983706 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419969.29 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 41996.93 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888431.912403 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377972.36 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 37797.24 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926229.148230 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340175.12 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 34017.51 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960246.660474 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306157.61 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 30615.76 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990862.421494 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275541.85 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 27554.18 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018416.606411 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247987.66 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 24798.77 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043215.372837 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223188.90 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 22318.89 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065534.262621 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200870.01 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 20087.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085621.263426 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180783.01 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 18078.30 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103699.564150 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162704.71 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 16270.47 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119970.034803 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146434.24 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 14643.42 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134613.458389 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131790.81 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 13179.08 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147792.539618 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118611.73 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 11861.17 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159653.712723 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106750.56 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 10675.06 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170328.768518 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96075.50 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 9607.55 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179936.318733 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86467.95 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 8646.80 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188583.113927 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77821.16 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 7782.12 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196365.229602 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70039.04 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 7003.90 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203369.133709 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63035.14 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 6303.51 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209672.647405 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56731.62 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 5673.16 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215345.809732 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51058.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 5105.85 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220451.655826 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45952.61 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 4595.26 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225046.917310 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41357.35 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 4135.74 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229182.652646 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37221.62 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3722.16 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232904.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33499.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236504.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29899.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240104.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26299.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243704.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22699.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247304.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19099.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250904.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15499.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254504.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11899.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258104.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8299.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261704.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4699.46 sec) 03:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265304.814449 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1099.46 sec) 03:16:00 [GPUGRID] [rr_sim] 2266404.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996716747.61G/439.78G) 03:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851205.18 03:16:00 [---] pick_jobs_to_run() start 03:16:00 [---] pick_jobs_to_run() end 03:16:00 [---] [rr_sim] end 03:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:16:00 [---] [cpu_sched_debug] preliminary job list: 03:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:16:00 [---] [cpu_sched_debug] final job list: 03:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:16:00 [---] [cpu_sched_debug] enforce_run_list: end 03:16:00 [---] choose_project(): 11760.000000 03:16:00 [iThena.Measurements] doing RPC to get work 03:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423839.09 cpu 2167532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:16:00 [---] Got 0 tasks 03:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:17:00 [---] [rr_sim] doing sim: CPU sched 03:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996689157.25G/439.78G) 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266341.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 226634.15 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226634.153379 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039707.38 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 203970.74 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430604.891420 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835736.64 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 183573.66 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614178.555657 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652162.98 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 165216.30 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779394.853470 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486946.68 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 148694.67 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928089.521502 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338252.01 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 133825.20 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061914.722731 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204426.81 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 120442.68 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182357.403837 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083984.13 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 108398.41 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290755.816832 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975585.72 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 97558.57 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388314.388528 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878027.15 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 87802.71 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476117.103054 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790224.43 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 79022.44 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555139.546127 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711201.99 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 71120.20 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626259.744894 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640081.79 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 64008.18 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690267.923783 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576073.61 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 57607.36 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747875.284784 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518466.25 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 51846.62 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799721.909684 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466619.62 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 46661.96 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846383.872095 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419957.66 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 41995.77 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888379.638264 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377961.90 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 37796.19 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926175.827817 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340165.71 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 34016.57 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960192.398414 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306149.14 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 30614.91 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990807.311952 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275534.22 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 27553.42 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018360.734135 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247980.80 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 24798.08 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043158.814101 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223182.72 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 22318.27 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065477.086070 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200864.45 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 20086.44 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085563.530842 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180778.00 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 18077.80 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103641.331136 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162700.20 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 16270.02 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119911.351402 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146430.18 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 14643.02 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134554.369640 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131787.16 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 13178.72 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147733.086055 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118608.45 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 11860.84 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159593.930829 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106747.60 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 10674.76 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170268.691125 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96072.84 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 9607.28 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179875.975391 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86465.56 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 8646.56 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188522.531231 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77819.00 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 7781.90 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196304.431487 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70037.10 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 7003.71 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203308.141717 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63033.39 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 6303.34 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209611.480924 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56730.05 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 5673.01 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215284.486211 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51057.05 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 5105.70 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220390.190969 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45951.34 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 4595.13 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224985.325251 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41356.21 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 4135.62 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229120.946105 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37220.59 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3722.06 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232843.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33498.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236443.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29898.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240043.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26298.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243643.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22698.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247243.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19098.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250843.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15498.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254443.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11898.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258043.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8298.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261643.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4698.53 sec) 03:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265243.004873 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1098.53 sec) 03:17:00 [GPUGRID] [rr_sim] 2266341.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996689157.25G/439.78G) 03:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851202.45 03:17:00 [---] pick_jobs_to_run() start 03:17:00 [---] pick_jobs_to_run() end 03:17:00 [---] [rr_sim] end 03:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:17:00 [---] [cpu_sched_debug] preliminary job list: 03:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:17:00 [---] [cpu_sched_debug] final job list: 03:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:17:00 [---] [cpu_sched_debug] enforce_run_list: end 03:17:00 [---] choose_project(): 11820.000000 03:17:00 [iThena.Measurements] doing RPC to get work 03:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423779.09 cpu 2167472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:17:00 [---] Got 0 tasks 03:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:18:00 [---] [rr_sim] doing sim: CPU sched 03:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996661566.90G/439.78G) 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266278.80 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 226627.88 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226627.879691 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039650.92 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 203965.09 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430592.971412 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835685.83 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 183568.58 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614161.553962 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652117.24 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 165211.72 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779373.278256 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486905.52 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 148690.55 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928063.830121 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338214.97 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 133821.50 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061885.326800 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204393.47 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 120439.35 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182324.673811 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083954.12 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 108395.41 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290720.086120 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975558.71 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 97555.87 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388275.957199 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878002.84 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 87800.28 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476076.241170 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790202.56 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 79020.26 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555096.496744 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711182.30 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 71118.23 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626214.726760 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640064.07 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 64006.41 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690221.133775 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576057.66 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 57605.77 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747826.900088 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518451.90 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 51845.19 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799672.089770 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466606.71 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 46660.67 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846332.760484 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419946.04 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 41994.60 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888327.364126 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377951.43 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 37795.14 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926122.507404 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340156.29 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 34015.63 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960138.136354 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306140.66 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 30614.07 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990752.202410 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275526.59 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 27552.66 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018304.861859 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247973.94 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 24797.39 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043102.255364 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223176.54 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 22317.65 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065419.909519 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200858.89 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 20085.89 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085505.798257 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180773.00 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 18077.30 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103583.098122 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162695.70 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 16269.57 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119852.668001 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146426.13 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 14642.61 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134495.280891 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131783.52 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 13178.35 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147673.632493 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118605.16 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 11860.52 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159534.148934 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106744.65 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 10674.46 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170208.613732 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96070.18 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 9607.02 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179815.632049 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86463.16 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 8646.32 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188461.948535 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77816.85 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 7781.68 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196243.633372 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70035.16 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 7003.52 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203247.149726 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63031.65 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 6303.16 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209550.314444 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56728.48 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 5672.85 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215223.162690 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51055.63 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 5105.56 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220328.726112 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45950.07 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 4595.01 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224923.733191 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41355.06 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 4135.51 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229059.239563 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37219.56 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3721.96 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232781.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33497.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236381.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29897.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239981.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26297.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243581.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22697.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247181.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19097.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250781.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15497.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254381.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11897.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257981.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8297.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261581.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4697.60 sec) 03:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265181.195297 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1097.60 sec) 03:18:00 [GPUGRID] [rr_sim] 2266278.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996661566.90G/439.78G) 03:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851199.71 03:18:00 [---] pick_jobs_to_run() start 03:18:00 [---] pick_jobs_to_run() end 03:18:00 [---] [rr_sim] end 03:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:18:00 [---] [cpu_sched_debug] preliminary job list: 03:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:18:00 [---] [cpu_sched_debug] final job list: 03:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:18:00 [---] [cpu_sched_debug] enforce_run_list: end 03:18:00 [---] choose_project(): 11880.000000 03:18:00 [iThena.Measurements] doing RPC to get work 03:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423719.09 cpu 2167412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:18:00 [---] Got 0 tasks 03:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:19:00 [---] [rr_sim] doing sim: CPU sched 03:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996633976.54G/439.78G) 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266216.06 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 226621.61 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226621.606003 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039594.45 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 203959.45 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430581.051405 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835635.01 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 183563.50 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614144.552267 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652071.51 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 165207.15 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779351.703043 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486864.36 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 148686.44 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928038.138741 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338177.92 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 133817.79 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061855.930869 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204360.13 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 120436.01 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182291.943785 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083924.12 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 108392.41 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290684.355409 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975531.70 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 97553.17 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388237.525870 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877978.53 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 87797.85 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1476035.379286 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790180.68 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 79018.07 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555053.447360 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711162.61 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 71116.26 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626169.708626 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640046.35 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 64004.64 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690174.343766 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576041.72 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 57604.17 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747778.515392 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518437.54 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 51843.75 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799622.269855 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466593.79 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 46659.38 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846281.648872 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419934.41 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 41993.44 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888275.089988 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377940.97 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 37794.10 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926069.186991 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340146.87 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 34014.69 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960083.874295 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306132.19 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 30613.22 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990697.092868 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275518.97 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 27551.90 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018248.989584 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247967.07 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 24796.71 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043045.696628 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223170.36 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 22317.04 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065362.732967 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200853.33 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 20085.33 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085448.065673 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180767.99 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 18076.80 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103524.865108 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162691.19 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 16269.12 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119793.984600 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146422.08 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 14642.21 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134436.192143 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131779.87 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 13177.99 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147614.178931 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118601.88 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 11860.19 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159474.367040 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106741.69 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 10674.17 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170148.536339 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96067.52 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 9606.75 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179755.288707 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86460.77 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 8646.08 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188401.365839 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77814.69 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 7781.47 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196182.835258 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70033.22 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 7003.32 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203186.157734 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63029.90 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 6302.99 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209489.147963 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56726.91 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 5672.69 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215161.839170 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51054.22 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 5105.42 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220267.261255 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45948.80 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 4594.88 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224862.141132 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41353.92 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 4135.39 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228997.533021 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37218.53 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3721.85 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232719.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33496.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236319.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29896.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239919.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26296.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243519.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22696.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247119.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19096.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250719.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15496.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254319.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11896.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257919.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8296.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261519.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4696.67 sec) 03:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265119.385722 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1096.67 sec) 03:19:00 [GPUGRID] [rr_sim] 2266216.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996633976.54G/439.78G) 03:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851196.97 03:19:00 [---] pick_jobs_to_run() start 03:19:00 [---] pick_jobs_to_run() end 03:19:00 [---] [rr_sim] end 03:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:19:00 [---] [cpu_sched_debug] preliminary job list: 03:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:19:00 [---] [cpu_sched_debug] final job list: 03:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:19:00 [---] [cpu_sched_debug] enforce_run_list: end 03:19:00 [---] choose_project(): 11940.000000 03:19:00 [iThena.Measurements] doing RPC to get work 03:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423659.09 cpu 2167352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:19:00 [---] Got 0 tasks 03:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:20:00 [---] [rr_sim] doing sim: CPU sched 03:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996606386.19G/439.78G) 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266153.32 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 226615.33 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226615.332314 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039537.99 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 203953.80 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430569.131397 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835584.19 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 183558.42 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614127.550572 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1652025.77 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 165202.58 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779330.127829 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486823.20 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 148682.32 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 928012.447360 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338140.88 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 133814.09 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061826.534939 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204326.79 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 120432.68 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182259.213759 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083894.11 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 108389.41 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290648.624697 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975504.70 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 97550.47 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388199.094542 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877954.23 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 87795.42 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475994.517402 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790158.81 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 79015.88 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1555010.397976 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711142.93 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 71114.29 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626124.690493 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640028.63 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 64002.86 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690127.553758 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576025.77 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 57602.58 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747730.130696 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518423.19 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 51842.32 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799572.449941 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466580.87 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 46658.09 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846230.537261 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419922.79 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 41992.28 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888222.815849 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377930.51 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 37793.05 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926015.866579 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340137.46 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 34013.75 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960029.612235 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306123.71 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 30612.37 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990641.983326 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275511.34 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 27551.13 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018193.117308 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247960.21 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 24796.02 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042989.137891 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223164.19 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 22316.42 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065305.556416 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200847.77 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 20084.78 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085390.333089 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180762.99 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 18076.30 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103466.632094 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162686.69 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 16268.67 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119735.301199 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146418.02 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 14641.80 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134377.103394 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131776.22 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 13177.62 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147554.725368 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118598.60 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 11859.86 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159414.585146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106738.74 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 10673.87 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170088.458946 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96064.86 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 9606.49 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179694.945365 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86458.38 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 8645.84 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188340.783143 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77812.54 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 7781.25 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196122.037143 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70031.29 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 7003.13 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203125.165743 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63028.16 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 6302.82 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209427.981483 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56725.34 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 5672.53 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215100.515649 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51052.81 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 5105.28 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220205.796398 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45947.53 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 4594.75 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224800.549073 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41352.77 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 4135.28 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228935.826480 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37217.50 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3721.75 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232657.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33495.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236257.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29895.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239857.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26295.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243457.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22695.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247057.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19095.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250657.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15495.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254257.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11895.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257857.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8295.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261457.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4695.75 sec) 03:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2265057.576146 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1095.75 sec) 03:20:00 [GPUGRID] [rr_sim] 2266153.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996606386.19G/439.78G) 03:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851194.24 03:20:00 [---] pick_jobs_to_run() start 03:20:00 [---] pick_jobs_to_run() end 03:20:00 [---] [rr_sim] end 03:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:20:00 [---] [cpu_sched_debug] preliminary job list: 03:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:20:00 [---] [cpu_sched_debug] final job list: 03:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:20:00 [---] [cpu_sched_debug] enforce_run_list: end 03:20:00 [---] choose_project(): 12000.000000 03:20:00 [iThena.Measurements] doing RPC to get work 03:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423599.09 cpu 2167292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:20:00 [---] Got 0 tasks 03:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:21:00 [---] [rr_sim] doing sim: CPU sched 03:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996578795.83G/439.78G) 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266090.59 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 226609.06 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226609.058626 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039481.53 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 203948.15 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430557.211390 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835533.37 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 183553.34 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614110.548877 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651980.04 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 165198.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779308.552615 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486782.03 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 148678.20 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927986.755980 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338103.83 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 133810.38 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061797.139008 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204293.45 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 120429.34 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182226.483733 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083864.10 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 108386.41 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290612.893986 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975477.69 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 97547.77 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388160.663213 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877929.92 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 87792.99 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475953.655518 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790136.93 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 79013.69 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554967.348592 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711123.24 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 71112.32 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626079.672359 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640010.91 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 64001.09 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690080.763749 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576009.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 57600.98 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747681.746000 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518408.84 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 51840.88 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799522.630026 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466567.96 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 46656.80 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846179.425650 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419911.16 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 41991.12 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888170.541711 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377920.04 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 37792.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925962.546166 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340128.04 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 34012.80 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959975.350175 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306115.24 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 30611.52 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990586.873784 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275503.71 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 27550.37 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018137.245032 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247953.34 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 24795.33 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042932.579155 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223158.01 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 22315.80 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065248.379865 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200842.21 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 20084.22 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085332.600505 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180757.99 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 18075.80 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103408.399080 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162682.19 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 16268.22 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119676.617798 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146413.97 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 14641.40 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134318.014645 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131772.57 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 13177.26 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147495.271806 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118595.31 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 11859.53 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159354.803252 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106735.78 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 10673.58 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170028.381553 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96062.20 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 9606.22 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179634.602023 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86455.98 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 8645.60 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188280.200447 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77810.39 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 7781.04 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196061.239028 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70029.35 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 7002.93 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203064.173752 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63026.41 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 6302.64 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209366.815003 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56723.77 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 5672.38 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215039.192128 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51051.39 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 5105.14 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220144.331542 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45946.25 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 4594.63 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224738.957014 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41351.63 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 4135.16 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228874.119938 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37216.47 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3721.65 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232595.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33494.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236195.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29894.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239795.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26294.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243395.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22694.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246995.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19094.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250595.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15494.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254195.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11894.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257795.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8294.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261395.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4694.82 sec) 03:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264995.766571 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1094.82 sec) 03:21:00 [GPUGRID] [rr_sim] 2266090.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996578795.83G/439.78G) 03:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851191.50 03:21:00 [---] pick_jobs_to_run() start 03:21:00 [---] pick_jobs_to_run() end 03:21:00 [---] [rr_sim] end 03:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:21:00 [---] [cpu_sched_debug] preliminary job list: 03:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:21:00 [---] [cpu_sched_debug] final job list: 03:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:21:00 [---] [cpu_sched_debug] enforce_run_list: end 03:21:00 [---] choose_project(): 12060.000000 03:21:00 [iThena.Measurements] doing RPC to get work 03:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423539.09 cpu 2167232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:21:00 [---] Got 0 tasks 03:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:22:00 [---] [rr_sim] doing sim: CPU sched 03:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996551205.47G/439.78G) 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2266027.85 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 226602.78 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226602.784938 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039425.06 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 203942.51 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430545.291382 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835482.56 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 183548.26 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614093.547182 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651934.30 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 165193.43 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779286.977401 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486740.87 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 148674.09 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927961.064599 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338066.78 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 133806.68 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061767.743077 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204260.11 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 120426.01 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182193.753707 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083834.10 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 108383.41 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290577.163274 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975450.69 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 97545.07 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388122.231885 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877905.62 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 87790.56 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475912.793634 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790115.06 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 79011.51 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554924.299209 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711103.55 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 71110.36 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1626034.654226 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639993.20 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 63999.32 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690033.973741 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575993.88 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 57599.39 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747633.361305 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518394.49 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 51839.45 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799472.810112 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466555.04 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 46655.50 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846128.314039 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419899.54 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 41989.95 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888118.267573 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377909.58 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 37790.96 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925909.225753 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340118.62 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 34011.86 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959921.088116 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306106.76 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 30610.68 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990531.764242 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275496.09 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 27549.61 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018081.372756 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247946.48 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 24794.65 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042876.020418 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223151.83 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 22315.18 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065191.203314 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200836.65 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 20083.66 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085274.867920 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180752.98 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 18075.30 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103350.166066 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162677.68 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 16267.77 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119617.934398 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146409.91 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 14640.99 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134258.925896 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131768.92 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 13176.89 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147435.818244 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118592.03 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 11859.20 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159295.021357 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106732.83 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 10673.28 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169968.304160 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96059.55 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 9605.95 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179574.258681 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86453.59 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 8645.36 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188219.617751 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77808.23 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 7780.82 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196000.440914 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70027.41 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 7002.74 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203003.181760 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63024.67 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 6302.47 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209305.648522 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56722.20 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 5672.22 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214977.868608 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51049.98 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 5105.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220082.866685 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45944.98 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 4594.50 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224677.364954 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41350.48 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 4135.05 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228812.413397 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37215.44 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3721.54 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232533.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33493.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236133.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29893.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239733.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26293.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243333.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22693.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246933.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19093.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250533.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15493.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254133.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11893.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257733.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8293.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261333.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4693.89 sec) 03:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264933.956995 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1093.89 sec) 03:22:00 [GPUGRID] [rr_sim] 2266027.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996551205.47G/439.78G) 03:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851188.76 03:22:00 [---] pick_jobs_to_run() start 03:22:00 [---] pick_jobs_to_run() end 03:22:00 [---] [rr_sim] end 03:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:22:00 [---] [cpu_sched_debug] preliminary job list: 03:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:22:00 [---] [cpu_sched_debug] final job list: 03:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:22:00 [---] [cpu_sched_debug] enforce_run_list: end 03:22:00 [---] choose_project(): 12120.000000 03:22:00 [iThena.Measurements] doing RPC to get work 03:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423479.09 cpu 2167172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:22:00 [---] Got 0 tasks 03:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:23:00 [---] [rr_sim] doing sim: CPU sched 03:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996523615.12G/439.78G) 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265965.11 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 226596.51 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226596.511250 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039368.60 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 203936.86 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430533.371374 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835431.74 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 183543.17 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614076.545487 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651888.57 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 165188.86 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779265.402188 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486699.71 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 148669.97 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927935.373218 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1338029.74 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 133802.97 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061738.347146 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204226.77 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 120422.68 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182161.023681 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083804.09 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 108380.41 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290541.432563 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975423.68 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 97542.37 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388083.800556 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877881.31 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 87788.13 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475871.931750 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790093.18 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 79009.32 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554881.249825 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711083.86 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 71108.39 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625989.636092 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639975.48 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 63997.55 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689987.183732 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575977.93 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 57597.79 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747584.976609 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518380.14 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 51838.01 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799422.990198 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466542.12 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 46654.21 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846077.202427 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419887.91 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 41988.79 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888065.993434 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377899.12 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 37789.91 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925855.905341 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340109.21 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 34010.92 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959866.826056 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306098.29 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 30609.83 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990476.654700 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275488.46 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 27548.85 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018025.500480 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247939.61 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 24793.96 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042819.461681 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223145.65 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 22314.57 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065134.026763 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200831.09 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 20083.11 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085217.135336 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180747.98 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 18074.80 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103291.933052 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162673.18 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 16267.32 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119559.250997 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146405.86 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 14640.59 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134199.837147 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131765.28 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 13176.53 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147376.364682 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118588.75 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 11858.87 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159235.239463 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106729.87 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 10672.99 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169908.226766 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96056.89 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 9605.69 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179513.915339 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86451.20 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 8645.12 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188159.035055 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77806.08 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 7780.61 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195939.642799 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70025.47 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 7002.55 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202942.189769 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63022.92 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 6302.29 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209244.482042 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56720.63 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 5672.06 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214916.545087 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51048.57 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 5104.86 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220021.401828 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45943.71 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 4594.37 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224615.772895 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41349.34 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 4134.93 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228750.706855 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37214.41 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3721.44 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232472.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33492.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236072.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29892.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239672.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26292.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243272.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22692.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246872.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19092.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250472.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15492.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254072.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11892.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257672.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8292.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261272.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4692.97 sec) 03:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264872.147419 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1092.97 sec) 03:23:00 [GPUGRID] [rr_sim] 2265965.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996523615.12G/439.78G) 03:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851186.03 03:23:00 [---] pick_jobs_to_run() start 03:23:00 [---] pick_jobs_to_run() end 03:23:00 [---] [rr_sim] end 03:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:23:00 [---] [cpu_sched_debug] preliminary job list: 03:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:23:00 [---] [cpu_sched_debug] final job list: 03:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:23:00 [---] [cpu_sched_debug] enforce_run_list: end 03:23:00 [---] choose_project(): 12180.000000 03:23:00 [iThena.Measurements] doing RPC to get work 03:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423419.09 cpu 2167112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:23:00 [---] Got 0 tasks 03:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:24:00 [---] [rr_sim] doing sim: CPU sched 03:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996496024.76G/439.78G) 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265902.38 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 226590.24 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226590.237561 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039312.14 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 203931.21 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430521.451367 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835380.92 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 183538.09 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614059.543791 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651842.83 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 165184.28 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779243.826974 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486658.55 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 148665.85 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927909.681838 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337992.69 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 133799.27 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061708.951215 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204193.42 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 120419.34 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182128.293655 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083774.08 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 108377.41 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290505.701851 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975396.67 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 97539.67 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388045.369228 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877857.01 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 87785.70 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475831.069866 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790071.31 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 79007.13 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554838.200441 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711064.18 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 71106.42 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625944.617958 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639957.76 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 63995.78 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689940.393724 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575961.98 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 57596.20 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747536.591913 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518365.78 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 51836.58 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799373.170283 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466529.21 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 46652.92 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846026.090816 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419876.28 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 41987.63 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888013.719296 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377888.66 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 37788.87 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925802.584928 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340099.79 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 34009.98 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959812.563996 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306089.81 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 30608.98 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990421.545158 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275480.83 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 27548.08 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017969.628204 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247932.75 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 24793.27 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042762.902945 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223139.47 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 22313.95 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065076.850212 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200825.53 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 20082.55 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085159.402752 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180742.97 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 18074.30 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103233.700038 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162668.68 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 16266.87 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119500.567596 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146401.81 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 14640.18 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134140.748398 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131761.63 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 13176.16 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147316.911119 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118585.46 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 11858.55 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159175.457569 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106726.92 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 10672.69 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169848.149373 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96054.23 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 9605.42 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179453.571997 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86448.80 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 8644.88 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188098.452359 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77803.92 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 7780.39 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195878.844685 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70023.53 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 7002.35 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202881.197778 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63021.18 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 6302.12 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209183.315561 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56719.06 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 5671.91 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214855.221567 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51047.15 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 5104.72 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219959.936971 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45942.44 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 4594.24 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224554.180836 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41348.19 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 4134.82 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228689.000313 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37213.38 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3721.34 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232410.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33492.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236010.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29892.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239610.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26292.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243210.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22692.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246810.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19092.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250410.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15492.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254010.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11892.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257610.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8292.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261210.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4692.04 sec) 03:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264810.337844 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1092.04 sec) 03:24:00 [GPUGRID] [rr_sim] 2265902.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996496024.76G/439.78G) 03:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851183.29 03:24:00 [---] pick_jobs_to_run() start 03:24:00 [---] pick_jobs_to_run() end 03:24:00 [---] [rr_sim] end 03:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:24:00 [---] [cpu_sched_debug] preliminary job list: 03:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:24:00 [---] [cpu_sched_debug] final job list: 03:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:24:00 [---] [cpu_sched_debug] enforce_run_list: end 03:24:00 [---] choose_project(): 12240.000000 03:24:00 [iThena.Measurements] doing RPC to get work 03:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423359.09 cpu 2167052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2167052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2167052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:24:00 [---] Got 0 tasks 03:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:25:00 [---] [rr_sim] doing sim: CPU sched 03:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996468434.40G/439.78G) 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265839.64 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 226583.96 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226583.963873 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039255.67 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 203925.57 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430509.531359 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835330.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 183533.01 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614042.542096 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651797.10 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 165179.71 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779222.251760 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486617.39 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 148661.74 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927883.990457 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337955.65 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 133795.56 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061679.555285 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204160.08 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 120416.01 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182095.563629 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083744.08 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 108374.41 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290469.971140 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975369.67 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 97536.97 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1388006.937899 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877832.70 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 87783.27 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475790.207982 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790049.43 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 79004.94 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554795.151057 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711044.49 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 71104.45 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625899.599825 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639940.04 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 63994.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689893.603715 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575946.04 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 57594.60 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747488.207217 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518351.43 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 51835.14 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799323.350369 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466516.29 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 46651.63 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845974.979205 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419864.66 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 41986.47 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887961.445158 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377878.19 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 37787.82 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925749.264515 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340090.37 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 34009.04 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959758.301937 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306081.34 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 30608.13 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990366.435616 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275473.20 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 27547.32 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017913.755928 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247925.88 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 24792.59 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042706.344208 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223133.29 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 22313.33 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065019.673661 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200819.97 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 20082.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085101.670168 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180737.97 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 18073.80 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103175.467024 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162664.17 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 16266.42 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119441.884195 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146397.75 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 14639.78 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134081.659649 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131757.98 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 13175.80 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147257.457557 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118582.18 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 11858.22 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159115.675675 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106723.96 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 10672.40 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169788.071980 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96051.57 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 9605.16 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179393.228655 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86446.41 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 8644.64 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188037.869663 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77801.77 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 7780.18 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195818.046570 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70021.59 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 7002.16 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202820.205786 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63019.43 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 6301.94 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209122.149081 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56717.49 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 5671.75 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214793.898046 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51045.74 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 5104.57 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219898.472115 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45941.17 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 4594.12 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224492.588776 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41347.05 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 4134.70 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228627.293772 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37212.34 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3721.23 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232348.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33491.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235948.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29891.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239548.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26291.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243148.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22691.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246748.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19091.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250348.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15491.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253948.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11891.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257548.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8291.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261148.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4691.11 sec) 03:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264748.528268 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1091.11 sec) 03:25:00 [GPUGRID] [rr_sim] 2265839.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996468434.40G/439.78G) 03:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851180.55 03:25:00 [---] pick_jobs_to_run() start 03:25:00 [---] pick_jobs_to_run() end 03:25:00 [---] [rr_sim] end 03:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:25:00 [---] [cpu_sched_debug] preliminary job list: 03:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:25:00 [---] [cpu_sched_debug] final job list: 03:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:25:00 [---] [cpu_sched_debug] enforce_run_list: end 03:25:00 [---] choose_project(): 12300.000000 03:25:00 [iThena.Measurements] doing RPC to get work 03:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423299.09 cpu 2166992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:25:00 [---] Got 0 tasks 03:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:26:00 [---] [rr_sim] doing sim: CPU sched 03:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996440844.05G/439.78G) 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265776.90 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 226577.69 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226577.690185 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039199.21 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 203919.92 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430497.611351 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835279.29 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 183527.93 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614025.540401 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651751.36 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 165175.14 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779200.676546 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486576.23 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 148657.62 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927858.299077 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337918.60 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 133791.86 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061650.159354 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204126.74 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 120412.67 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182062.833603 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083714.07 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 108371.41 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290434.240428 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975342.66 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 97534.27 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387968.506570 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877808.40 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 87780.84 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475749.346098 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790027.56 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 79002.76 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554752.101673 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711024.80 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 71102.48 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625854.581691 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639922.32 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 63992.23 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689846.813707 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575930.09 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 57593.01 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747439.822521 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518337.08 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 51833.71 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799273.530454 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466503.37 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 46650.34 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845923.867594 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419853.03 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 41985.30 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887909.171019 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377867.73 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 37786.77 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925695.944102 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340080.96 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 34008.10 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959704.039877 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306072.86 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 30607.29 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990311.326074 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275465.58 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 27546.56 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017857.883652 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247919.02 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 24791.90 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042649.785472 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223127.12 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 22312.71 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064962.497110 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200814.40 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 20081.44 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085043.937584 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180732.96 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 18073.30 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103117.234010 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162659.67 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 16265.97 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119383.200794 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146393.70 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 14639.37 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134022.570900 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131754.33 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 13175.43 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147198.003995 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118578.90 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 11857.89 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159055.893780 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106721.01 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 10672.10 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169727.994587 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96048.91 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 9604.89 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179332.885314 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86444.02 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 8644.40 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187977.286967 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77799.61 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 7779.96 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195757.248455 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70019.65 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 7001.97 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202759.213795 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63017.69 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 6301.77 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209060.982600 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56715.92 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 5671.59 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214732.574525 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51044.33 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 5104.43 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219837.007258 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45939.89 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 4593.99 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224430.996717 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41345.91 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 4134.59 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228565.587230 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37211.31 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3721.13 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232286.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33490.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235886.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29890.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239486.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26290.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243086.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22690.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246686.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19090.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250286.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15490.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253886.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11890.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257486.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8290.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261086.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4690.18 sec) 03:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264686.718692 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1090.18 sec) 03:26:00 [GPUGRID] [rr_sim] 2265776.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996440844.05G/439.78G) 03:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851177.81 03:26:00 [---] pick_jobs_to_run() start 03:26:00 [---] pick_jobs_to_run() end 03:26:00 [---] [rr_sim] end 03:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:26:00 [---] [cpu_sched_debug] preliminary job list: 03:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:26:00 [---] [cpu_sched_debug] final job list: 03:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:26:00 [---] [cpu_sched_debug] enforce_run_list: end 03:26:00 [---] choose_project(): 12360.000000 03:26:00 [iThena.Measurements] doing RPC to get work 03:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 596039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423239.09 cpu 2166932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:26:00 [---] Got 0 tasks 03:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:27:00 [---] [rr_sim] doing sim: CPU sched 03:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996413253.69G/439.78G) 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265714.16 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 226571.42 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226571.416497 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039142.75 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 203914.27 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430485.691344 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835228.47 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 183522.85 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 614008.538706 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651705.63 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 165170.56 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779179.101332 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486535.06 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 148653.51 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927832.607696 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337881.56 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 133788.16 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061620.763423 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204093.40 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 120409.34 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1182030.103578 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083684.06 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 108368.41 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290398.509717 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975315.66 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 97531.57 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387930.075242 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877784.09 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 87778.41 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475708.484214 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790005.68 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 79000.57 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554709.052290 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711005.11 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 71100.51 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625809.563557 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639904.60 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 63990.46 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689800.023698 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575914.14 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 57591.41 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747391.437825 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518322.73 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 51832.27 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799223.710540 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466490.45 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 46649.05 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845872.755982 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419841.41 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 41984.14 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887856.896881 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377857.27 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 37785.73 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925642.623690 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340071.54 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 34007.15 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959649.777817 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306064.39 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 30606.44 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990256.216532 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275457.95 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 27545.79 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017802.011376 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247912.15 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 24791.22 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042593.226735 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223120.94 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 22312.09 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064905.320558 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200808.84 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 20080.88 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084986.204999 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180727.96 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 18072.80 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103059.000996 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162655.16 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 16265.52 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119324.517393 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146389.65 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 14638.96 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133963.482151 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131750.68 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 13175.07 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147138.550432 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118575.61 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 11857.56 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158996.111886 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106718.05 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 10671.81 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169667.917194 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96046.25 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 9604.62 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179272.541972 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86441.62 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 8644.16 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187916.704271 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77797.46 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 7779.75 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195696.450341 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70017.71 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 7001.77 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202698.221804 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63015.94 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 6301.59 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208999.816120 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56714.35 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 5671.43 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214671.251005 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51042.91 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 5104.29 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219775.542401 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45938.62 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 4593.86 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224369.404658 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41344.76 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 4134.48 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228503.880689 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37210.28 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3721.03 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232224.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33489.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235824.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29889.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239424.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26289.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243024.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22689.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246624.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19089.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250224.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15489.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253824.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11889.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257424.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8289.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261024.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4689.26 sec) 03:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264624.909117 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1089.26 sec) 03:27:00 [GPUGRID] [rr_sim] 2265714.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996413253.69G/439.78G) 03:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851175.08 03:27:00 [---] pick_jobs_to_run() start 03:27:00 [---] pick_jobs_to_run() end 03:27:00 [---] [rr_sim] end 03:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:27:00 [---] [cpu_sched_debug] preliminary job list: 03:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:27:00 [---] [cpu_sched_debug] final job list: 03:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:27:00 [---] [cpu_sched_debug] enforce_run_list: end 03:27:00 [---] choose_project(): 12420.000000 03:27:00 [iThena.Measurements] doing RPC to get work 03:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423179.09 cpu 2166872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:27:00 [---] Got 0 tasks 03:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:28:00 [---] [rr_sim] doing sim: CPU sched 03:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996385663.34G/439.78G) 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265651.43 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 226565.14 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226565.142809 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039086.29 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 203908.63 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430473.771336 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835177.66 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 183517.77 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613991.537011 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651659.89 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 165165.99 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779157.526119 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486493.90 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 148649.39 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927806.916315 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337844.51 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 133784.45 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061591.367492 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204060.06 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 120406.01 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181997.373552 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083654.05 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 108365.41 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290362.779005 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975288.65 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 97528.86 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387891.643913 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877759.78 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 87775.98 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475667.622330 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789983.81 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 78998.38 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554666.002906 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710985.43 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 71098.54 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625764.545424 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639886.88 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 63988.69 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689753.233690 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575898.19 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 57589.82 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747343.053130 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518308.37 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 51830.84 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799173.890625 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466477.54 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 46647.75 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845821.644371 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419829.78 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 41982.98 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887804.622743 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377846.81 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 37784.68 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925589.303277 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340062.12 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 34006.21 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959595.515758 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306055.91 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 30605.59 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990201.106991 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275450.32 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 27545.03 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017746.139100 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247905.29 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 24790.53 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042536.667999 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223114.76 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 22311.48 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064848.144007 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200803.28 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 20080.33 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084928.472415 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180722.96 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 18072.30 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103000.767982 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162650.66 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 16265.07 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119265.833993 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146385.59 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 14638.56 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133904.393402 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131747.03 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 13174.70 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147079.096870 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118572.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 11857.23 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158936.329992 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106715.10 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 10671.51 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169607.839801 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96043.59 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 9604.36 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179212.198630 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86439.23 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 8643.92 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187856.121575 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77795.31 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 7779.53 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195635.652226 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70015.78 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 7001.58 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202637.229812 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63014.20 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 6301.42 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208938.649639 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56712.78 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 5671.28 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214609.927484 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51041.50 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 5104.15 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219714.077544 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45937.35 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 4593.74 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224307.812598 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41343.62 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 4134.36 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228442.174147 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37209.25 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3720.93 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232163.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33488.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235763.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29888.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239363.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26288.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242963.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22688.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246563.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19088.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250163.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15488.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253763.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11888.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257363.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8288.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260963.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4688.33 sec) 03:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264563.099541 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1088.33 sec) 03:28:00 [GPUGRID] [rr_sim] 2265651.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996385663.34G/439.78G) 03:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851172.34 03:28:00 [---] pick_jobs_to_run() start 03:28:00 [---] pick_jobs_to_run() end 03:28:00 [---] [rr_sim] end 03:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:28:00 [---] [cpu_sched_debug] preliminary job list: 03:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:28:00 [---] [cpu_sched_debug] final job list: 03:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:28:00 [---] [cpu_sched_debug] enforce_run_list: end 03:28:00 [---] choose_project(): 12480.000000 03:28:00 [iThena.Measurements] doing RPC to get work 03:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423119.09 cpu 2166812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:28:00 [---] Got 0 tasks 03:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:29:00 [---] [rr_sim] doing sim: CPU sched 03:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996358072.98G/439.78G) 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265588.69 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 226558.87 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226558.869120 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2039029.82 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 203902.98 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430461.851329 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835126.84 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 183512.68 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613974.535316 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651614.16 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 165161.42 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779135.950905 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486452.74 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 148645.27 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927781.224935 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337807.47 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 133780.75 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061561.971562 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1204026.72 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 120402.67 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181964.643526 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083624.05 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 108362.40 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290327.048294 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975261.64 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 97526.16 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387853.212585 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877735.48 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 87773.55 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475626.760446 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789961.93 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 78996.19 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554622.953522 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710965.74 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 71096.57 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625719.527290 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639869.16 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 63986.92 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689706.443682 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575882.25 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 57588.22 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747294.668434 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518294.02 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 51829.40 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799124.070711 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466464.62 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 46646.46 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845770.532760 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419818.16 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 41981.82 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887752.348604 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377836.34 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 37783.63 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925535.982864 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340052.71 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 34005.27 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959541.253698 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306047.44 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 30604.74 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990145.997449 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275442.69 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 27544.27 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017690.266824 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247898.42 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 24789.84 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042480.109262 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223108.58 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 22310.86 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064790.967456 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200797.72 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 20079.77 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084870.739831 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180717.95 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 18071.80 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102942.534968 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162646.16 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 16264.62 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119207.150592 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146381.54 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 14638.15 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133845.304653 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131743.39 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 13174.34 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147019.643308 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118569.05 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 11856.90 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158876.548097 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106712.14 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 10671.21 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169547.762408 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96040.93 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 9604.09 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179151.855288 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86436.84 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 8643.68 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187795.538879 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77793.15 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 7779.32 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195574.854112 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70013.84 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 7001.38 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202576.237821 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63012.45 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 6301.25 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208877.483159 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56711.21 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 5671.12 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214548.603963 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51040.09 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 5104.01 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219652.612687 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45936.08 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 4593.61 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224246.220539 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41342.47 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 4134.25 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228380.467605 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37208.22 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3720.82 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232101.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33487.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235701.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29887.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239301.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26287.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242901.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22687.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246501.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19087.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250101.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15487.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253701.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11887.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257301.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8287.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260901.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4687.40 sec) 03:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264501.289965 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1087.40 sec) 03:29:00 [GPUGRID] [rr_sim] 2265588.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996358072.98G/439.78G) 03:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851169.60 03:29:00 [---] pick_jobs_to_run() start 03:29:00 [---] pick_jobs_to_run() end 03:29:00 [---] [rr_sim] end 03:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:29:00 [---] [cpu_sched_debug] preliminary job list: 03:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:29:00 [---] [cpu_sched_debug] final job list: 03:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:29:00 [---] [cpu_sched_debug] enforce_run_list: end 03:29:00 [---] choose_project(): 12540.000000 03:29:00 [iThena.Measurements] doing RPC to get work 03:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 423059.09 cpu 2166752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:29:00 [---] Got 0 tasks 03:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:30:00 [---] [rr_sim] doing sim: CPU sched 03:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996330482.62G/439.78G) 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265525.95 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 226552.60 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226552.595432 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038973.36 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 203897.34 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430449.931321 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835076.02 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 183507.60 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613957.533621 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651568.42 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 165156.84 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779114.375691 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486411.58 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 148641.16 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927755.533554 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337770.42 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 133777.04 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061532.575631 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203993.38 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 120399.34 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181931.913500 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083594.04 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 108359.40 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290291.317582 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975234.64 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 97523.46 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387814.781256 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877711.17 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 87771.12 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475585.898562 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789940.06 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 78994.01 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554579.904138 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710946.05 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 71094.61 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625674.509157 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639851.45 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 63985.14 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689659.653673 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575866.30 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 57586.63 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747246.283738 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518279.67 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 51827.97 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799074.250796 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466451.70 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 46645.17 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845719.421149 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419806.53 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 41980.65 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887700.074466 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377825.88 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 37782.59 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925482.662452 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340043.29 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 34004.33 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959486.991638 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306038.96 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 30603.90 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990090.887907 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275435.07 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 27543.51 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017634.394548 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247891.56 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 24789.16 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042423.550526 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223102.40 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 22310.24 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064733.790905 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200792.16 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 20079.22 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084813.007247 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180712.95 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 18071.29 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102884.301954 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162641.65 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 16264.17 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119148.467191 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146377.49 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 14637.75 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133786.215904 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131739.74 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 13173.97 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146960.189746 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118565.76 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 11856.58 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158816.766203 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106709.19 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 10670.92 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169487.685015 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96038.27 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 9603.83 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179091.511946 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86434.44 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 8643.44 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187734.956183 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77791.00 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 7779.10 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195514.055997 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70011.90 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 7001.19 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202515.245829 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63010.71 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 6301.07 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208816.316679 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56709.64 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 5670.96 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214487.280443 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51038.67 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 5103.87 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219591.147831 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45934.81 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 4593.48 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224184.628480 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41341.33 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 4134.13 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228318.761064 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37207.19 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3720.72 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232039.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33486.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235639.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29886.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239239.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26286.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242839.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22686.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246439.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19086.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250039.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15486.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253639.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11886.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257239.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8286.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260839.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4686.47 sec) 03:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264439.480390 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1086.47 sec) 03:30:00 [GPUGRID] [rr_sim] 2265525.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996330482.62G/439.78G) 03:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851166.87 03:30:00 [---] pick_jobs_to_run() start 03:30:00 [---] pick_jobs_to_run() end 03:30:00 [---] [rr_sim] end 03:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:30:00 [---] [cpu_sched_debug] preliminary job list: 03:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:30:00 [---] [cpu_sched_debug] final job list: 03:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:30:00 [---] [cpu_sched_debug] enforce_run_list: end 03:30:00 [---] choose_project(): 12600.000000 03:30:00 [iThena.Measurements] doing RPC to get work 03:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422999.09 cpu 2166692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:30:00 [---] Got 0 tasks 03:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:31:00 [---] [rr_sim] doing sim: CPU sched 03:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996302892.27G/439.78G) 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265463.22 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 226546.32 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226546.321744 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038916.90 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 203891.69 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430438.011313 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1835025.21 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 183502.52 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613940.531926 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651522.69 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 165152.27 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779092.800477 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486370.42 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 148637.04 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927729.842173 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337733.38 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 133773.34 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061503.179700 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203960.04 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 120396.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181899.183474 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083564.03 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 108356.40 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290255.586870 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975207.63 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 97520.76 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387776.349927 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877686.87 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 87768.69 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475545.036678 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789918.18 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 78991.82 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554536.854755 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710926.36 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 71092.64 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625629.491023 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639833.73 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 63983.37 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689612.863665 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575850.35 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 57585.04 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747197.899042 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518265.32 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 51826.53 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799024.430882 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466438.79 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 46643.88 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845668.309537 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419794.91 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 41979.49 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887647.800328 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377815.42 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 37781.54 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925429.342039 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340033.88 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 34003.39 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959432.729579 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306030.49 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 30603.05 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990035.778365 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275427.44 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 27542.74 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017578.522272 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247884.70 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 24788.47 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042366.991789 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223096.23 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 22309.62 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064676.614354 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200786.60 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 20078.66 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084755.274662 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180707.94 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 18070.79 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102826.068940 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162637.15 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 16263.71 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119089.783790 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146373.43 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 14637.34 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133727.127155 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131736.09 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 13173.61 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146900.736183 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118562.48 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 11856.25 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158756.984309 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106706.23 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 10670.62 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169427.607622 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96035.61 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 9603.56 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179031.168604 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86432.05 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 8643.20 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187674.373487 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77788.84 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 7778.88 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195453.257882 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70009.96 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 7001.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202454.253838 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63008.96 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 6300.90 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208755.150198 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56708.07 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 5670.81 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214425.956922 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51037.26 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 5103.73 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219529.682974 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45933.53 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 4593.35 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224123.036420 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41340.18 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 4134.02 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228257.054522 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37206.16 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3720.62 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231977.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33485.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235577.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29885.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239177.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26285.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242777.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22685.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246377.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19085.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249977.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15485.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253577.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11885.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257177.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8285.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260777.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4685.55 sec) 03:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264377.670814 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1085.55 sec) 03:31:00 [GPUGRID] [rr_sim] 2265463.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996302892.27G/439.78G) 03:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851164.13 03:31:00 [---] pick_jobs_to_run() start 03:31:00 [---] pick_jobs_to_run() end 03:31:00 [---] [rr_sim] end 03:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:31:00 [---] [cpu_sched_debug] preliminary job list: 03:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:31:00 [---] [cpu_sched_debug] final job list: 03:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:31:00 [---] [cpu_sched_debug] enforce_run_list: end 03:31:00 [---] choose_project(): 12660.000000 03:31:00 [iThena.Measurements] doing RPC to get work 03:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422939.09 cpu 2166632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:31:00 [---] Got 0 tasks 03:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:32:00 [---] [rr_sim] doing sim: CPU sched 03:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996275301.91G/439.78G) 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265400.48 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 226540.05 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226540.048056 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038860.43 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 203886.04 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430426.091306 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834974.39 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 183497.44 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613923.530231 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651476.95 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 165147.70 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779071.225264 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486329.26 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 148632.93 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927704.150793 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337696.33 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 133769.63 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061473.783769 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203926.70 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 120392.67 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181866.453448 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083534.03 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 108353.40 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290219.856159 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975180.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 97518.06 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387737.918599 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877662.56 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 87766.26 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475504.174795 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789896.31 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 78989.63 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554493.805371 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710906.68 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 71090.67 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625584.472889 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639816.01 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 63981.60 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689566.073656 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575834.41 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 57583.44 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747149.514346 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518250.97 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 51825.10 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798974.610967 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466425.87 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 46642.59 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845617.197926 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419783.28 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 41978.33 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887595.526189 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377804.95 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 37780.50 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925376.021626 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340024.46 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 34002.45 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959378.467519 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306022.01 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 30602.20 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989980.668823 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275419.81 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 27541.98 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017522.649996 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247877.83 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 24787.78 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042310.433052 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223090.05 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 22309.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064619.437803 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200781.04 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 20078.10 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084697.542078 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180702.94 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 18070.29 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102767.835926 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162632.64 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 16263.26 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119031.100389 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146369.38 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 14636.94 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133668.038406 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131732.44 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 13173.24 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146841.282621 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118559.20 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 11855.92 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158697.202415 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106703.28 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 10670.33 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169367.530229 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96032.95 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 9603.30 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178970.825262 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86429.66 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 8642.97 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187613.790791 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77786.69 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 7778.67 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195392.459768 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70008.02 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 7000.80 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202393.261847 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63007.22 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 6300.72 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208693.983718 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56706.50 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 5670.65 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214364.633402 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51035.85 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 5103.58 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219468.218117 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45932.26 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 4593.23 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224061.444361 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41339.04 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 4133.90 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228195.347981 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37205.13 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3720.51 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231915.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33484.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235515.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29884.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239115.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26284.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242715.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22684.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246315.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19084.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249915.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15484.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253515.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11884.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257115.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8284.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260715.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4684.62 sec) 03:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264315.861238 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1084.62 sec) 03:32:00 [GPUGRID] [rr_sim] 2265400.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996275301.91G/439.78G) 03:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851161.39 03:32:00 [---] pick_jobs_to_run() start 03:32:00 [---] pick_jobs_to_run() end 03:32:00 [---] [rr_sim] end 03:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:32:00 [---] [cpu_sched_debug] preliminary job list: 03:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:32:00 [---] [cpu_sched_debug] final job list: 03:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:32:00 [---] [cpu_sched_debug] enforce_run_list: end 03:32:00 [---] choose_project(): 12720.000000 03:32:00 [iThena.Measurements] doing RPC to get work 03:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422879.09 cpu 2166572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:32:00 [---] Got 0 tasks 03:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:33:00 [---] [rr_sim] doing sim: CPU sched 03:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996247711.56G/439.78G) 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265337.74 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 226533.77 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226533.774367 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038803.97 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 203880.40 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430414.171298 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834923.57 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 183492.36 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613906.528536 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651431.22 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 165143.12 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779049.650050 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486288.09 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 148628.81 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927678.459412 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337659.28 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 133765.93 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061444.387838 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203893.36 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 120389.34 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181833.723422 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083504.02 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 108350.40 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290184.125447 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975153.62 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 97515.36 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387699.487270 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877638.26 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 87763.83 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475463.312911 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789874.43 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 78987.44 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554450.755987 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710886.99 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 71088.70 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625539.454756 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639798.29 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 63979.83 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689519.283648 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575818.46 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 57581.85 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747101.129650 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518236.61 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 51823.66 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798924.791053 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466412.95 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 46641.30 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845566.086315 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419771.66 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 41977.17 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887543.252051 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377794.49 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 37779.45 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925322.701213 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340015.04 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 34001.50 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959324.205459 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306013.54 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 30601.35 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989925.559281 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275412.18 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 27541.22 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017466.777720 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247870.97 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 24787.10 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042253.874316 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223083.87 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 22308.39 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064562.261252 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200775.48 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 20077.55 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084639.809494 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180697.93 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 18069.79 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102709.602912 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162628.14 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 16262.81 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118972.416988 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146365.33 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 14636.53 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133608.949657 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131728.79 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 13172.88 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146781.829059 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118555.91 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 11855.59 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158637.420520 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106700.32 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 10670.03 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169307.452836 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96030.29 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 9603.03 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178910.481920 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86427.26 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 8642.73 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187553.208095 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77784.54 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 7778.45 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195331.661653 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70006.08 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 7000.61 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202332.269855 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63005.47 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 6300.55 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208632.817237 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56704.93 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 5670.49 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214303.309881 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51034.43 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 5103.44 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219406.753260 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45930.99 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 4593.10 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223999.852302 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41337.89 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 4133.79 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228133.641439 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37204.10 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3720.41 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231854.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33483.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235454.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29883.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239054.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26283.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242654.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22683.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246254.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19083.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249854.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15483.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253454.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11883.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257054.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8283.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260654.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4683.69 sec) 03:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264254.051663 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083.69 sec) 03:33:00 [GPUGRID] [rr_sim] 2265337.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996247711.56G/439.78G) 03:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851158.66 03:33:00 [---] pick_jobs_to_run() start 03:33:00 [---] pick_jobs_to_run() end 03:33:00 [---] [rr_sim] end 03:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:33:00 [---] [cpu_sched_debug] preliminary job list: 03:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:33:00 [---] [cpu_sched_debug] final job list: 03:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:33:00 [---] [cpu_sched_debug] enforce_run_list: end 03:33:00 [---] choose_project(): 12780.000000 03:33:00 [iThena.Measurements] doing RPC to get work 03:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422819.09 cpu 2166512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:33:00 [---] Got 0 tasks 03:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:34:00 [---] [rr_sim] doing sim: CPU sched 03:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996220121.20G/439.78G) 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265275.01 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 226527.50 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226527.500679 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038747.51 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 203874.75 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430402.251291 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834872.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 183487.28 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613889.526841 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651385.48 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 165138.55 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779028.074836 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486246.93 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 148624.69 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927652.768032 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337622.24 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 133762.22 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061414.991908 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203860.01 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 120386.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181800.993396 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083474.01 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 108347.40 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290148.394736 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975126.61 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 97512.66 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387661.055942 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877613.95 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 87761.40 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475422.451027 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789852.56 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 78985.26 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554407.706603 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710867.30 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 71086.73 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625494.436622 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639780.57 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 63978.06 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689472.493639 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575802.51 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 57580.25 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747052.744955 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518222.26 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 51822.23 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798874.971138 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466400.04 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 46640.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845514.974704 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419760.03 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 41976.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887490.977913 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377784.03 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 37778.40 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925269.380801 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340005.63 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 34000.56 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959269.943400 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306005.06 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 30600.51 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989870.449739 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275404.56 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 27540.46 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017410.905444 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247864.10 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 24786.41 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042197.315579 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223077.69 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 22307.77 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064505.084701 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200769.92 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 20076.99 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084582.076910 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180692.93 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 18069.29 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102651.369898 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162623.64 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 16262.36 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118913.733588 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146361.27 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 14636.13 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133549.860908 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131725.15 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 13172.51 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146722.375496 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118552.63 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 11855.26 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158577.638626 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106697.37 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 10669.74 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169247.375443 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96027.63 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 9602.76 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178850.138578 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86424.87 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 8642.49 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187492.625399 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77782.38 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 7778.24 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195270.863539 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70004.14 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 7000.41 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202271.277864 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63003.73 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 6300.37 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208571.650757 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56703.36 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 5670.34 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214241.986360 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51033.02 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 5103.30 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219345.288404 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45929.72 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 4592.97 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223938.260243 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41336.75 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 4133.67 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228071.934898 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37203.07 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3720.31 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231792.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33482.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235392.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29882.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238992.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26282.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242592.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22682.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246192.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19082.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249792.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15482.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253392.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11882.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256992.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8282.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260592.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4682.76 sec) 03:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264192.242087 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082.76 sec) 03:34:00 [GPUGRID] [rr_sim] 2265275.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996220121.20G/439.78G) 03:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851155.92 03:34:00 [---] pick_jobs_to_run() start 03:34:00 [---] pick_jobs_to_run() end 03:34:00 [---] [rr_sim] end 03:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:34:00 [---] [cpu_sched_debug] preliminary job list: 03:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:34:00 [---] [cpu_sched_debug] final job list: 03:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:34:00 [---] [cpu_sched_debug] enforce_run_list: end 03:34:00 [---] choose_project(): 12840.000000 03:34:00 [iThena.Measurements] doing RPC to get work 03:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422759.09 cpu 2166452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:34:00 [---] Got 0 tasks 03:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:35:00 [---] [rr_sim] doing sim: CPU sched 03:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996192530.84G/439.78G) 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265212.27 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 226521.23 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226521.226991 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038691.04 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 203869.10 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430390.331283 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834821.94 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 183482.19 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613872.525146 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651339.74 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 165133.97 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 779006.499622 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486205.77 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 148620.58 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927627.076651 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337585.19 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 133758.52 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061385.595977 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203826.67 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 120382.67 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181768.263370 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083444.01 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 108344.40 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290112.664024 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975099.61 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 97509.96 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387622.624613 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877589.65 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 87758.96 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475381.589143 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789830.68 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 78983.07 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554364.657219 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710847.61 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 71084.76 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625449.418489 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639762.85 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 63976.29 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689425.703631 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575786.57 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 57578.66 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747004.360259 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518207.91 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 51820.79 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798825.151224 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466387.12 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 46638.71 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845463.863093 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419748.41 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 41974.84 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887438.703774 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377773.57 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 37777.36 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925216.060388 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339996.21 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 33999.62 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959215.681340 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305996.59 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 30599.66 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989815.340197 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275396.93 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 27539.69 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017355.033169 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247857.24 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 24785.72 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042140.756843 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223071.51 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 22307.15 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064447.908149 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200764.36 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 20076.44 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084524.344326 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180687.93 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 18068.79 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102593.136884 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162619.13 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 16261.91 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118855.050187 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146357.22 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 14635.72 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133490.772159 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131721.50 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 13172.15 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146662.921934 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118549.35 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 11854.93 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158517.856732 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106694.41 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 10669.44 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169187.298050 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96024.97 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 9602.50 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178789.795236 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86422.47 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 8642.25 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187432.042703 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77780.23 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 7778.02 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195210.065424 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70002.20 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 7000.22 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202210.285873 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63001.98 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 6300.20 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208510.484276 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56701.79 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 5670.18 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214180.662840 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51031.61 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 5103.16 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219283.823547 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45928.45 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 4592.84 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223876.668183 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41335.60 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 4133.56 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228010.228356 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37202.04 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3720.20 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231730.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33481.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235330.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29881.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238930.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26281.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242530.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22681.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246130.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19081.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249730.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15481.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253330.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11881.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256930.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8281.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260530.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4681.84 sec) 03:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264130.432511 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081.84 sec) 03:35:00 [GPUGRID] [rr_sim] 2265212.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996192530.84G/439.78G) 03:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851153.18 03:35:00 [---] pick_jobs_to_run() start 03:35:00 [---] pick_jobs_to_run() end 03:35:00 [---] [rr_sim] end 03:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:35:00 [---] [cpu_sched_debug] preliminary job list: 03:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:35:00 [---] [cpu_sched_debug] final job list: 03:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:35:00 [---] [cpu_sched_debug] enforce_run_list: end 03:35:00 [---] choose_project(): 12900.000000 03:35:00 [iThena.Measurements] doing RPC to get work 03:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422699.09 cpu 2166392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:35:00 [---] Got 0 tasks 03:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:36:00 [---] [rr_sim] doing sim: CPU sched 03:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996164940.49G/439.78G) 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265149.53 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 226514.95 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226514.953303 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038634.58 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 203863.46 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430378.411275 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834771.12 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 183477.11 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613855.523451 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651294.01 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 165129.40 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778984.924408 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486164.61 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 148616.46 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927601.385270 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337548.15 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 133754.81 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061356.200046 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203793.33 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 120379.33 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181735.533344 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083414.00 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 108341.40 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290076.933313 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975072.60 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 97507.26 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387584.193284 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877565.34 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 87756.53 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475340.727259 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789808.81 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 78980.88 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554321.607836 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710827.93 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 71082.79 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625404.400355 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639745.13 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 63974.51 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689378.913622 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575770.62 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 57577.06 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746955.975563 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518193.56 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 51819.36 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798775.331309 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466374.20 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 46637.42 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845412.751481 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419736.78 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 41973.68 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887386.429636 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377763.10 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 37776.31 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925162.739975 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339986.79 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 33998.68 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959161.419281 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305988.11 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 30598.81 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989760.230655 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275389.30 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 27538.93 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017299.160893 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247850.37 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 24785.04 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042084.198106 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223065.33 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 22306.53 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064390.731598 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200758.80 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 20075.88 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084466.611741 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180682.92 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 18068.29 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102534.903870 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162614.63 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 16261.46 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118796.366786 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146353.17 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 14635.32 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133431.683410 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131717.85 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 13171.78 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146603.468372 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118546.06 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 11854.61 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158458.074838 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106691.46 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 10669.15 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169127.220657 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96022.31 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 9602.23 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178729.451894 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86420.08 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 8642.01 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187371.460007 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77778.07 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 7777.81 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195149.267309 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70000.27 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 7000.03 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202149.293881 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63000.24 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 6300.02 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208449.317796 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56700.22 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 5670.02 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214119.339319 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51030.19 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 5103.02 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219222.358690 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45927.17 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 4592.72 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223815.076124 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41334.46 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 4133.45 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227948.521814 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37201.01 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3720.10 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231668.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33480.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235268.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29880.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238868.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26280.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242468.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22680.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246068.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19080.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249668.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15480.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253268.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11880.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256868.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8280.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260468.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4680.91 sec) 03:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264068.622936 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080.91 sec) 03:36:00 [GPUGRID] [rr_sim] 2265149.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996164940.49G/439.78G) 03:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851150.45 03:36:00 [---] pick_jobs_to_run() start 03:36:00 [---] pick_jobs_to_run() end 03:36:00 [---] [rr_sim] end 03:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:36:00 [---] [cpu_sched_debug] preliminary job list: 03:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:36:00 [---] [cpu_sched_debug] final job list: 03:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:36:00 [---] [cpu_sched_debug] enforce_run_list: end 03:36:00 [---] choose_project(): 12960.000000 03:36:00 [iThena.Measurements] doing RPC to get work 03:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422639.09 cpu 2166332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:36:00 [---] Got 0 tasks 03:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:37:00 [---] [rr_sim] doing sim: CPU sched 03:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996137350.13G/439.78G) 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265086.80 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 226508.68 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226508.679615 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038578.12 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 203857.81 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430366.491268 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834720.30 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 183472.03 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613838.521756 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651248.27 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 165124.83 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778963.349195 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486123.45 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 148612.34 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927575.693890 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337511.10 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 133751.11 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061326.804115 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203759.99 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 120376.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181702.803318 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083383.99 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 108338.40 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290041.202601 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975045.59 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 97504.56 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387545.761956 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877541.03 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 87754.10 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475299.865375 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789786.93 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 78978.69 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554278.558452 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710808.24 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 71080.82 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625359.382221 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639727.41 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 63972.74 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689332.123614 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575754.67 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 57575.47 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746907.590867 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518179.21 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 51817.92 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798725.511395 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466361.28 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 46636.13 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845361.639870 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419725.16 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 41972.52 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887334.155498 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377752.64 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 37775.26 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925109.419562 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339977.38 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 33997.74 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959107.157221 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305979.64 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 30597.96 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989705.121113 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275381.68 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 27538.17 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017243.288617 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247843.51 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 24784.35 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042027.639370 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223059.16 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 22305.92 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064333.555047 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200753.24 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 20075.32 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084408.879157 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180677.92 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 18067.79 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102476.670856 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162610.13 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 16261.01 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118737.683385 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146349.11 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 14634.91 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133372.594661 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131714.20 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 13171.42 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146544.014810 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118542.78 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 11854.28 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158398.292943 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106688.50 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 10668.85 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169067.143264 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96019.65 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 9601.97 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178669.108552 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86417.69 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 8641.77 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187310.877311 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77775.92 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 7777.59 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195088.469195 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69998.33 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 6999.83 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202088.301890 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62998.49 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 6299.85 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208388.151315 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56698.64 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 5669.86 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214058.015799 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51028.78 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 5102.88 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219160.893833 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45925.90 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 4592.59 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223753.484065 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41333.31 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 4133.33 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227886.815273 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37199.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3720.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231606.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33479.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235206.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29879.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238806.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26279.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242406.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22679.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246006.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19079.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249606.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15479.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253206.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11879.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256806.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8279.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260406.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4679.98 sec) 03:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2264006.813360 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079.98 sec) 03:37:00 [GPUGRID] [rr_sim] 2265086.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996137350.13G/439.78G) 03:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851147.71 03:37:00 [---] pick_jobs_to_run() start 03:37:00 [---] pick_jobs_to_run() end 03:37:00 [---] [rr_sim] end 03:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:37:00 [---] [cpu_sched_debug] preliminary job list: 03:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:37:00 [---] [cpu_sched_debug] final job list: 03:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:37:00 [---] [cpu_sched_debug] enforce_run_list: end 03:37:00 [---] choose_project(): 13020.000000 03:37:00 [iThena.Measurements] doing RPC to get work 03:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422579.09 cpu 2166272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:37:00 [---] Got 0 tasks 03:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:38:00 [---] [rr_sim] doing sim: CPU sched 03:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996109759.77G/439.78G) 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2265024.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 226502.41 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226502.405926 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038521.65 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 203852.17 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430354.571260 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834669.49 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 183466.95 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613821.520061 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651202.54 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 165120.25 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778941.773981 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486082.29 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 148608.23 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927550.002509 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337474.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 133747.41 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061297.408185 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203726.65 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 120372.67 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181670.073293 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083353.99 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 108335.40 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1290005.471890 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975018.59 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 97501.86 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387507.330627 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877516.73 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 87751.67 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475259.003491 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789765.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 78976.51 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554235.509068 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710788.55 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 71078.86 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625314.364088 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639709.70 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 63970.97 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689285.333605 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575738.73 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 57573.87 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746859.206171 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518164.85 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 51816.49 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798675.691480 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466348.37 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 46634.84 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845310.528259 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419713.53 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 41971.35 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887281.881359 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377742.18 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 37774.22 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925056.099150 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339967.96 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 33996.80 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959052.895161 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305971.16 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 30597.12 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989650.011571 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275374.05 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 27537.40 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017187.416341 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247836.64 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 24783.66 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041971.080633 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223052.98 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 22305.30 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064276.378496 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200747.68 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 20074.77 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084351.146573 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180672.91 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 18067.29 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102418.437842 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162605.62 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 16260.56 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118678.999984 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146345.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 14634.51 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133313.505912 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131710.55 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 13171.06 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146484.561247 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118539.50 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 11853.95 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158338.511049 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106685.55 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 10668.55 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169007.065870 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96016.99 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 9601.70 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178608.765210 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86415.29 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 8641.53 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187250.294615 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77773.76 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 7777.38 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195027.671080 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69996.39 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 6999.64 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202027.309898 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62996.75 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 6299.67 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208326.984835 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56697.07 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 5669.71 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213996.692278 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51027.37 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 5102.74 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219099.428977 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45924.63 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 4592.46 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223691.892005 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41332.17 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 4133.22 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227825.108731 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37198.95 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3719.90 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231545.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33479.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235145.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29879.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238745.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26279.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242345.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22679.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245945.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19079.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249545.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15479.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253145.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11879.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256745.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8279.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260345.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4679.06 sec) 03:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263945.003784 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079.06 sec) 03:38:00 [GPUGRID] [rr_sim] 2265024.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996109759.77G/439.78G) 03:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851144.97 03:38:00 [---] pick_jobs_to_run() start 03:38:00 [---] pick_jobs_to_run() end 03:38:00 [---] [rr_sim] end 03:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:38:00 [---] [cpu_sched_debug] preliminary job list: 03:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:38:00 [---] [cpu_sched_debug] final job list: 03:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:38:00 [---] [cpu_sched_debug] enforce_run_list: end 03:38:00 [---] choose_project(): 13080.000000 03:38:00 [iThena.Measurements] doing RPC to get work 03:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422519.09 cpu 2166212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:38:00 [---] Got 0 tasks 03:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:39:00 [---] [rr_sim] doing sim: CPU sched 03:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996082169.42G/439.78G) 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264961.32 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 226496.13 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226496.132238 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038465.19 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 203846.52 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430342.651253 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834618.67 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 183461.87 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613804.518365 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651156.80 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 165115.68 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778920.198767 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1486041.12 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 148604.11 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927524.311129 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337437.01 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 133743.70 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061268.012254 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203693.31 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 120369.33 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181637.343267 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083323.98 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 108332.40 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289969.741178 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974991.58 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 97499.16 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387468.899299 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877492.42 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 87749.24 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475218.141607 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789743.18 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 78974.32 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554192.459684 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710768.86 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 71076.89 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625269.345954 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639691.98 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 63969.20 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689238.543597 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575722.78 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 57572.28 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746810.821475 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518150.50 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 51815.05 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798625.871566 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466335.45 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 46633.55 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845259.416648 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419701.91 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 41970.19 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887229.607221 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377731.72 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 37773.17 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925002.778737 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339958.54 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 33995.85 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958998.633102 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305962.69 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 30596.27 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989594.902030 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275366.42 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 27536.64 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017131.544065 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247829.78 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 24782.98 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041914.521896 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223046.80 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 22304.68 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064219.201945 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200742.12 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 20074.21 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084293.413989 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180667.91 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 18066.79 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102360.204828 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162601.12 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 16260.11 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118620.316583 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146341.01 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 14634.10 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133254.417163 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131706.91 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 13170.69 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146425.107685 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118536.21 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 11853.62 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158278.729155 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106682.59 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 10668.26 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168946.988477 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96014.33 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 9601.43 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178548.421868 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86412.90 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 8641.29 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187189.711919 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77771.61 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 7777.16 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194966.872966 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69994.45 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 6999.44 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201966.317907 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62995.00 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 6299.50 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208265.818355 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56695.50 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 5669.55 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213935.368757 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51025.95 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 5102.60 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219037.964120 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45923.36 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 4592.34 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223630.299946 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41331.02 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 4133.10 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227763.402190 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37197.92 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3719.79 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231483.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33478.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235083.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29878.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238683.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26278.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242283.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22678.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245883.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19078.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249483.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15478.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253083.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11878.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256683.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8278.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260283.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4678.13 sec) 03:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263883.194209 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078.13 sec) 03:39:00 [GPUGRID] [rr_sim] 2264961.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996082169.42G/439.78G) 03:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851142.24 03:39:00 [---] pick_jobs_to_run() start 03:39:00 [---] pick_jobs_to_run() end 03:39:00 [---] [rr_sim] end 03:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:39:00 [---] [cpu_sched_debug] preliminary job list: 03:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:39:00 [---] [cpu_sched_debug] final job list: 03:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:39:00 [---] [cpu_sched_debug] enforce_run_list: end 03:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:40:00 [---] [rr_sim] doing sim: CPU sched 03:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996054579.06G/439.78G) 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264898.59 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 226489.86 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226489.858550 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038408.73 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 203840.87 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430330.731245 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834567.85 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 183456.79 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613787.516670 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651111.07 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 165111.11 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778898.623553 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485999.96 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 148600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927498.619748 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337399.97 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 133740.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061238.616323 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203659.97 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 120366.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181604.613241 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083293.97 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 108329.40 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289934.010467 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974964.58 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 97496.46 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387430.467970 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877468.12 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 87746.81 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475177.279723 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789721.31 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 78972.13 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554149.410301 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710749.18 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 71074.92 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625224.327820 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639674.26 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 63967.43 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689191.753588 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575706.83 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 57570.68 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746762.436780 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518136.15 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 51813.61 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798576.051652 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466322.53 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 46632.25 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845208.305036 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419690.28 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 41969.03 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887177.333083 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377721.25 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 37772.13 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924949.458324 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339949.13 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 33994.91 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958944.371042 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305954.21 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 30595.42 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989539.792488 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275358.79 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 27535.88 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017075.671789 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247822.91 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 24782.29 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041857.963160 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223040.62 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 22304.06 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064162.025394 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200736.56 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 20073.66 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084235.681404 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180662.90 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 18066.29 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102301.971814 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162596.61 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 16259.66 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118561.633183 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146336.95 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 14633.70 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133195.328414 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131703.26 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 13170.33 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146365.654123 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118532.93 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 11853.29 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158218.947260 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106679.64 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 10667.96 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168886.911084 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96011.67 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 9601.17 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178488.078526 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86410.51 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 8641.05 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187129.129223 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77769.46 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 7776.95 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194906.074851 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69992.51 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 6999.25 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201905.325916 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62993.26 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 6299.33 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208204.651874 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56693.93 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 5669.39 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213874.045237 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51024.54 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 5102.45 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218976.499263 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45922.09 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 4592.21 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223568.707887 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41329.88 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 4132.99 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227701.695648 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37196.89 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3719.69 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231421.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33477.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235021.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29877.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238621.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26277.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242221.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22677.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245821.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19077.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249421.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15477.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253021.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11877.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256621.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8277.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260221.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4677.20 sec) 03:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263821.384633 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077.20 sec) 03:40:00 [GPUGRID] [rr_sim] 2264898.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996054579.06G/439.78G) 03:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851139.50 03:40:00 [---] pick_jobs_to_run() start 03:40:00 [---] pick_jobs_to_run() end 03:40:00 [---] [rr_sim] end 03:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:40:00 [---] [cpu_sched_debug] preliminary job list: 03:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:40:00 [---] [cpu_sched_debug] final job list: 03:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:40:00 [---] [cpu_sched_debug] enforce_run_list: end 03:40:00 [---] choose_project(): 13200.000000 03:40:00 [iThena.Measurements] doing RPC to get work 03:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422399.09 cpu 2166092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:40:00 [---] Got 0 tasks 03:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:41:00 [---] [rr_sim] doing sim: CPU sched 03:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (996026988.71G/439.78G) 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264835.85 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 226483.58 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226483.584862 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038352.26 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 203835.23 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430318.811237 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834517.04 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 183451.70 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613770.514975 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651065.33 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 165106.53 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778877.048340 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485958.80 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 148595.88 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927472.928367 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337362.92 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 133736.29 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061209.220392 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203626.63 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 120362.66 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181571.883215 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083263.97 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 108326.40 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289898.279755 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974937.57 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 97493.76 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387392.036641 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877443.81 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 87744.38 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475136.417839 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789699.43 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 78969.94 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554106.360917 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710729.49 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 71072.95 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625179.309687 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639656.54 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 63965.65 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689144.963580 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575690.89 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 57569.09 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746714.052084 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518121.80 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 51812.18 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798526.231737 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466309.62 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 46630.96 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845157.193425 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419678.66 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 41967.87 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887125.058944 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377710.79 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 37771.08 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924896.137912 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339939.71 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 33993.97 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958890.108982 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305945.74 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 30594.57 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989484.682946 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275351.17 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 27535.12 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017019.799513 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247816.05 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 24781.60 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041801.404423 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223034.44 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 22303.44 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064104.848843 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200731.00 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 20073.10 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084177.948820 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180657.90 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 18065.79 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102243.738800 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162592.11 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 16259.21 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118502.949782 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146332.90 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 14633.29 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133136.239665 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131699.61 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 13169.96 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146306.200560 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118529.65 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 11852.96 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158159.165366 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106676.68 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 10667.67 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168826.833691 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96009.01 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 9600.90 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178427.735184 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86408.11 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 8640.81 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187068.546527 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77767.30 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 7776.73 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194845.276736 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69990.57 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 6999.06 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201844.333924 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62991.51 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 6299.15 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208143.485394 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56692.36 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 5669.24 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213812.721716 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51023.13 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 5102.31 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218915.034406 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45920.81 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 4592.08 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223507.115827 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41328.73 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 4132.87 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227639.989106 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37195.86 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3719.59 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231359.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33476.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234959.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29876.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238559.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26276.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242159.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22676.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245759.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19076.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249359.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15476.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252959.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11876.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256559.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8276.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260159.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4676.27 sec) 03:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263759.575057 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076.27 sec) 03:41:00 [GPUGRID] [rr_sim] 2264835.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (996026988.71G/439.78G) 03:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851136.76 03:41:00 [---] pick_jobs_to_run() start 03:41:00 [---] pick_jobs_to_run() end 03:41:00 [---] [rr_sim] end 03:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:41:00 [---] [cpu_sched_debug] preliminary job list: 03:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:41:00 [---] [cpu_sched_debug] final job list: 03:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:41:00 [---] [cpu_sched_debug] enforce_run_list: end 03:41:00 [---] choose_project(): 13260.000000 03:41:00 [iThena.Measurements] doing RPC to get work 03:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422339.09 cpu 2166032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2166032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2166032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:41:00 [---] Got 0 tasks 03:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:42:00 [---] [rr_sim] doing sim: CPU sched 03:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995999398.35G/439.78G) 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264773.11 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 226477.31 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226477.311174 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038295.80 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 203829.58 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430306.891230 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834466.22 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 183446.62 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613753.513280 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1651019.60 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 165101.96 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778855.473126 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485917.64 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 148591.76 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927447.236987 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337325.87 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 133732.59 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061179.824462 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203593.29 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 120359.33 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181539.153189 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083233.96 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 108323.40 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289862.549044 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974910.56 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 97491.06 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387353.605313 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877419.51 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 87741.95 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475095.555955 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789677.56 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 78967.76 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554063.311533 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710709.80 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 71070.98 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625134.291553 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639638.82 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 63963.88 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689098.173571 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575674.94 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 57567.49 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746665.667388 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518107.44 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 51810.74 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798476.411823 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466296.70 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 46629.67 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845106.081814 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419667.03 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 41966.70 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887072.784806 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377700.33 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 37770.03 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924842.817499 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339930.29 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 33993.03 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958835.846923 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305937.26 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 30593.73 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989429.573404 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275343.54 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 27534.35 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016963.927237 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247809.18 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 24780.92 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041744.845687 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223028.27 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 22302.83 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064047.672292 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200725.44 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 20072.54 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084120.216236 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180652.90 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 18065.29 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102185.505786 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162587.61 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 16258.76 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118444.266381 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146328.85 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 14632.88 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133077.150916 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131695.96 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 13169.60 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146246.746998 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118526.36 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 11852.64 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158099.383472 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106673.73 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 10667.37 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168766.756298 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96006.36 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 9600.64 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178367.391842 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86405.72 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 8640.57 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187007.963831 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77765.15 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 7776.51 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194784.478622 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69988.63 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 6998.86 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201783.341933 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62989.77 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 6298.98 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208082.318913 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56690.79 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 5669.08 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213751.398195 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51021.71 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 5102.17 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218853.569549 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45919.54 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 4591.95 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223445.523768 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41327.59 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 4132.76 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227578.282565 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37194.83 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3719.48 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231297.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33475.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234897.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29875.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238497.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26275.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242097.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22675.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245697.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19075.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249297.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15475.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252897.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11875.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256497.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8275.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260097.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4675.35 sec) 03:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263697.765482 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075.35 sec) 03:42:00 [GPUGRID] [rr_sim] 2264773.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995999398.35G/439.78G) 03:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851134.02 03:42:00 [---] pick_jobs_to_run() start 03:42:00 [---] pick_jobs_to_run() end 03:42:00 [---] [rr_sim] end 03:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:42:00 [---] [cpu_sched_debug] preliminary job list: 03:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:42:00 [---] [cpu_sched_debug] final job list: 03:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:42:00 [---] [cpu_sched_debug] enforce_run_list: end 03:42:00 [---] choose_project(): 13320.000000 03:42:00 [iThena.Measurements] doing RPC to get work 03:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422279.09 cpu 2165972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:42:00 [---] Got 0 tasks 03:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:43:00 [---] [rr_sim] doing sim: CPU sched 03:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995971807.99G/439.78G) 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264710.37 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 226471.04 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226471.037485 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038239.34 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 203823.93 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430294.971222 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834415.40 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 183441.54 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613736.511585 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650973.86 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 165097.39 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778833.897912 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485876.48 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 148587.65 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927421.545606 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337288.83 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 133728.88 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061150.428531 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203559.95 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 120355.99 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181506.423163 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083203.95 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 108320.40 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289826.818332 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974883.56 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 97488.36 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387315.173984 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877395.20 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 87739.52 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475054.694071 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789655.68 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 78965.57 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1554020.262149 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710690.11 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 71069.01 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625089.273420 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639621.10 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 63962.11 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689051.383563 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575658.99 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 57565.90 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746617.282692 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518093.09 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 51809.31 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798426.591908 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466283.78 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 46628.38 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845054.970203 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419655.40 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 41965.54 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887020.510668 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377689.86 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 37768.99 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924789.497086 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339920.88 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 33992.09 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958781.584863 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305928.79 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 30592.88 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989374.463862 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275335.91 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 27533.59 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016908.054961 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247802.32 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 24780.23 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041688.286950 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223022.09 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 22302.21 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063990.495741 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200719.88 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 20071.99 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084062.483652 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180647.89 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 18064.79 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102127.272772 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162583.10 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 16258.31 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118385.582980 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146324.79 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 14632.48 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133018.062167 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131692.31 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 13169.23 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146187.293436 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118523.08 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 11852.31 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158039.601578 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106670.77 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 10667.08 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168706.678905 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96003.70 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 9600.37 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178307.048500 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86403.33 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 8640.33 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186947.381135 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77762.99 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 7776.30 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194723.680507 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69986.69 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 6998.67 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201722.349942 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62988.02 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 6298.80 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208021.152433 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56689.22 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 5668.92 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213690.074675 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51020.30 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 5102.03 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218792.104693 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45918.27 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 4591.83 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223383.931709 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41326.44 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 4132.64 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227516.576023 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37193.80 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3719.38 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231235.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33474.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234835.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29874.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238435.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26274.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242035.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22674.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245635.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19074.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249235.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15474.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252835.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11874.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256435.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8274.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260035.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4674.42 sec) 03:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263635.955906 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074.42 sec) 03:43:00 [GPUGRID] [rr_sim] 2264710.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995971807.99G/439.78G) 03:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851131.29 03:43:00 [---] pick_jobs_to_run() start 03:43:00 [---] pick_jobs_to_run() end 03:43:00 [---] [rr_sim] end 03:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:43:00 [---] [cpu_sched_debug] preliminary job list: 03:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:43:00 [---] [cpu_sched_debug] final job list: 03:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:43:00 [---] [cpu_sched_debug] enforce_run_list: end 03:43:00 [---] choose_project(): 13380.000000 03:43:00 [iThena.Measurements] doing RPC to get work 03:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 595019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422219.09 cpu 2165912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:43:00 [---] Got 0 tasks 03:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:44:00 [---] [rr_sim] doing sim: CPU sched 03:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995944217.64G/439.78G) 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264647.64 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 226464.76 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226464.763797 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038182.87 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 203818.29 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430283.051214 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834364.59 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 183436.46 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613719.509890 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650928.13 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 165092.81 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778812.322698 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485835.32 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 148583.53 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927395.854225 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337251.78 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 133725.18 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061121.032600 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203526.61 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 120352.66 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181473.693137 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083173.94 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 108317.39 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289791.087620 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974856.55 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 97485.66 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387276.742656 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877370.90 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 87737.09 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1475013.832187 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789633.81 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 78963.38 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553977.212765 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710670.43 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 71067.04 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1625044.255286 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639603.38 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 63960.34 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689004.593555 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575643.04 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 57564.30 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746568.897996 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518078.74 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 51807.87 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798376.771994 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466270.87 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 46627.09 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845003.858591 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419643.78 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 41964.38 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886968.236529 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377679.40 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 37767.94 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924736.176673 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339911.46 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 33991.15 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958727.322803 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305920.32 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 30592.03 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989319.354320 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275328.28 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 27532.83 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016852.182685 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247795.46 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 24779.55 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041631.728214 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223015.91 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 22301.59 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063933.319189 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200714.32 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 20071.43 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084004.751068 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180642.89 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 18064.29 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102069.039758 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162578.60 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 16257.86 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118326.899579 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146320.74 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 14632.07 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132958.973418 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131688.66 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 13168.87 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146127.839874 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118519.80 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 11851.98 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157979.819683 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106667.82 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 10666.78 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168646.601512 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96001.04 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 9600.10 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178246.705158 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86400.93 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 8640.09 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186886.798439 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77760.84 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 7776.08 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194662.882392 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69984.76 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 6998.48 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201661.357950 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62986.28 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 6298.63 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207959.985952 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56687.65 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 5668.77 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213628.751154 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51018.89 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 5101.89 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218730.639836 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45917.00 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 4591.70 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223322.339649 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41325.30 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 4132.53 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227454.869482 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37192.77 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3719.28 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231174.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33473.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234774.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29873.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238374.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26273.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241974.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22673.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245574.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19073.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249174.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15473.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252774.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11873.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256374.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8273.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259974.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4673.49 sec) 03:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263574.146330 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073.49 sec) 03:44:00 [GPUGRID] [rr_sim] 2264647.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995944217.64G/439.78G) 03:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851128.55 03:44:00 [---] pick_jobs_to_run() start 03:44:00 [---] pick_jobs_to_run() end 03:44:00 [---] [rr_sim] end 03:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:44:00 [---] [cpu_sched_debug] preliminary job list: 03:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:44:00 [---] [cpu_sched_debug] final job list: 03:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:44:00 [---] [cpu_sched_debug] enforce_run_list: end 03:44:00 [---] choose_project(): 13440.000000 03:44:00 [iThena.Measurements] doing RPC to get work 03:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422159.09 cpu 2165852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:44:00 [---] Got 0 tasks 03:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:45:00 [---] [rr_sim] doing sim: CPU sched 03:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995916627.28G/439.78G) 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264584.90 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 226458.49 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226458.490109 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038126.41 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 203812.64 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430271.131207 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834313.77 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 183431.38 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613702.508195 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650882.39 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 165088.24 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778790.747484 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485794.15 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 148579.42 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927370.162845 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337214.74 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 133721.47 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061091.636669 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203493.26 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 120349.33 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181440.963111 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083143.94 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 108314.39 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289755.356909 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974829.54 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 97482.95 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387238.311327 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877346.59 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 87734.66 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474972.970303 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789611.93 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 78961.19 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553934.163382 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710650.74 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 71065.07 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624999.237152 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639585.66 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 63958.57 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688957.803546 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575627.10 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 57562.71 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746520.513300 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518064.39 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 51806.44 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798326.952079 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466257.95 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 46625.79 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844952.746980 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419632.15 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 41963.22 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886915.962391 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377668.94 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 37766.89 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924682.856261 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339902.04 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 33990.20 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958673.060744 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305911.84 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 30591.18 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989264.244778 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275320.66 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 27532.07 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016796.310409 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247788.59 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 24778.86 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041575.169477 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223009.73 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 22300.97 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063876.142638 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200708.76 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 20070.88 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083947.018483 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180637.88 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 18063.79 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102010.806744 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162574.09 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 16257.41 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118268.216178 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146316.68 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 14631.67 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132899.884669 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131685.02 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 13168.50 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146068.386311 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118516.51 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 11851.65 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157920.037789 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106664.86 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 10666.49 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168586.524119 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95998.38 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 9599.84 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178186.361816 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86398.54 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 8639.85 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186826.215743 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77758.69 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 7775.87 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194602.084278 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69982.82 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 6998.28 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201600.365959 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62984.54 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 6298.45 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207898.819472 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56686.08 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 5668.61 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213567.427634 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51017.47 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 5101.75 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218669.174979 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45915.73 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 4591.57 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223260.747590 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41324.15 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 4132.42 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227393.162940 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37191.74 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3719.17 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231112.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33472.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234712.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29872.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238312.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26272.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241912.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22672.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245512.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19072.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249112.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15472.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252712.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11872.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256312.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8272.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259912.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4672.56 sec) 03:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263512.336755 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072.56 sec) 03:45:00 [GPUGRID] [rr_sim] 2264584.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995916627.28G/439.78G) 03:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851125.81 03:45:00 [---] pick_jobs_to_run() start 03:45:00 [---] pick_jobs_to_run() end 03:45:00 [---] [rr_sim] end 03:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:45:00 [---] [cpu_sched_debug] preliminary job list: 03:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:45:00 [---] [cpu_sched_debug] final job list: 03:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:45:00 [---] [cpu_sched_debug] enforce_run_list: end 03:45:00 [---] choose_project(): 13500.000000 03:45:00 [iThena.Measurements] doing RPC to get work 03:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422099.09 cpu 2165792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:45:00 [---] Got 0 tasks 03:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:46:00 [---] [rr_sim] doing sim: CPU sched 03:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995889036.92G/439.78G) 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264522.16 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 226452.22 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226452.216421 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038069.95 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 203806.99 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430259.211199 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834262.95 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 183426.30 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613685.506500 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650836.66 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 165083.67 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778769.172271 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485752.99 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 148575.30 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927344.471464 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337177.69 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 133717.77 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061062.240738 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203459.92 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 120345.99 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181408.233085 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083113.93 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 108311.39 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289719.626197 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974802.54 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 97480.25 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387199.879998 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877322.28 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 87732.23 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474932.108419 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789590.06 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 78959.01 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553891.113998 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710631.05 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 71063.11 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624954.219019 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639567.95 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 63956.79 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688911.013538 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575611.15 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 57561.12 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746472.128604 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518050.04 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 51805.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798277.132165 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466245.03 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 46624.50 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844901.635369 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419620.53 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 41962.05 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886863.688253 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377658.48 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 37765.85 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924629.535848 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339892.63 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 33989.26 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958618.798684 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305903.37 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 30590.34 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989209.135236 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275313.03 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 27531.30 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016740.438133 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247781.73 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 24778.17 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041518.610741 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223003.55 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 22300.36 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063818.966087 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200703.20 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 20070.32 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083889.285899 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180632.88 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 18063.29 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101952.573730 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162569.59 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 16256.96 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118209.532778 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146312.63 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 14631.26 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132840.795920 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131681.37 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 13168.14 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146008.932749 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118513.23 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 11851.32 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157860.255895 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106661.91 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 10666.19 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168526.446726 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95995.72 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 9599.57 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178126.018474 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86396.15 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 8639.61 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186765.633047 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77756.53 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 7775.65 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194541.286163 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69980.88 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 6998.09 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201539.373968 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62982.79 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 6298.28 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207837.652991 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56684.51 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 5668.45 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213506.104113 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51016.06 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 5101.61 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218607.710122 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45914.45 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 4591.45 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223199.155531 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41323.01 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 4132.30 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227331.456398 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37190.71 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3719.07 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231050.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33471.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234650.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29871.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238250.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26271.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241850.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22671.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245450.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19071.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249050.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15471.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252650.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11871.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256250.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8271.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259850.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4671.64 sec) 03:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263450.527179 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071.64 sec) 03:46:00 [GPUGRID] [rr_sim] 2264522.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995889036.92G/439.78G) 03:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851123.08 03:46:00 [---] pick_jobs_to_run() start 03:46:00 [---] pick_jobs_to_run() end 03:46:00 [---] [rr_sim] end 03:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:46:00 [---] [cpu_sched_debug] preliminary job list: 03:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:46:00 [---] [cpu_sched_debug] final job list: 03:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:46:00 [---] [cpu_sched_debug] enforce_run_list: end 03:46:00 [---] choose_project(): 13560.000000 03:46:00 [iThena.Measurements] doing RPC to get work 03:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 422039.09 cpu 2165732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:46:00 [---] Got 0 tasks 03:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:47:00 [---] [rr_sim] doing sim: CPU sched 03:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995861446.57G/439.78G) 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264459.43 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 226445.94 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226445.942732 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2038013.48 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 203801.35 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430247.291192 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834212.14 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 183421.21 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613668.504805 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650790.92 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 165079.09 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778747.597057 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485711.83 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 148571.18 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927318.780084 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337140.65 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 133714.06 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061032.844808 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203426.58 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 120342.66 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181375.503059 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083083.92 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 108308.39 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289683.895486 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974775.53 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 97477.55 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387161.448670 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877297.98 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 87729.80 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474891.246535 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789568.18 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 78956.82 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553848.064614 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710611.36 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 71061.14 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624909.200885 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639550.23 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 63955.02 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688864.223529 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575595.20 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 57559.52 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746423.743909 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518035.68 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 51803.57 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798227.312250 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466232.12 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 46623.21 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844850.523758 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419608.90 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 41960.89 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886811.414114 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377648.01 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 37764.80 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924576.215435 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339883.21 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 33988.32 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958564.536624 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305894.89 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 30589.49 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989154.025694 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275305.40 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 27530.54 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016684.565857 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247774.86 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 24777.49 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041462.052004 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222997.38 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 22299.74 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063761.789536 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200697.64 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 20069.76 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083831.553315 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180627.87 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 18062.79 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101894.340716 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162565.09 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 16256.51 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118150.849377 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146308.58 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 14630.86 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132781.707171 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131677.72 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 13167.77 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145949.479187 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118509.95 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 11850.99 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157800.474001 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106658.95 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 10665.90 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168466.369333 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95993.06 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 9599.31 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178065.675132 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86393.75 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 8639.38 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186705.050351 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77754.38 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 7775.44 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194480.488049 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69978.94 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 6997.89 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201478.381976 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62981.05 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 6298.10 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207776.486511 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56682.94 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 5668.29 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213444.780592 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51014.65 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 5101.46 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218546.245266 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45913.18 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 4591.32 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223137.563471 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41321.86 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 4132.19 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227269.749857 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37189.68 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3718.97 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230988.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33470.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234588.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29870.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238188.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26270.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241788.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22670.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245388.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19070.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248988.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15470.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252588.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11870.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256188.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8270.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259788.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4670.71 sec) 03:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263388.717604 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070.71 sec) 03:47:00 [GPUGRID] [rr_sim] 2264459.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995861446.57G/439.78G) 03:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851120.34 03:47:00 [---] pick_jobs_to_run() start 03:47:00 [---] pick_jobs_to_run() end 03:47:00 [---] [rr_sim] end 03:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:47:00 [---] [cpu_sched_debug] preliminary job list: 03:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:47:00 [---] [cpu_sched_debug] final job list: 03:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:47:00 [---] [cpu_sched_debug] enforce_run_list: end 03:47:00 [---] choose_project(): 13620.000000 03:47:00 [iThena.Measurements] doing RPC to get work 03:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421979.09 cpu 2165672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:47:00 [---] Got 0 tasks 03:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:48:00 [---] [rr_sim] doing sim: CPU sched 03:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995833856.21G/439.78G) 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264396.69 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 226439.67 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226439.669044 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037957.02 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 203795.70 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430235.371184 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834161.32 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 183416.13 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613651.503110 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650745.19 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 165074.52 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778726.021843 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485670.67 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 148567.07 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927293.088703 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337103.60 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 133710.36 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1061003.448877 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203393.24 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 120339.32 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181342.773033 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083053.92 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 108305.39 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289648.164774 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974748.53 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 97474.85 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387123.017341 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877273.67 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 87727.37 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474850.384651 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789546.31 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 78954.63 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553805.015230 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710591.68 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 71059.17 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624864.182752 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639532.51 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 63953.25 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688817.433521 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575579.26 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 57557.93 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746375.359213 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518021.33 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 51802.13 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798177.492336 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466219.20 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 46621.92 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844799.412146 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419597.28 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 41959.73 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886759.139976 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377637.55 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 37763.76 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924522.895023 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339873.80 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 33987.38 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958510.274565 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305886.42 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 30588.64 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989098.916152 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275297.77 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 27529.78 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016628.693581 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247768.00 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 24776.80 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041405.493267 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222991.20 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 22299.12 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063704.612985 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200692.08 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 20069.21 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083773.820731 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180622.87 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 18062.29 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101836.107702 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162560.58 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 16256.06 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118092.165976 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146304.52 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 14630.45 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132722.618423 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131674.07 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 13167.41 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145890.025624 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118506.66 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 11850.67 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157740.692106 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106656.00 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 10665.60 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168406.291940 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95990.40 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 9599.04 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178005.331790 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86391.36 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 8639.14 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186644.467655 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77752.22 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 7775.22 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194419.689934 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69977.00 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 6997.70 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201417.389985 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62979.30 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 6297.93 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207715.320031 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56681.37 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 5668.14 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213383.457072 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51013.23 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 5101.32 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218484.780409 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45911.91 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 4591.19 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223075.971412 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41320.72 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 4132.07 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227208.043315 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37188.65 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3718.86 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230926.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33469.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234526.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29869.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238126.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26269.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241726.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22669.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245326.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19069.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248926.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15469.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252526.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11869.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256126.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8269.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259726.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4669.78 sec) 03:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263326.908028 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069.78 sec) 03:48:00 [GPUGRID] [rr_sim] 2264396.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995833856.21G/439.78G) 03:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851117.60 03:48:00 [---] pick_jobs_to_run() start 03:48:00 [---] pick_jobs_to_run() end 03:48:00 [---] [rr_sim] end 03:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:48:00 [---] [cpu_sched_debug] preliminary job list: 03:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:48:00 [---] [cpu_sched_debug] final job list: 03:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:48:00 [---] [cpu_sched_debug] enforce_run_list: end 03:48:00 [---] choose_project(): 13680.000000 03:48:00 [iThena.Measurements] doing RPC to get work 03:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421919.09 cpu 2165612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:48:00 [---] Got 0 tasks 03:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:49:00 [---] [rr_sim] doing sim: CPU sched 03:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995806265.86G/439.78G) 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264333.95 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 226433.40 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226433.395356 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037900.56 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 203790.06 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430223.451176 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834110.50 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 183411.05 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613634.501415 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650699.45 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 165069.95 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778704.446629 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485629.51 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 148562.95 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927267.397322 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337066.56 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 133706.66 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060974.052946 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203359.90 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 120335.99 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181310.043008 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1083023.91 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 108302.39 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289612.434063 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974721.52 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 97472.15 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387084.586013 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877249.37 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 87724.94 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474809.522767 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789524.43 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 78952.44 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553761.965847 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710571.99 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 71057.20 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624819.164618 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639514.79 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 63951.48 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688770.643512 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575563.31 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 57556.33 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746326.974517 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518006.98 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 51800.70 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798127.672421 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466206.28 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 46620.63 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844748.300535 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419585.65 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 41958.57 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886706.865838 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377627.09 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 37762.71 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924469.574610 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339864.38 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 33986.44 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958456.012505 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305877.94 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 30587.79 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989043.806610 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275290.15 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 27529.01 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016572.821305 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247761.13 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 24776.11 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041348.934531 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222985.02 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 22298.50 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063647.436434 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200686.52 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 20068.65 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083716.088146 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180617.87 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 18061.79 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101777.874688 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162556.08 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 16255.61 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118033.482575 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146300.47 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 14630.05 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132663.529674 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131670.42 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 13167.04 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145830.572062 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118503.38 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 11850.34 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157680.910212 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106653.04 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 10665.30 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168346.214547 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95987.74 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 9598.77 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177944.988448 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86388.97 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 8638.90 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186583.884959 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77750.07 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 7775.01 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194358.891819 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69975.06 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 6997.51 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201356.397993 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62977.56 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 6297.76 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207654.153550 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56679.80 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 5667.98 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213322.133551 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51011.82 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 5101.18 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218423.315552 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45910.64 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 4591.06 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223014.379353 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41319.57 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 4131.96 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227146.336774 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37187.62 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3718.76 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230865.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33468.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234465.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29868.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238065.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26268.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241665.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22668.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245265.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19068.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248865.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15468.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252465.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11868.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256065.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8268.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259665.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4668.86 sec) 03:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263265.098452 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068.86 sec) 03:49:00 [GPUGRID] [rr_sim] 2264333.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995806265.86G/439.78G) 03:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851114.87 03:49:00 [---] pick_jobs_to_run() start 03:49:00 [---] pick_jobs_to_run() end 03:49:00 [---] [rr_sim] end 03:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:49:00 [---] [cpu_sched_debug] preliminary job list: 03:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:49:00 [---] [cpu_sched_debug] final job list: 03:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:49:00 [---] [cpu_sched_debug] enforce_run_list: end 03:49:00 [---] choose_project(): 13740.000000 03:49:00 [iThena.Measurements] doing RPC to get work 03:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421859.09 cpu 2165552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:49:00 [---] Got 0 tasks 03:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:50:00 [---] [rr_sim] doing sim: CPU sched 03:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995778675.50G/439.78G) 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264271.22 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 226427.12 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226427.121668 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037844.10 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 203784.41 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430211.531169 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834059.69 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 183405.97 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613617.499720 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650653.72 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 165065.37 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778682.871416 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485588.35 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 148558.83 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927241.705942 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1337029.51 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 133702.95 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060944.657015 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203326.56 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 120332.66 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181277.312982 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082993.90 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 108299.39 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289576.703351 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974694.51 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 97469.45 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387046.154684 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877225.06 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 87722.51 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474768.660883 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789502.56 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 78950.26 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553718.916463 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710552.30 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 71055.23 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624774.146484 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639497.07 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 63949.71 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688723.853504 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575547.36 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 57554.74 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746278.589821 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517992.63 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 51799.26 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798077.852507 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466193.36 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 46619.34 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844697.188924 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419574.03 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 41957.40 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886654.591699 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377616.62 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 37761.66 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924416.254197 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339854.96 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 33985.50 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958401.750445 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305869.47 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 30586.95 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988988.697069 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275282.52 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 27528.25 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016516.949029 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247754.27 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 24775.43 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041292.375794 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222978.84 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 22297.88 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063590.259883 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200680.96 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 20068.10 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083658.355562 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180612.86 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 18061.29 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101719.641674 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162551.58 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 16255.16 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117974.799174 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146296.42 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 14629.64 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132604.440925 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131666.78 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 13166.68 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145771.118500 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118500.10 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 11850.01 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157621.128318 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106650.09 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 10665.01 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168286.137154 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95985.08 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 9598.51 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177884.645106 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86386.57 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 8638.66 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186523.302263 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77747.91 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 7774.79 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194298.093705 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69973.12 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 6997.31 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201295.406002 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62975.81 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 6297.58 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207592.987070 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56678.23 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 5667.82 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213260.810031 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51010.41 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 5101.04 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218361.850695 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45909.37 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 4590.94 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222952.787294 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41318.43 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 4131.84 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227084.630232 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37186.59 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3718.66 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230803.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33467.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234403.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29867.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238003.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26267.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241603.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22667.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245203.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19067.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248803.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15467.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252403.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11867.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256003.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8267.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259603.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4667.93 sec) 03:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263203.288877 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067.93 sec) 03:50:00 [GPUGRID] [rr_sim] 2264271.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995778675.50G/439.78G) 03:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851112.13 03:50:00 [---] pick_jobs_to_run() start 03:50:00 [---] pick_jobs_to_run() end 03:50:00 [---] [rr_sim] end 03:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:50:00 [---] [cpu_sched_debug] preliminary job list: 03:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:50:00 [---] [cpu_sched_debug] final job list: 03:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:50:00 [---] [cpu_sched_debug] enforce_run_list: end 03:50:00 [---] choose_project(): 13800.000000 03:50:00 [iThena.Measurements] doing RPC to get work 03:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421799.09 cpu 2165492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:50:00 [---] Got 0 tasks 03:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:51:00 [---] [rr_sim] doing sim: CPU sched 03:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995751085.14G/439.78G) 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264208.48 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 226420.85 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226420.847980 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037787.63 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 203778.76 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430199.611161 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1834008.87 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 183400.89 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613600.498025 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650607.98 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 165060.80 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778661.296202 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485547.18 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 148554.72 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927216.014561 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336992.47 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 133699.25 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060915.261085 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203293.22 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 120329.32 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181244.582956 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082963.90 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 108296.39 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289540.972640 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974667.51 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 97466.75 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1387007.723355 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877200.76 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 87720.08 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474727.798999 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789480.68 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 78948.07 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553675.867079 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710532.61 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 71053.26 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624729.128351 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639479.35 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 63947.94 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688677.063495 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575531.42 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 57553.14 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746230.205125 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517978.27 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 51797.83 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798028.032592 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466180.45 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 46618.04 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844646.077313 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419562.40 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 41956.24 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886602.317561 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377606.16 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 37760.62 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924362.933784 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339845.55 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 33984.55 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958347.488386 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305860.99 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 30586.10 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988933.587527 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275274.89 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 27527.49 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016461.076754 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247747.40 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 24774.74 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041235.817058 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222972.66 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 22297.27 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063533.083332 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200675.40 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 20067.54 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083600.622978 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180607.86 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 18060.79 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101661.408660 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162547.07 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 16254.71 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117916.115773 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146292.36 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 14629.24 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132545.352176 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131663.13 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 13166.31 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145711.664938 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118496.81 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 11849.68 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157561.346423 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106647.13 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 10664.71 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168226.059761 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95982.42 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 9598.24 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177824.301764 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86384.18 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 8638.42 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186462.719567 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77745.76 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 7774.58 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194237.295590 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69971.18 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 6997.12 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201234.414011 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62974.07 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 6297.41 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207531.820589 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56676.66 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 5667.67 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213199.486510 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51008.99 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 5100.90 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218300.385838 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45908.09 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 4590.81 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222891.195234 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41317.28 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 4131.73 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227022.923690 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37185.56 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3718.56 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230741.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33467.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234341.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29867.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237941.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26267.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241541.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22667.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245141.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19067.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248741.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15467.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252341.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11867.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255941.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8267.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259541.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4667.00 sec) 03:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263141.479301 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067.00 sec) 03:51:00 [GPUGRID] [rr_sim] 2264208.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995751085.14G/439.78G) 03:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851109.39 03:51:00 [---] pick_jobs_to_run() start 03:51:00 [---] pick_jobs_to_run() end 03:51:00 [---] [rr_sim] end 03:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:51:00 [---] [cpu_sched_debug] preliminary job list: 03:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:51:00 [---] [cpu_sched_debug] final job list: 03:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:51:00 [---] [cpu_sched_debug] enforce_run_list: end 03:51:00 [---] choose_project(): 13860.000000 03:51:00 [iThena.Measurements] doing RPC to get work 03:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421739.09 cpu 2165432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:51:00 [---] Got 0 tasks 03:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:52:00 [---] [rr_sim] doing sim: CPU sched 03:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995723494.79G/439.78G) 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264145.74 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 226414.57 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226414.574291 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037731.17 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 203773.12 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430187.691154 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833958.05 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 183395.81 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613583.496330 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650562.25 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 165056.22 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778639.720988 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485506.02 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 148550.60 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927190.323181 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336955.42 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 133695.54 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060885.865154 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203259.88 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 120325.99 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181211.852930 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082933.89 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 108293.39 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289505.241928 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974640.50 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 97464.05 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386969.292027 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877176.45 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 87717.65 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474686.937115 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789458.81 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 78945.88 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553632.817695 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710512.93 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 71051.29 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624684.110217 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639461.63 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 63946.16 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688630.273487 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575515.47 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 57551.55 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746181.820429 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517963.92 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 51796.39 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797978.212678 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466167.53 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 46616.75 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844594.965701 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419550.78 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 41955.08 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886550.043423 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377595.70 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 37759.57 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924309.613372 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339836.13 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 33983.61 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958293.226326 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305852.52 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 30585.25 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988878.477985 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275267.26 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 27526.73 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016405.204478 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247740.54 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 24774.05 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041179.258321 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222966.48 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 22296.65 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063475.906780 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200669.84 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 20066.98 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083542.890394 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180602.85 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 18060.29 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101603.175646 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162542.57 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 16254.26 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117857.432373 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146288.31 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 14628.83 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132486.263427 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131659.48 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 13165.95 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145652.211375 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118493.53 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 11849.35 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157501.564529 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106644.18 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 10664.42 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168165.982368 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95979.76 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 9597.98 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177763.958422 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86381.78 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 8638.18 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186402.136871 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77743.61 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 7774.36 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194176.497476 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69969.25 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 6996.92 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201173.422019 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62972.32 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 6297.23 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207470.654109 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56675.09 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 5667.51 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213138.162989 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51007.58 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 5100.76 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218238.920982 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45906.82 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 4590.68 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222829.603175 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41316.14 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 4131.61 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226961.217149 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37184.53 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3718.45 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230679.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33466.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234279.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29866.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237879.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26266.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241479.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22666.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245079.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19066.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248679.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15466.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252279.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11866.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255879.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8266.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259479.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4666.07 sec) 03:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263079.669725 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066.07 sec) 03:52:00 [GPUGRID] [rr_sim] 2264145.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995723494.79G/439.78G) 03:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851106.66 03:52:00 [---] pick_jobs_to_run() start 03:52:00 [---] pick_jobs_to_run() end 03:52:00 [---] [rr_sim] end 03:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:52:00 [---] [cpu_sched_debug] preliminary job list: 03:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:52:00 [---] [cpu_sched_debug] final job list: 03:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:52:00 [---] [cpu_sched_debug] enforce_run_list: end 03:52:00 [---] choose_project(): 13920.000000 03:52:00 [iThena.Measurements] doing RPC to get work 03:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421679.09 cpu 2165372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:52:00 [---] Got 0 tasks 03:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:53:00 [---] [rr_sim] doing sim: CPU sched 03:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995695904.43G/439.78G) 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264083.01 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 226408.30 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226408.300603 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037674.71 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 203767.47 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430175.771146 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833907.23 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 183390.72 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613566.494635 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650516.51 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 165051.65 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778618.145774 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485464.86 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 148546.49 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927164.631800 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336918.37 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 133691.84 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060856.469223 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203226.54 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 120322.65 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181179.122904 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082903.88 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 108290.39 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289469.511217 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974613.49 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 97461.35 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386930.860698 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877152.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 87715.21 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474646.075231 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789436.93 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 78943.69 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553589.768311 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710493.24 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 71049.32 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624639.092083 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639443.91 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 63944.39 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688583.483478 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575499.52 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 57549.95 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746133.435734 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517949.57 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 51794.96 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797928.392763 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466154.61 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 46615.46 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844543.854090 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419539.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 41953.92 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886497.769284 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377585.24 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 37758.52 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924256.292959 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339826.71 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 33982.67 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958238.964266 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305844.04 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 30584.40 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988823.368443 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275259.64 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 27525.96 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016349.332202 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247733.67 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 24773.37 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041122.699585 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222960.31 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 22296.03 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063418.730229 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200664.28 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 20066.43 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083485.157810 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180597.85 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 18059.78 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101544.942632 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162538.06 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 16253.81 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117798.748972 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146284.26 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 14628.43 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132427.174678 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131655.83 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 13165.58 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145592.757813 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118490.25 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 11849.02 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157441.782635 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106641.22 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 10664.12 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168105.904975 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95977.10 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 9597.71 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177703.615080 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86379.39 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 8637.94 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186341.554175 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77741.45 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 7774.15 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194115.699361 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69967.31 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 6996.73 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201112.430028 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62970.58 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 6297.06 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207409.487628 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56673.52 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 5667.35 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213076.839469 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51006.17 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 5100.62 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218177.456125 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45905.55 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 4590.55 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222768.011116 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41314.99 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 4131.50 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226899.510607 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37183.50 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3718.35 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230617.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33465.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234217.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29865.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237817.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26265.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241417.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22665.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245017.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19065.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248617.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15465.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252217.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11865.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255817.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8265.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259417.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4665.15 sec) 03:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2263017.860150 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065.15 sec) 03:53:00 [GPUGRID] [rr_sim] 2264083.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995695904.43G/439.78G) 03:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851103.92 03:53:00 [---] pick_jobs_to_run() start 03:53:00 [---] pick_jobs_to_run() end 03:53:00 [---] [rr_sim] end 03:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:53:00 [---] [cpu_sched_debug] preliminary job list: 03:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:53:00 [---] [cpu_sched_debug] final job list: 03:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:53:00 [---] [cpu_sched_debug] enforce_run_list: end 03:53:00 [---] choose_project(): 13980.000000 03:53:00 [iThena.Measurements] doing RPC to get work 03:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421619.09 cpu 2165312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:53:00 [---] Got 0 tasks 03:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:54:00 [---] [rr_sim] doing sim: CPU sched 03:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995668314.07G/439.78G) 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264020.27 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 226402.03 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226402.026915 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037618.24 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 203761.82 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430163.851138 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833856.42 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 183385.64 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613549.492939 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650470.78 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 165047.08 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778596.570560 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485423.70 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 148542.37 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927138.940419 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336881.33 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 133688.13 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060827.073292 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203193.20 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 120319.32 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181146.392878 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082873.88 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 108287.39 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289433.780505 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974586.49 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 97458.65 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386892.429370 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877127.84 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 87712.78 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474605.213348 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789415.06 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 78941.51 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553546.718928 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710473.55 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 71047.36 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624594.073950 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639426.20 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 63942.62 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688536.693470 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575483.58 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 57548.36 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746085.051038 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517935.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 51793.52 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797878.572849 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466141.70 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 46614.17 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844492.742479 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419527.53 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 41952.75 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886445.495146 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377574.77 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 37757.48 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924202.972546 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339817.30 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 33981.73 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958184.702207 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305835.57 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 30583.56 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988768.258901 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275252.01 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 27525.20 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016293.459926 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247726.81 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 24772.68 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041066.140848 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222954.13 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 22295.41 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063361.553678 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200658.72 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 20065.87 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083427.425225 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180592.84 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 18059.28 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101486.709618 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162533.56 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 16253.36 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117740.065571 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146280.20 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 14628.02 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132368.085929 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131652.18 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 13165.22 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145533.304251 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118486.96 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 11848.70 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157382.000741 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106638.27 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 10663.83 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168045.827581 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95974.44 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 9597.44 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177643.271738 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86377.00 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 8637.70 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186280.971479 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77739.30 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 7773.93 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194054.901246 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69965.37 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 6996.54 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201051.438037 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62968.83 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 6296.88 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207348.321148 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56671.95 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 5667.19 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213015.515948 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51004.75 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 5100.48 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218115.991268 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45904.28 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 4590.43 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222706.419056 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41313.85 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 4131.39 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226837.804066 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37182.47 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3718.25 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230556.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33464.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234156.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29864.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237756.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26264.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241356.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22664.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244956.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19064.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248556.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15464.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252156.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11864.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255756.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8264.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259356.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4664.22 sec) 03:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262956.050574 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064.22 sec) 03:54:00 [GPUGRID] [rr_sim] 2264020.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995668314.07G/439.78G) 03:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851101.18 03:54:00 [---] pick_jobs_to_run() start 03:54:00 [---] pick_jobs_to_run() end 03:54:00 [---] [rr_sim] end 03:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:54:00 [---] [cpu_sched_debug] preliminary job list: 03:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:54:00 [---] [cpu_sched_debug] final job list: 03:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:54:00 [---] [cpu_sched_debug] enforce_run_list: end 03:54:00 [---] choose_project(): 14040.000000 03:54:00 [iThena.Measurements] doing RPC to get work 03:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421559.09 cpu 2165252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:54:00 [---] Got 0 tasks 03:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:55:00 [---] [rr_sim] doing sim: CPU sched 03:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995668314.07G/439.78G) 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2264020.27 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 226402.03 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226402.026915 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037618.24 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 203761.82 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430163.851138 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833856.42 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 183385.64 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613549.492939 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650470.78 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 165047.08 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778596.570560 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485423.70 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 148542.37 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927138.940419 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336881.33 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 133688.13 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060827.073292 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203193.20 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 120319.32 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181146.392878 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082873.88 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 108287.39 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289433.780505 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974586.49 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 97458.65 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386892.429370 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877127.84 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 87712.78 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474605.213348 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789415.06 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 78941.51 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553546.718928 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710473.55 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 71047.36 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624594.073950 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639426.20 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 63942.62 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688536.693470 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575483.58 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 57548.36 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746085.051038 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517935.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 51793.52 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797878.572849 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466141.70 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 46614.17 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844492.742479 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419527.53 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 41952.75 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886445.495146 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377574.77 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 37757.48 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924202.972546 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339817.30 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 33981.73 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958184.702207 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305835.57 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 30583.56 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988768.258901 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275252.01 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 27525.20 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016293.459926 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247726.81 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 24772.68 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041066.140848 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222954.13 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 22295.41 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063361.553678 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200658.72 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 20065.87 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083427.425225 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180592.84 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 18059.28 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101486.709618 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162533.56 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 16253.36 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117740.065571 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146280.20 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 14628.02 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132368.085929 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131652.18 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 13165.22 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145533.304251 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118486.96 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 11848.70 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157382.000741 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106638.27 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 10663.83 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168045.827581 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95974.44 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 9597.44 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177643.271738 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86377.00 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 8637.70 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186280.971479 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77739.30 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 7773.93 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194054.901246 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69965.37 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 6996.54 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201051.438037 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62968.83 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 6296.88 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207348.321148 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56671.95 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 5667.19 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213015.515948 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51004.75 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 5100.48 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218115.991268 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45904.28 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 4590.43 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222706.419056 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41313.85 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 4131.39 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226837.804066 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37182.47 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3718.25 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230556.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33464.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234156.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29864.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237756.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26264.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241356.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22664.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244956.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19064.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248556.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15464.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252156.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11864.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255756.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8264.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259356.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4664.22 sec) 03:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262956.050574 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064.22 sec) 03:55:00 [GPUGRID] [rr_sim] 2264020.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995668314.07G/439.78G) 03:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851161.18 03:55:00 [---] pick_jobs_to_run() start 03:55:00 [---] pick_jobs_to_run() end 03:55:00 [---] [rr_sim] end 03:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:55:00 [---] [cpu_sched_debug] preliminary job list: 03:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:55:00 [---] [cpu_sched_debug] final job list: 03:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:55:00 [---] [cpu_sched_debug] enforce_run_list: end 03:55:00 [---] choose_project(): 14100.000000 03:55:00 [iThena.Measurements] doing RPC to get work 03:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421499.09 cpu 2165252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:55:00 [---] Got 0 tasks 03:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:56:00 [---] [rr_sim] doing sim: CPU sched 03:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995640723.72G/439.78G) 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263957.53 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 226395.75 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226395.753227 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037561.78 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 203756.18 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430151.931131 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833805.60 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 183380.56 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613532.491244 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650425.04 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 165042.50 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778574.995347 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485382.54 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 148538.25 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927113.249039 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336844.28 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 133684.43 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060797.677362 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203159.85 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 120315.99 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181113.662852 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082843.87 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 108284.39 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289398.049794 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974559.48 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 97455.95 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386853.998041 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877103.53 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 87710.35 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474564.351464 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789393.18 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 78939.32 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553503.669544 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710453.86 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 71045.39 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624549.055816 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639408.48 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 63940.85 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688489.903461 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575467.63 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 57546.76 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746036.666342 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517920.87 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 51792.09 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797828.752934 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466128.78 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 46612.88 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844441.630868 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419515.90 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 41951.59 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886393.221008 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377564.31 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 37756.43 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924149.652134 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339807.88 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 33980.79 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958130.440147 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305827.09 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 30582.71 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988713.149359 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275244.38 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 27524.44 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016237.587650 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247719.94 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 24771.99 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041009.582112 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222947.95 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 22294.80 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063304.377127 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200653.16 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 20065.32 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083369.692641 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180587.84 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 18058.78 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101428.476604 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162529.06 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 16252.91 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117681.382170 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146276.15 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 14627.62 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132308.997180 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131648.54 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 13164.85 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145473.850688 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118483.68 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 11848.37 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157322.218846 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106635.31 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 10663.53 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167985.750188 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95971.78 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 9597.18 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177582.928396 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86374.60 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 8637.46 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186220.388783 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77737.14 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 7773.71 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193994.103132 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69963.43 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 6996.34 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200990.446045 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62967.09 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 6296.71 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207287.154667 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56670.38 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 5667.04 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212954.192427 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51003.34 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 5100.33 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218054.526411 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45903.01 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 4590.30 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222644.826997 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41312.71 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 4131.27 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226776.097524 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37181.43 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3718.14 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230494.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33463.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234094.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29863.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237694.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26263.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241294.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22663.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244894.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19063.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248494.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15463.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252094.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11863.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255694.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8263.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259294.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4663.29 sec) 03:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262894.240998 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063.29 sec) 03:56:00 [GPUGRID] [rr_sim] 2263957.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995640723.72G/439.78G) 03:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851158.44 03:56:00 [---] pick_jobs_to_run() start 03:56:00 [---] pick_jobs_to_run() end 03:56:00 [---] [rr_sim] end 03:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:56:00 [---] [cpu_sched_debug] preliminary job list: 03:56:00 [---] [cpu_sched_debug] final job list: 03:56:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 03:56:00 [---] [work_fetch] Request work fetch: CPUs idle 03:56:00 [---] [cpu_sched_debug] enforce_run_list: end 03:56:00 [---] choose_project(): 14160.000000 03:56:00 [iThena.Measurements] doing RPC to get work 03:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421439.09 cpu 2165192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:56:00 [---] Got 0 tasks 03:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:57:00 [---] [rr_sim] doing sim: CPU sched 03:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995640723.72G/439.78G) 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263957.53 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 226395.75 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226395.753227 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037561.78 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 203756.18 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430151.931131 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833805.60 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 183380.56 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613532.491244 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650425.04 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 165042.50 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778574.995347 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485382.54 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 148538.25 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927113.249039 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336844.28 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 133684.43 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060797.677362 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203159.85 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 120315.99 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181113.662852 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082843.87 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 108284.39 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289398.049794 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974559.48 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 97455.95 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386853.998041 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877103.53 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 87710.35 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474564.351464 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789393.18 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 78939.32 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553503.669544 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710453.86 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 71045.39 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624549.055816 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639408.48 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 63940.85 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688489.903461 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575467.63 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 57546.76 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746036.666342 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517920.87 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 51792.09 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797828.752934 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466128.78 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 46612.88 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844441.630868 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419515.90 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 41951.59 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886393.221008 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377564.31 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 37756.43 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924149.652134 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339807.88 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 33980.79 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958130.440147 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305827.09 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 30582.71 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988713.149359 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275244.38 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 27524.44 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016237.587650 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247719.94 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 24771.99 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041009.582112 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222947.95 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 22294.80 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063304.377127 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200653.16 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 20065.32 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083369.692641 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180587.84 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 18058.78 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101428.476604 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162529.06 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 16252.91 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117681.382170 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146276.15 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 14627.62 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132308.997180 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131648.54 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 13164.85 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145473.850688 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118483.68 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 11848.37 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157322.218846 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106635.31 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 10663.53 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167985.750188 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95971.78 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 9597.18 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177582.928396 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86374.60 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 8637.46 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186220.388783 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77737.14 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 7773.71 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193994.103132 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69963.43 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 6996.34 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200990.446045 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62967.09 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 6296.71 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207287.154667 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56670.38 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 5667.04 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212954.192427 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51003.34 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 5100.33 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218054.526411 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45903.01 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 4590.30 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222644.826997 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41312.71 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 4131.27 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226776.097524 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37181.43 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3718.14 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230494.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33463.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234094.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29863.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237694.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26263.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241294.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22663.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244894.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19063.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248494.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15463.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252094.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11863.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255694.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8263.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259294.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4663.29 sec) 03:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262894.240998 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063.29 sec) 03:57:00 [GPUGRID] [rr_sim] 2263957.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995640723.72G/439.78G) 03:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851218.44 03:57:00 [---] pick_jobs_to_run() start 03:57:00 [---] pick_jobs_to_run() end 03:57:00 [---] [rr_sim] end 03:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:57:00 [---] [cpu_sched_debug] preliminary job list: 03:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:57:00 [---] [cpu_sched_debug] final job list: 03:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 03:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:57:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 995640723.72G 03:57:00 [---] [cpu_sched_debug] enforce_run_list: end 03:57:00 [---] choose_project(): 14220.000000 03:57:00 [iThena.Measurements] doing RPC to get work 03:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421379.09 cpu 2165192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:57:00 [---] Got 0 tasks 03:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:58:00 [---] [rr_sim] doing sim: CPU sched 03:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995613133.36G/439.78G) 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263894.80 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 226389.48 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226389.479538 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037505.32 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 203750.53 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430140.011123 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833754.78 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 183375.48 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613515.489549 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650379.31 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 165037.93 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778553.420133 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485341.38 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 148534.14 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927087.557658 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336807.24 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 133680.72 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060768.281431 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203126.51 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 120312.65 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181080.932826 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082813.86 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 108281.39 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289362.319082 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974532.48 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 97453.25 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386815.566712 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877079.23 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 87707.92 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474523.489580 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789371.31 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 78937.13 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553460.620160 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710434.18 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 71043.42 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624504.037683 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639390.76 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 63939.08 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688443.113453 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575451.68 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 57545.17 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745988.281646 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517906.51 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 51790.65 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797778.933020 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466115.86 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 46611.59 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844390.519256 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419504.28 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 41950.43 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886340.946869 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377553.85 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 37755.38 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924096.331721 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339798.46 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 33979.85 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958076.178087 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305818.62 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 30581.86 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988658.039817 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275236.76 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 27523.68 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016181.715374 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247713.08 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 24771.31 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040953.023375 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222941.77 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 22294.18 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063247.200576 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200647.59 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 20064.76 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083311.960057 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180582.84 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 18058.28 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101370.243590 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162524.55 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 16252.46 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117622.698769 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146272.10 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 14627.21 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132249.908431 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131644.89 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 13164.49 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145414.397126 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118480.40 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 11848.04 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157262.436952 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106632.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 10663.24 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167925.672795 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95969.12 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 9596.91 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177522.585054 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86372.21 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 8637.22 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186159.806087 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77734.99 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 7773.50 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193933.305017 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69961.49 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 6996.15 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200929.454054 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62965.34 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 6296.53 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207225.988187 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56668.81 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 5666.88 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212892.868907 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51001.93 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 5100.19 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217993.061555 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45901.73 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 4590.17 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222583.234938 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41311.56 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 4131.16 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226714.390982 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37180.40 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3718.04 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230432.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33462.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234032.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29862.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237632.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26262.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241232.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22662.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244832.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19062.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248432.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15462.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252032.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11862.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255632.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8262.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259232.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4662.36 sec) 03:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262832.431423 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062.36 sec) 03:58:00 [GPUGRID] [rr_sim] 2263894.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995613133.36G/439.78G) 03:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851215.71 03:58:00 [---] pick_jobs_to_run() start 03:58:00 [---] pick_jobs_to_run() end 03:58:00 [---] [rr_sim] end 03:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:58:00 [---] [cpu_sched_debug] preliminary job list: 03:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 03:58:00 [---] [cpu_sched_debug] final job list: 03:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:58:00 [---] [cpu_sched_debug] enforce_run_list: end 03:58:00 [---] choose_project(): 14280.000000 03:58:00 [iThena.Measurements] doing RPC to get work 03:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421319.09 cpu 2165132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:58:00 [---] Got 0 tasks 03:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 03:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 03:59:00 [---] [rr_sim] doing sim: CPU sched 03:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995585543.01G/439.78G) 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263832.06 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 226383.21 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226383.205850 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037448.85 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 203744.89 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430128.091116 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833703.97 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 183370.40 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613498.487854 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650333.57 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 165033.36 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778531.844919 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485300.21 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 148530.02 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927061.866277 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336770.19 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 133677.02 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060738.885500 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203093.17 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 120309.32 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181048.202800 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082783.86 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 108278.39 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289326.588371 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974505.47 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 97450.55 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386777.135384 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877054.92 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 87705.49 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474482.627696 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789349.43 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 78934.94 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553417.570776 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710414.49 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 71041.45 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624459.019549 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639373.04 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 63937.30 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688396.323444 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575435.74 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 57543.57 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745939.896950 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517892.16 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 51789.22 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797729.113105 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466102.95 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 46610.29 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844339.407645 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419492.65 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 41949.27 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886288.672731 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377543.39 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 37754.34 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924043.011308 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339789.05 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 33978.90 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958021.916028 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305810.14 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 30581.01 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988602.930275 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275229.13 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 27522.91 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016125.843098 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247706.22 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 24770.62 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040896.464638 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222935.59 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 22293.56 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063190.024025 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200642.03 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 20064.20 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083254.227473 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180577.83 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 18057.78 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101312.010576 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162520.05 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 16252.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117564.015368 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146268.04 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 14626.80 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132190.819682 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131641.24 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 13164.12 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145354.943564 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118477.11 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 11847.71 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157202.655058 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106629.40 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 10662.94 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167865.595402 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95966.46 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 9596.65 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177462.241712 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86369.82 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 8636.98 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186099.223391 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77732.84 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 7773.28 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193872.506903 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69959.55 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 6995.96 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200868.462063 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62963.60 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 6296.36 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207164.821707 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56667.24 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 5666.72 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212831.545386 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51000.51 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 5100.05 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217931.596698 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45900.46 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 4590.05 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222521.642878 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41310.42 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 4131.04 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226652.684441 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37179.37 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3717.94 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230370.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33461.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233970.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29861.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237570.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26261.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241170.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22661.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244770.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19061.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248370.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15461.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251970.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11861.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255570.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8261.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259170.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4661.44 sec) 03:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262770.621847 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061.44 sec) 03:59:00 [GPUGRID] [rr_sim] 2263832.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995585543.01G/439.78G) 03:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851212.97 03:59:00 [---] pick_jobs_to_run() start 03:59:00 [---] pick_jobs_to_run() end 03:59:00 [---] [rr_sim] end 03:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 03:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 03:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 03:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 03:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 03:59:00 [---] [cpu_sched_debug] preliminary job list: 03:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:59:00 [---] [cpu_sched_debug] final job list: 03:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 03:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 03:59:00 [---] [cpu_sched_debug] enforce_run_list: end 03:59:00 [---] choose_project(): 14340.000000 03:59:00 [iThena.Measurements] doing RPC to get work 03:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 594059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421259.09 cpu 2165072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 03:59:00 [---] Got 0 tasks 04:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:00:00 [---] [rr_sim] doing sim: CPU sched 04:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995557952.65G/439.78G) 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263769.32 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 226376.93 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226376.932162 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037392.39 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 203739.24 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430116.171108 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833653.15 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 183365.32 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613481.486159 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650287.84 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 165028.78 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778510.269705 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485259.05 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 148525.91 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927036.174897 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336733.15 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 133673.31 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060709.489569 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203059.83 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 120305.98 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1181015.472774 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082753.85 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 108275.38 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289290.857659 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974478.46 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 97447.85 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386738.704055 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877030.62 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 87703.06 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474441.765812 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789327.56 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 78932.76 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553374.521393 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710394.80 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 71039.48 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624414.001415 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639355.32 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 63935.53 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688349.533436 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575419.79 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 57541.98 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745891.512254 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517877.81 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 51787.78 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797679.293191 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466090.03 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 46609.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844288.296034 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419481.03 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 41948.10 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886236.398593 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377532.92 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 37753.29 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923989.690895 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339779.63 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 33977.96 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957967.653968 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305801.67 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 30580.17 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988547.820733 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275221.50 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 27522.15 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016069.970822 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247699.35 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 24769.94 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040839.905902 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222929.42 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 22292.94 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063132.847474 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200636.47 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 20063.65 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083196.494888 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180572.83 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 18057.28 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101253.777562 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162515.54 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 16251.55 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117505.331968 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146263.99 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 14626.40 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132131.730933 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131637.59 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 13163.76 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145295.490002 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118473.83 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 11847.38 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157142.873164 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106626.45 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 10662.64 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167805.518009 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95963.80 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 9596.38 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177401.898370 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86367.42 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 8636.74 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186038.640695 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77730.68 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 7773.07 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193811.708788 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69957.61 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 6995.76 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200807.470071 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62961.85 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 6296.19 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207103.655226 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56665.67 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 5666.57 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212770.221866 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50999.10 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 5099.91 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217870.131841 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45899.19 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 4589.92 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222460.050819 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41309.27 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 4130.93 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226590.977899 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37178.34 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3717.83 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230308.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33460.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233908.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29860.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237508.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26260.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241108.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22660.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244708.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19060.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248308.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15460.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251908.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11860.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255508.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8260.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259108.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4660.51 sec) 04:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262708.812271 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060.51 sec) 04:00:00 [GPUGRID] [rr_sim] 2263769.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995557952.65G/439.78G) 04:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851210.23 04:00:00 [---] pick_jobs_to_run() start 04:00:00 [---] pick_jobs_to_run() end 04:00:00 [---] [rr_sim] end 04:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:00:00 [---] [cpu_sched_debug] preliminary job list: 04:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:00:00 [---] [cpu_sched_debug] final job list: 04:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:00:00 [---] [cpu_sched_debug] enforce_run_list: end 04:00:00 [---] choose_project(): 14400.000000 04:00:00 [iThena.Measurements] doing RPC to get work 04:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421199.09 cpu 2165012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2165012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2165012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:00:00 [---] Got 0 tasks 04:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:01:00 [---] [rr_sim] doing sim: CPU sched 04:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:01:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851207.50 04:01:00 [---] pick_jobs_to_run() start 04:01:00 [---] pick_jobs_to_run() end 04:01:00 [---] [rr_sim] end 04:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:01:00 [---] [cpu_sched_debug] preliminary job list: 04:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:01:00 [---] [cpu_sched_debug] final job list: 04:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:01:00 [---] [cpu_sched_debug] enforce_run_list: end 04:01:00 [---] choose_project(): 14460.000000 04:01:00 [iThena.Measurements] doing RPC to get work 04:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421139.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:01:00 [---] Got 0 tasks 04:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:02:00 [---] [rr_sim] doing sim: CPU sched 04:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:02:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851267.50 04:02:00 [---] pick_jobs_to_run() start 04:02:00 [---] pick_jobs_to_run() end 04:02:00 [---] [rr_sim] end 04:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:02:00 [---] [cpu_sched_debug] preliminary job list: 04:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:02:00 [---] [cpu_sched_debug] final job list: 04:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:02:00 [---] [cpu_sched_debug] enforce_run_list: end 04:02:00 [---] choose_project(): 14520.000000 04:02:00 [iThena.Measurements] doing RPC to get work 04:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421079.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:02:00 [---] Got 0 tasks 04:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:03:00 [---] [rr_sim] doing sim: CPU sched 04:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:03:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851327.50 04:03:00 [---] pick_jobs_to_run() start 04:03:00 [---] pick_jobs_to_run() end 04:03:00 [---] [rr_sim] end 04:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:03:00 [---] [cpu_sched_debug] preliminary job list: 04:03:00 [---] [cpu_sched_debug] final job list: 04:03:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:03:00 [---] [work_fetch] Request work fetch: CPUs idle 04:03:00 [---] [cpu_sched_debug] enforce_run_list: end 04:03:00 [---] choose_project(): 14580.000000 04:03:00 [iThena.Measurements] doing RPC to get work 04:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 421019.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:03:00 [---] Got 0 tasks 04:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:04:00 [---] [rr_sim] doing sim: CPU sched 04:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:04:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851387.50 04:04:00 [---] pick_jobs_to_run() start 04:04:00 [---] pick_jobs_to_run() end 04:04:00 [---] [rr_sim] end 04:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:04:00 [---] [cpu_sched_debug] preliminary job list: 04:04:00 [---] [cpu_sched_debug] final job list: 04:04:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:04:00 [---] [work_fetch] Request work fetch: CPUs idle 04:04:00 [---] [cpu_sched_debug] enforce_run_list: end 04:04:00 [---] choose_project(): 14640.000000 04:04:00 [iThena.Measurements] doing RPC to get work 04:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420959.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:04:00 [---] Got 0 tasks 04:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:05:00 [---] [rr_sim] doing sim: CPU sched 04:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:05:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851447.50 04:05:00 [---] pick_jobs_to_run() start 04:05:00 [---] pick_jobs_to_run() end 04:05:00 [---] [rr_sim] end 04:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:05:00 [---] [cpu_sched_debug] preliminary job list: 04:05:00 [---] [cpu_sched_debug] final job list: 04:05:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:05:00 [---] [work_fetch] Request work fetch: CPUs idle 04:05:00 [---] [cpu_sched_debug] enforce_run_list: end 04:05:00 [---] choose_project(): 14700.000000 04:05:00 [iThena.Measurements] doing RPC to get work 04:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420899.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:05:00 [---] Got 0 tasks 04:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:06:00 [---] [rr_sim] doing sim: CPU sched 04:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:06:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851507.50 04:06:00 [---] pick_jobs_to_run() start 04:06:00 [---] pick_jobs_to_run() end 04:06:00 [---] [rr_sim] end 04:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:06:00 [---] [cpu_sched_debug] preliminary job list: 04:06:00 [---] [cpu_sched_debug] final job list: 04:06:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:06:00 [---] [work_fetch] Request work fetch: CPUs idle 04:06:00 [---] [cpu_sched_debug] enforce_run_list: end 04:06:00 [---] choose_project(): 14760.000000 04:06:00 [iThena.Measurements] doing RPC to get work 04:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420839.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:06:00 [---] Got 0 tasks 04:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:07:00 [---] [rr_sim] doing sim: CPU sched 04:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:07:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851567.50 04:07:00 [---] pick_jobs_to_run() start 04:07:00 [---] pick_jobs_to_run() end 04:07:00 [---] [rr_sim] end 04:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:07:00 [---] [cpu_sched_debug] preliminary job list: 04:07:00 [---] [cpu_sched_debug] final job list: 04:07:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:07:00 [---] [work_fetch] Request work fetch: CPUs idle 04:07:00 [---] [cpu_sched_debug] enforce_run_list: end 04:07:00 [---] choose_project(): 14820.000000 04:07:00 [iThena.Measurements] doing RPC to get work 04:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420779.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:07:00 [---] Got 0 tasks 04:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:08:00 [---] [rr_sim] doing sim: CPU sched 04:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:08:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851627.50 04:08:00 [---] pick_jobs_to_run() start 04:08:00 [---] pick_jobs_to_run() end 04:08:00 [---] [rr_sim] end 04:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:08:00 [---] [cpu_sched_debug] preliminary job list: 04:08:00 [---] [cpu_sched_debug] final job list: 04:08:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:08:00 [---] [work_fetch] Request work fetch: CPUs idle 04:08:00 [---] [cpu_sched_debug] enforce_run_list: end 04:08:00 [---] choose_project(): 14880.000000 04:08:00 [iThena.Measurements] doing RPC to get work 04:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420719.09 cpu 2164952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:08:00 [---] Got 0 tasks 04:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:09:00 [---] [rr_sim] doing sim: CPU sched 04:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263706.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 226370.66 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226370.658474 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037335.93 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 203733.59 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430104.251100 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833602.33 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 183360.23 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613464.484464 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650242.10 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 165024.21 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778488.694492 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485217.89 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 148521.79 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 927010.483516 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336696.10 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 133669.61 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060680.093638 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1203026.49 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 120302.65 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180982.742748 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082723.84 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 108272.38 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289255.126947 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974451.46 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 97445.15 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386700.272727 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877006.31 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 87700.63 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474400.903928 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789305.68 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 78930.57 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553331.472009 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710375.11 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 71037.51 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624368.983282 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639337.60 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 63933.76 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688302.743427 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575403.84 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 57540.38 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745843.127559 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517863.46 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 51786.35 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797629.473277 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466077.11 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 46607.71 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844237.184423 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419469.40 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 41946.94 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886184.124454 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377522.46 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 37752.25 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923936.370483 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339770.21 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 33977.02 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957913.391908 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305793.19 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 30579.32 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988492.711191 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275213.87 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 27521.39 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016014.098546 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247692.49 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 24769.25 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040783.347165 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222923.24 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 22292.32 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063075.670923 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200630.91 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 20063.09 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083138.762304 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180567.82 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 18056.78 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101195.544548 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162511.04 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 16251.10 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117446.648567 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146259.94 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 14625.99 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132072.642184 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131633.94 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 13163.39 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145236.036439 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118470.55 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 11847.05 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157083.091269 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106623.49 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 10662.35 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167745.440616 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95961.14 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 9596.11 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177341.555028 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86365.03 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 8636.50 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185978.057999 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77728.53 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 7772.85 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193750.910673 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69955.67 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 6995.57 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200746.478080 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62960.11 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 6296.01 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207042.488746 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56664.10 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 5666.41 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.898345 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50997.69 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 5099.77 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217808.666984 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45897.92 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 4589.79 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222398.458760 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41308.13 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 4130.81 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226529.271358 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37177.31 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3717.73 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230247.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33459.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233847.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29859.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237447.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26259.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241047.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22659.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244647.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19059.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248247.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15459.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251847.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11859.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255447.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8259.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259047.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4659.58 sec) 04:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262647.002696 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059.58 sec) 04:09:00 [GPUGRID] [rr_sim] 2263706.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995530362.29G/439.78G) 04:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851687.50 04:09:00 [---] pick_jobs_to_run() start 04:09:00 [---] pick_jobs_to_run() end 04:09:00 [---] [rr_sim] end 04:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:09:00 [---] [cpu_sched_debug] preliminary job list: 04:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:09:00 [---] [cpu_sched_debug] final job list: 04:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:09:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 995530362.29G 04:09:00 [---] [cpu_sched_debug] enforce_run_list: end 04:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:10:00 [---] [rr_sim] doing sim: CPU sched 04:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995502771.94G/439.78G) 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263643.85 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 226364.38 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226364.384786 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037279.46 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 203727.95 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430092.331093 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833551.52 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 183355.15 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613447.482769 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650196.37 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 165019.64 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778467.119278 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485176.73 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 148517.67 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926984.792136 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336659.06 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 133665.91 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060650.697708 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202993.15 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 120299.32 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180950.012723 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082693.84 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 108269.38 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289219.396236 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974424.45 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 97442.45 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386661.841398 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876982.01 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 87698.20 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474360.042044 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789283.81 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 78928.38 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553288.422625 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710355.43 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 71035.54 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624323.965148 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639319.88 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 63931.99 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688255.953419 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575387.89 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 57538.79 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745794.742863 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517849.10 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 51784.91 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797579.653362 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466064.19 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 46606.42 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844186.072812 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419457.78 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 41945.78 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886131.850316 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377512.00 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 37751.20 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923883.050070 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339760.80 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 33976.08 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957859.129849 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305784.72 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 30578.47 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988437.601649 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275206.25 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 27520.62 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015958.226270 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247685.62 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 24768.56 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040726.788429 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222917.06 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 22291.71 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063018.494371 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200625.35 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 20062.54 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083081.029720 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180562.82 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 18056.28 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101137.311534 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162506.54 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 16250.65 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117387.965166 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146255.88 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 14625.59 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132013.553435 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131630.29 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 13163.03 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145176.582877 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118467.26 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 11846.73 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157023.309375 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106620.54 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 10662.05 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167685.363223 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95958.48 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 9595.85 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177281.211686 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86362.64 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 8636.26 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185917.475303 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77726.37 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 7772.64 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193690.112559 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69953.74 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 6995.37 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200685.486088 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62958.36 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 6295.84 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206981.322265 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56662.53 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 5666.25 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212647.574824 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50996.27 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 5099.63 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217747.202128 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45896.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 4589.66 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222336.866700 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41306.98 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 4130.70 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226467.564816 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37176.28 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3717.63 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230185.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33458.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233785.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29858.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237385.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26258.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240985.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22658.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244585.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19058.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248185.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15458.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251785.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11858.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255385.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8258.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258985.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4658.65 sec) 04:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262585.193120 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058.65 sec) 04:10:00 [GPUGRID] [rr_sim] 2263643.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995502771.94G/439.78G) 04:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851684.76 04:10:00 [---] pick_jobs_to_run() start 04:10:00 [---] pick_jobs_to_run() end 04:10:00 [---] [rr_sim] end 04:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:10:00 [---] [cpu_sched_debug] preliminary job list: 04:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:10:00 [---] [cpu_sched_debug] final job list: 04:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:10:00 [---] [cpu_sched_debug] enforce_run_list: end 04:10:00 [---] choose_project(): 15000.000000 04:10:00 [iThena.Measurements] doing RPC to get work 04:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420599.09 cpu 2164892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:10:00 [---] Got 0 tasks 04:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:11:00 [---] [rr_sim] doing sim: CPU sched 04:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995475181.58G/439.78G) 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263581.11 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 226358.11 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226358.111097 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037223.00 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 203722.30 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430080.411085 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833500.70 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 183350.07 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613430.481074 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650150.63 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 165015.06 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778445.544064 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485135.57 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 148513.56 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926959.100755 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336622.01 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 133662.20 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060621.301777 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202959.81 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 120295.98 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180917.282697 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082663.83 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 108266.38 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289183.665524 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974397.45 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 97439.74 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386623.410069 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876957.70 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 87695.77 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474319.180160 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789261.93 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 78926.19 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553245.373241 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710335.74 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 71033.57 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624278.947015 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639302.16 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 63930.22 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688209.163411 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575371.95 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 57537.19 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745746.358167 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517834.75 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 51783.48 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797529.833448 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466051.28 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 46605.13 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844134.961200 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419446.15 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 41944.61 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886079.576178 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377501.53 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 37750.15 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923829.729657 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339751.38 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 33975.14 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957804.867789 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305776.24 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 30577.62 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988382.492107 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275198.62 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 27519.86 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015902.353994 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247678.76 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 24767.88 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040670.229692 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222910.88 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 22291.09 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062961.317820 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200619.79 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 20061.98 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083023.297136 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180557.81 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 18055.78 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101079.078520 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162502.03 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 16250.20 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117329.281765 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146251.83 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 14625.18 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131954.464686 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131626.65 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 13162.66 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145117.129315 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118463.98 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 11846.40 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156963.527481 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106617.58 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 10661.76 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167625.285830 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95955.83 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 9595.58 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177220.868344 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86360.24 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 8636.02 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185856.892607 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77724.22 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 7772.42 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193629.314444 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69951.80 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 6995.18 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200624.494097 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62956.62 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 6295.66 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206920.155785 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56660.96 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 5666.10 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212586.251304 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50994.86 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 5099.49 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217685.737271 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45895.37 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 4589.54 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222275.274641 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41305.84 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 4130.58 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226405.858274 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37175.25 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3717.53 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230123.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33457.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233723.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29857.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237323.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26257.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240923.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22657.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244523.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19057.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248123.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15457.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251723.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11857.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255323.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8257.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258923.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4657.73 sec) 04:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262523.383544 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057.73 sec) 04:11:00 [GPUGRID] [rr_sim] 2263581.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995475181.58G/439.78G) 04:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851682.02 04:11:00 [---] pick_jobs_to_run() start 04:11:00 [---] pick_jobs_to_run() end 04:11:00 [---] [rr_sim] end 04:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:11:00 [---] [cpu_sched_debug] preliminary job list: 04:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:11:00 [---] [cpu_sched_debug] final job list: 04:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:11:00 [---] [cpu_sched_debug] enforce_run_list: end 04:11:00 [---] choose_project(): 15060.000000 04:11:00 [iThena.Measurements] doing RPC to get work 04:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420539.09 cpu 2164832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:11:00 [---] Got 0 tasks 04:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:12:00 [---] [rr_sim] doing sim: CPU sched 04:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995447591.23G/439.78G) 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263518.37 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 226351.84 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226351.837409 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037166.54 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 203716.65 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430068.491077 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833449.88 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 183344.99 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613413.479379 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650104.89 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 165010.49 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778423.968850 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485094.41 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 148509.44 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926933.409374 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336584.96 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 133658.50 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060591.905846 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202926.47 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 120292.65 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180884.552671 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082633.82 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 108263.38 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289147.934813 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974370.44 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 97437.04 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386584.978741 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876933.40 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 87693.34 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474278.318276 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789240.06 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 78924.01 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553202.323858 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710316.05 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 71031.61 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624233.928881 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639284.45 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 63928.44 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688162.373402 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575356.00 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 57535.60 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745697.973471 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517820.40 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 51782.04 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797480.013533 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466038.36 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 46603.84 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844083.849589 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419434.52 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 41943.45 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886027.302039 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377491.07 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 37749.11 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923776.409245 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339741.96 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 33974.20 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957750.605729 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305767.77 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 30576.78 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988327.382566 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275190.99 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 27519.10 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015846.481718 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247671.89 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 24767.19 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040613.670956 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222904.70 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 22290.47 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062904.141269 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200614.23 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 20061.42 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082965.564552 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180552.81 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 18055.28 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101020.845506 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162497.53 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 16249.75 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117270.598364 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146247.78 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 14624.78 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131895.375937 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131623.00 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 13162.30 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145057.675752 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118460.70 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 11846.07 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156903.745586 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106614.63 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 10661.46 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167565.208437 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95953.17 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 9595.32 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177160.525002 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86357.85 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 8635.78 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185796.309911 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77722.06 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 7772.21 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193568.516329 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69949.86 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 6994.99 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200563.502106 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62954.87 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 6295.49 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206858.989304 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56659.38 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 5665.94 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212524.927783 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50993.45 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 5099.34 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217624.272414 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45894.10 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 4589.41 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222213.682582 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41304.69 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 4130.47 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226344.151733 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37174.22 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3717.42 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230061.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33456.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233661.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29856.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237261.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26256.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240861.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22656.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244461.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19056.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248061.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15456.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251661.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11856.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255261.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8256.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258861.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4656.80 sec) 04:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262461.573969 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056.80 sec) 04:12:00 [GPUGRID] [rr_sim] 2263518.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995447591.23G/439.78G) 04:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851679.29 04:12:00 [---] pick_jobs_to_run() start 04:12:00 [---] pick_jobs_to_run() end 04:12:00 [---] [rr_sim] end 04:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:12:00 [---] [cpu_sched_debug] preliminary job list: 04:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:12:00 [---] [cpu_sched_debug] final job list: 04:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:12:00 [---] [cpu_sched_debug] enforce_run_list: end 04:12:00 [---] choose_project(): 15120.000000 04:12:00 [iThena.Measurements] doing RPC to get work 04:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420479.09 cpu 2164772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:12:00 [---] Got 0 tasks 04:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:13:00 [---] [rr_sim] doing sim: CPU sched 04:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995420000.87G/439.78G) 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263455.64 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 226345.56 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226345.563721 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037110.07 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 203711.01 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430056.571070 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833399.07 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 183339.91 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613396.477684 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650059.16 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 165005.92 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778402.393636 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485053.24 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 148505.32 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926907.717994 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336547.92 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 133654.79 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060562.509915 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202893.13 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 120289.31 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180851.822645 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082603.81 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 108260.38 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289112.204101 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974343.43 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 97434.34 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386546.547412 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876909.09 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 87690.91 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474237.456392 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789218.18 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 78921.82 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553159.274474 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710296.36 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 71029.64 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624188.910747 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639266.73 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 63926.67 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688115.583394 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575340.05 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 57534.01 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745649.588775 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517806.05 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 51780.60 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797430.193619 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466025.44 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 46602.54 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844032.737978 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419422.90 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 41942.29 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885975.027901 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377480.61 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 37748.06 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923723.088832 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339732.55 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 33973.25 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957696.343670 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305759.29 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 30575.93 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988272.273024 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275183.36 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 27518.34 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015790.609442 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247665.03 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 24766.50 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040557.112219 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222898.52 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 22289.85 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062846.964718 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200608.67 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 20060.87 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082907.831967 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180547.81 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 18054.78 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100962.612492 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162493.02 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 16249.30 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117211.914963 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146243.72 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 14624.37 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131836.287188 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131619.35 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 13161.94 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144998.222190 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118457.42 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 11845.74 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156843.963692 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106611.67 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 10661.17 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167505.131044 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95950.51 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 9595.05 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177100.181661 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86355.46 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 8635.55 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185735.727215 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77719.91 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 7771.99 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193507.718215 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69947.92 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 6994.79 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200502.510114 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62953.13 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 6295.31 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206797.822824 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56657.81 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 5665.78 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212463.604262 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50992.03 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 5099.20 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217562.807557 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45892.83 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 4589.28 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222152.090522 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41303.55 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 4130.35 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226282.445191 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37173.19 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3717.32 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229999.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33455.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233599.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29855.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237199.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26255.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240799.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22655.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244399.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19055.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247999.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15455.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251599.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11855.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255199.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8255.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258799.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4655.87 sec) 04:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262399.764393 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055.87 sec) 04:13:00 [GPUGRID] [rr_sim] 2263455.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995420000.87G/439.78G) 04:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851676.55 04:13:00 [---] pick_jobs_to_run() start 04:13:00 [---] pick_jobs_to_run() end 04:13:00 [---] [rr_sim] end 04:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:13:00 [---] [cpu_sched_debug] preliminary job list: 04:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:13:00 [---] [cpu_sched_debug] final job list: 04:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:13:00 [---] [cpu_sched_debug] enforce_run_list: end 04:13:00 [---] choose_project(): 15180.000000 04:13:00 [iThena.Measurements] doing RPC to get work 04:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420419.09 cpu 2164712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:13:00 [---] Got 0 tasks 04:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:14:00 [---] [rr_sim] doing sim: CPU sched 04:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995392410.51G/439.78G) 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263392.90 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 226339.29 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226339.290033 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2037053.61 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 203705.36 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430044.651062 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833348.25 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 183334.82 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613379.475989 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1650013.42 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 165001.34 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778380.818423 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1485012.08 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 148501.21 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926882.026613 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336510.87 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 133651.09 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060533.113985 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202859.79 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 120285.98 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180819.092619 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082573.81 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 108257.38 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289076.473390 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974316.43 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 97431.64 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386508.116084 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876884.78 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 87688.48 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474196.594508 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789196.31 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 78919.63 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553116.225090 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710276.68 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 71027.67 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624143.892614 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639249.01 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 63924.90 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688068.793385 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575324.11 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 57532.41 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745601.204079 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517791.70 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 51779.17 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797380.373704 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466012.53 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 46601.25 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843981.626367 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419411.27 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 41941.13 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885922.753763 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377470.15 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 37747.01 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923669.768419 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339723.13 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 33972.31 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957642.081610 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305750.82 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 30575.08 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988217.163482 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275175.74 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 27517.57 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015734.737166 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247658.16 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 24765.82 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040500.553482 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222892.35 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 22289.23 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062789.788167 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200603.11 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 20060.31 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082850.099383 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180542.80 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 18054.28 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100904.379478 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162488.52 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 16248.85 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117153.231563 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146239.67 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 14623.97 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131777.198439 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131615.70 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 13161.57 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144938.768628 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118454.13 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 11845.41 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156784.181798 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106608.72 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 10660.87 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167445.053651 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95947.85 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 9594.78 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177039.838319 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86353.06 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 8635.31 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185675.144519 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77717.76 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 7771.78 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193446.920100 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69945.98 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 6994.60 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200441.518123 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62951.38 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 6295.14 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206736.656343 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56656.24 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 5665.62 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212402.280742 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50990.62 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 5099.06 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217501.342700 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45891.56 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 4589.16 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222090.498463 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41302.40 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 4130.24 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226220.738650 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37172.16 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3717.22 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229937.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33454.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233537.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29854.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237137.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26254.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240737.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22654.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244337.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19054.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247937.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15454.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251537.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11854.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255137.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8254.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258737.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4654.95 sec) 04:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262337.954817 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054.95 sec) 04:14:00 [GPUGRID] [rr_sim] 2263392.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995392410.51G/439.78G) 04:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851673.81 04:14:00 [---] pick_jobs_to_run() start 04:14:00 [---] pick_jobs_to_run() end 04:14:00 [---] [rr_sim] end 04:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:14:00 [---] [cpu_sched_debug] preliminary job list: 04:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:14:00 [---] [cpu_sched_debug] final job list: 04:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:14:00 [---] [cpu_sched_debug] enforce_run_list: end 04:14:00 [---] choose_project(): 15240.000000 04:14:00 [iThena.Measurements] doing RPC to get work 04:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420359.09 cpu 2164652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:14:00 [---] Got 0 tasks 04:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:15:00 [---] [rr_sim] doing sim: CPU sched 04:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995364820.16G/439.78G) 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263330.16 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 226333.02 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226333.016345 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036997.15 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 203699.71 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430032.731055 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833297.43 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 183329.74 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613362.474294 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649967.69 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 164996.77 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778359.243209 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484970.92 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 148497.09 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926856.335233 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336473.83 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 133647.38 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060503.718054 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202826.45 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 120282.64 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180786.362593 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082543.80 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 108254.38 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289040.742678 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974289.42 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 97428.94 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386469.684755 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876860.48 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 87686.05 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474155.732624 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789174.43 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 78917.44 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553073.175706 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710256.99 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 71025.70 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624098.874480 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639231.29 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 63923.13 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1688022.003377 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575308.16 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 57530.82 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745552.819384 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517777.34 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 51777.73 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797330.553790 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465999.61 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 46599.96 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843930.514755 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419399.65 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 41939.96 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885870.479624 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377459.68 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 37745.97 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923616.448006 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339713.72 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 33971.37 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957587.819550 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305742.34 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 30574.23 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988162.053940 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275168.11 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 27516.81 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015678.864890 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247651.30 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 24765.13 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040443.994746 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222886.17 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 22288.62 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062732.611616 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200597.55 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 20059.76 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082792.366799 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180537.80 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 18053.78 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100846.146463 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162484.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 16248.40 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117094.548162 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146235.62 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 14623.56 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131718.109690 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131612.05 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 13161.21 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144879.315066 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118450.85 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 11845.08 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156724.399904 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106605.76 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 10660.58 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167384.976258 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95945.19 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 9594.52 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176979.494977 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86350.67 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 8635.07 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185614.561823 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77715.60 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 7771.56 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193386.121986 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69944.04 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 6994.40 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200380.526132 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62949.64 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 6294.96 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206675.489863 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56654.67 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 5665.47 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212340.957221 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50989.21 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 5098.92 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217439.877844 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45890.29 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 4589.03 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222028.906404 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41301.26 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 4130.13 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226159.032108 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37171.13 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3717.11 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229876.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33454.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233476.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29854.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237076.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26254.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240676.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22654.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244276.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19054.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247876.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15454.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251476.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11854.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255076.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8254.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258676.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4654.02 sec) 04:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262276.145242 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054.02 sec) 04:15:00 [GPUGRID] [rr_sim] 2263330.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995364820.16G/439.78G) 04:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851671.08 04:15:00 [---] pick_jobs_to_run() start 04:15:00 [---] pick_jobs_to_run() end 04:15:00 [---] [rr_sim] end 04:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:15:00 [---] [cpu_sched_debug] preliminary job list: 04:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:15:00 [---] [cpu_sched_debug] final job list: 04:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:15:00 [---] [cpu_sched_debug] enforce_run_list: end 04:15:00 [---] choose_project(): 15300.000000 04:15:00 [iThena.Measurements] doing RPC to get work 04:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420299.09 cpu 2164592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:15:00 [---] Got 0 tasks 04:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:16:00 [---] [rr_sim] doing sim: CPU sched 04:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995337229.80G/439.78G) 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263267.43 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 226326.74 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226326.742656 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036940.68 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 203694.07 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430020.811047 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833246.62 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 183324.66 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613345.472599 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649921.95 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 164992.20 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778337.667995 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484929.76 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 148492.98 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926830.643852 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336436.78 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 133643.68 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060474.322123 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202793.10 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 120279.31 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180753.632567 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082513.79 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 108251.38 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1289005.011967 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974262.41 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 97426.24 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386431.253426 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876836.17 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 87683.62 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474114.870740 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789152.56 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 78915.26 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1553030.126322 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710237.30 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 71023.73 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624053.856346 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639213.57 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 63921.36 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687975.213368 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575292.21 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 57529.22 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745504.434688 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517762.99 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 51776.30 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797280.733875 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465986.69 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 46598.67 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843879.403144 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419388.02 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 41938.80 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885818.205486 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377449.22 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 37744.92 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923563.127594 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339704.30 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 33970.43 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957533.557491 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305733.87 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 30573.39 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988106.944398 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275160.48 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 27516.05 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015622.992614 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247644.43 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 24764.44 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040387.436009 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222879.99 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 22288.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062675.435065 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200591.99 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 20059.20 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082734.634215 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180532.79 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 18053.28 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100787.913449 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162479.51 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 16247.95 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117035.864761 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146231.56 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 14623.16 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131659.020941 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131608.41 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 13160.84 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144819.861503 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118447.57 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 11844.76 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156664.618009 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106602.81 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 10660.28 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167324.898865 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95942.53 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 9594.25 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176919.151635 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86348.27 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 8634.83 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185553.979127 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77713.45 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 7771.34 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193325.323871 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69942.10 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 6994.21 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200319.534140 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62947.89 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 6294.79 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206614.323383 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56653.10 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 5665.31 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212279.633701 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50987.79 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 5098.78 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217378.412987 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45889.01 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 4588.90 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221967.314345 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41300.11 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 4130.01 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226097.325566 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37170.10 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3717.01 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229814.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33453.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233414.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29853.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237014.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26253.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240614.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22653.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244214.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19053.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247814.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15453.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251414.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11853.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255014.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8253.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258614.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4653.09 sec) 04:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262214.335666 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053.09 sec) 04:16:00 [GPUGRID] [rr_sim] 2263267.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995337229.80G/439.78G) 04:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851668.34 04:16:00 [---] pick_jobs_to_run() start 04:16:00 [---] pick_jobs_to_run() end 04:16:00 [---] [rr_sim] end 04:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:16:00 [---] [cpu_sched_debug] preliminary job list: 04:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:16:00 [---] [cpu_sched_debug] final job list: 04:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:16:00 [---] [cpu_sched_debug] enforce_run_list: end 04:16:00 [---] choose_project(): 15360.000000 04:16:00 [iThena.Measurements] doing RPC to get work 04:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 593039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420239.09 cpu 2164532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:16:00 [---] Got 0 tasks 04:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:17:00 [---] [rr_sim] doing sim: CPU sched 04:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995309639.44G/439.78G) 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263204.69 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 226320.47 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226320.468968 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036884.22 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 203688.42 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 430008.891039 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833195.80 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 183319.58 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613328.470904 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649876.22 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 164987.62 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778316.092781 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484888.60 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 148488.86 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926804.952471 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336399.74 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 133639.97 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060444.926192 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202759.76 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 120275.98 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180720.902541 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082483.79 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 108248.38 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288969.281255 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974235.41 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 97423.54 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386392.822098 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876811.87 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 87681.19 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474074.008856 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789130.68 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 78913.07 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552987.076939 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710217.61 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 71021.76 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1624008.838213 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639195.85 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 63919.59 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687928.423360 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575276.27 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 57527.63 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745456.049992 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517748.64 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 51774.86 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797230.913961 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465973.78 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 46597.38 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843828.291533 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419376.40 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 41937.64 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885765.931348 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377438.76 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 37743.88 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923509.807181 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339694.88 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 33969.49 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957479.295431 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305725.39 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 30572.54 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988051.834856 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275152.85 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 27515.29 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015567.120339 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247637.57 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 24763.76 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040330.877273 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222873.81 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 22287.38 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062618.258514 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200586.43 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 20058.64 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082676.901630 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180527.79 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 18052.78 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100729.680435 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162475.01 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 16247.50 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116977.181360 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146227.51 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 14622.75 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131599.932192 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131604.76 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 13160.48 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144760.407941 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118444.28 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 11844.43 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156604.836115 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106599.85 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 10659.99 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167264.821472 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95939.87 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 9593.99 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176858.808293 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86345.88 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 8634.59 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185493.396431 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77711.29 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 7771.13 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193264.525756 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69940.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 6994.02 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200258.542149 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62946.15 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 6294.61 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206553.156902 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56651.53 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 5665.15 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212218.310180 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50986.38 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 5098.64 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217316.948130 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45887.74 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 4588.77 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221905.722285 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41298.97 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 4129.90 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226035.619025 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37169.07 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3716.91 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229752.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33452.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233352.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29852.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236952.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26252.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240552.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22652.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244152.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19052.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247752.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15452.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251352.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11852.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254952.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8252.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258552.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4652.16 sec) 04:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262152.526090 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052.16 sec) 04:17:00 [GPUGRID] [rr_sim] 2263204.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995309639.44G/439.78G) 04:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851665.60 04:17:00 [---] pick_jobs_to_run() start 04:17:00 [---] pick_jobs_to_run() end 04:17:00 [---] [rr_sim] end 04:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:17:00 [---] [cpu_sched_debug] preliminary job list: 04:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:17:00 [---] [cpu_sched_debug] final job list: 04:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:17:00 [---] [cpu_sched_debug] enforce_run_list: end 04:17:00 [---] choose_project(): 15420.000000 04:17:00 [iThena.Measurements] doing RPC to get work 04:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420179.09 cpu 2164472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:17:00 [---] Got 0 tasks 04:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:18:00 [---] [rr_sim] doing sim: CPU sched 04:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995282049.09G/439.78G) 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263141.95 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 226314.20 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226314.195280 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036827.76 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 203682.78 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429996.971032 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833144.98 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 183314.50 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613311.469209 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649830.48 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 164983.05 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778294.517568 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484847.44 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 148484.74 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926779.261091 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336362.69 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 133636.27 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060415.530262 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202726.42 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 120272.64 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180688.172515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082453.78 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 108245.38 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288933.550544 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974208.40 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 97420.84 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386354.390769 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876787.56 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 87678.76 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1474033.146972 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789108.81 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 78910.88 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552944.027555 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710197.93 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 71019.79 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623963.820079 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639178.13 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 63917.81 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687881.633351 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575260.32 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 57526.03 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745407.665296 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517734.29 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 51773.43 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797181.094046 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465960.86 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 46596.09 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843777.179922 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419364.77 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 41936.48 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885713.657209 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377428.30 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 37742.83 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923456.486768 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339685.47 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 33968.55 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957425.033371 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305716.92 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 30571.69 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987996.725314 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275145.23 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 27514.52 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015511.248063 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247630.70 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 24763.07 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040274.318536 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222867.63 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 22286.76 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062561.081963 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200580.87 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 20058.09 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082619.169046 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180522.78 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 18052.28 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100671.447421 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162470.51 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 16247.05 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116918.497959 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146223.45 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 14622.35 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131540.843443 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131601.11 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 13160.11 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144700.954379 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118441.00 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 11844.10 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156545.054221 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106596.90 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 10659.69 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167204.744079 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95937.21 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 9593.72 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176798.464951 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86343.49 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 8634.35 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185432.813735 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77709.14 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 7770.91 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193203.727642 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69938.23 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 6993.82 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200197.550158 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62944.40 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 6294.44 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206491.990422 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56649.96 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 5665.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212156.986659 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50984.97 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 5098.50 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217255.483273 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45886.47 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 4588.65 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221844.130226 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41297.82 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 4129.78 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225973.912483 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37168.04 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3716.80 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229690.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33451.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233290.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29851.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236890.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26251.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240490.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22651.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244090.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19051.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247690.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15451.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251290.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11851.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254890.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8251.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258490.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4651.24 sec) 04:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262090.716515 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051.24 sec) 04:18:00 [GPUGRID] [rr_sim] 2263141.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995282049.09G/439.78G) 04:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851662.87 04:18:00 [---] pick_jobs_to_run() start 04:18:00 [---] pick_jobs_to_run() end 04:18:00 [---] [rr_sim] end 04:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:18:00 [---] [cpu_sched_debug] preliminary job list: 04:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:18:00 [---] [cpu_sched_debug] final job list: 04:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:18:00 [---] [cpu_sched_debug] enforce_run_list: end 04:18:00 [---] choose_project(): 15480.000000 04:18:00 [iThena.Measurements] doing RPC to get work 04:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420119.09 cpu 2164412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:18:00 [---] Got 0 tasks 04:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:19:00 [---] [rr_sim] doing sim: CPU sched 04:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995254458.73G/439.78G) 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263079.22 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 226307.92 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226307.921592 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036771.29 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 203677.13 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429985.051024 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833094.16 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 183309.42 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613294.467513 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649784.75 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 164978.47 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778272.942354 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484806.27 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 148480.63 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926753.569710 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336325.65 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 133632.56 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060386.134331 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202693.08 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 120269.31 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180655.442489 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082423.77 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 108242.38 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288897.819832 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974181.40 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 97418.14 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386315.959441 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876763.26 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 87676.33 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473992.285088 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789086.93 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 78908.69 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552900.978171 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710178.24 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 71017.82 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623918.801946 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639160.41 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 63916.04 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687834.843343 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575244.37 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 57524.44 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745359.280600 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517719.94 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 51771.99 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797131.274132 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465947.94 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 46594.79 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843726.068310 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419353.15 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 41935.31 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885661.383071 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377417.83 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 37741.78 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923403.166356 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339676.05 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 33967.60 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957370.771312 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305708.44 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 30570.84 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987941.615772 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275137.60 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 27513.76 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015455.375787 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247623.84 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 24762.38 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040217.759800 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222861.46 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 22286.15 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062503.905411 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200575.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 20057.53 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082561.436462 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180517.78 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 18051.78 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100613.214407 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162466.00 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 16246.60 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116859.814558 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146219.40 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 14621.94 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131481.754694 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131597.46 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 13159.75 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144641.500816 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118437.72 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 11843.77 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156485.272327 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106593.94 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 10659.39 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167144.666686 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95934.55 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 9593.45 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176738.121609 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86341.09 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 8634.11 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185372.231039 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77706.98 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 7770.70 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193142.929527 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69936.29 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 6993.63 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200136.558166 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62942.66 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 6294.27 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206430.823941 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56648.39 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 5664.84 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212095.663139 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50983.55 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 5098.36 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217194.018417 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45885.20 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 4588.52 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221782.538167 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41296.68 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 4129.67 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225912.205942 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37167.01 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3716.70 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229628.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33450.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233228.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29850.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236828.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26250.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240428.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22650.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244028.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19050.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247628.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15450.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251228.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11850.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254828.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8250.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258428.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4650.31 sec) 04:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2262028.906939 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050.31 sec) 04:19:00 [GPUGRID] [rr_sim] 2263079.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995254458.73G/439.78G) 04:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851660.13 04:19:00 [---] pick_jobs_to_run() start 04:19:00 [---] pick_jobs_to_run() end 04:19:00 [---] [rr_sim] end 04:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:19:00 [---] [cpu_sched_debug] preliminary job list: 04:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:19:00 [---] [cpu_sched_debug] final job list: 04:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:19:00 [---] [cpu_sched_debug] enforce_run_list: end 04:19:00 [---] choose_project(): 15540.000000 04:19:00 [iThena.Measurements] doing RPC to get work 04:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 420059.09 cpu 2164352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:19:00 [---] Got 0 tasks 04:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:20:00 [---] [rr_sim] doing sim: CPU sched 04:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995226868.38G/439.78G) 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2263016.48 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 226301.65 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226301.647903 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036714.83 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 203671.48 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429973.131017 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1833043.35 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 183304.33 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613277.465818 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649739.01 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 164973.90 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778251.367140 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484765.11 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 148476.51 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926727.878329 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336288.60 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 133628.86 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060356.738400 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202659.74 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 120265.97 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180622.712463 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082393.77 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 108239.38 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288862.089121 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974154.39 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 97415.44 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386277.528112 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876738.95 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 87673.90 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473951.423204 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789065.06 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 78906.51 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552857.928787 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710158.55 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 71015.86 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623873.783812 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639142.70 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 63914.27 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687788.053334 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575228.43 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 57522.84 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745310.895904 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517705.58 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 51770.56 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797081.454217 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465935.02 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 46593.50 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843674.956699 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419341.52 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 41934.15 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885609.108933 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377407.37 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 37740.74 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923349.845943 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339666.63 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 33966.66 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957316.509252 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305699.97 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 30570.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987886.506230 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275129.97 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 27513.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015399.503511 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247616.98 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 24761.70 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040161.201063 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222855.28 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 22285.53 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062446.728860 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200569.75 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 20056.98 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082503.703878 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180512.78 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 18051.28 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100554.981393 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162461.50 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 16246.15 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116801.131158 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146215.35 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 14621.53 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131422.665945 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131593.81 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 13159.38 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144582.047254 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118434.43 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 11843.44 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156425.490432 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106590.99 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 10659.10 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167084.589292 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95931.89 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 9593.19 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176677.778267 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86338.70 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 8633.87 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185311.648343 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77704.83 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 7770.48 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193082.131413 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69934.35 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 6993.43 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200075.566175 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62940.91 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 6294.09 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206369.657461 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56646.82 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 5664.68 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212034.339618 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50982.14 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 5098.21 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217132.553560 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45883.93 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 4588.39 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221720.946107 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41295.53 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 4129.55 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225850.499400 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37165.98 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3716.60 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229567.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33449.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233167.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29849.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236767.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26249.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240367.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22649.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243967.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19049.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247567.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15449.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251167.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11849.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254767.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8249.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258367.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4649.38 sec) 04:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261967.097364 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049.38 sec) 04:20:00 [GPUGRID] [rr_sim] 2263016.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995226868.38G/439.78G) 04:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851657.39 04:20:00 [---] pick_jobs_to_run() start 04:20:00 [---] pick_jobs_to_run() end 04:20:00 [---] [rr_sim] end 04:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:20:00 [---] [cpu_sched_debug] preliminary job list: 04:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:20:00 [---] [cpu_sched_debug] final job list: 04:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:20:00 [---] [cpu_sched_debug] enforce_run_list: end 04:20:00 [---] choose_project(): 15600.000000 04:20:00 [iThena.Measurements] doing RPC to get work 04:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419999.09 cpu 2164292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:20:00 [---] Got 0 tasks 04:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:21:00 [---] [rr_sim] doing sim: CPU sched 04:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995199278.02G/439.78G) 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262953.74 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 226295.37 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226295.374215 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036658.37 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 203665.84 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429961.211009 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832992.53 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 183299.25 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613260.464123 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649693.28 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 164969.33 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778229.791926 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484723.95 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 148472.40 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926702.186949 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336251.56 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 133625.16 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060327.342469 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202626.40 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 120262.64 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180589.982438 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082363.76 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 108236.38 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288826.358409 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974127.38 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 97412.74 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386239.096783 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876714.65 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 87671.46 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473910.561320 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789043.18 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 78904.32 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552814.879404 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710138.86 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 71013.89 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623828.765678 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639124.98 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 63912.50 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687741.263326 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575212.48 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 57521.25 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745262.511208 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517691.23 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 51769.12 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797031.634303 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465922.11 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 46592.21 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843623.845088 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419329.90 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 41932.99 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885556.834794 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377396.91 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 37739.69 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923296.525530 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339657.22 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 33965.72 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957262.247192 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305691.49 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 30569.15 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987831.396688 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275122.35 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 27512.23 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015343.631235 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247610.11 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 24761.01 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040104.642327 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222849.10 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 22284.91 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062389.552309 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200564.19 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 20056.42 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082445.971293 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180507.77 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 18050.78 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100496.748379 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162456.99 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 16245.70 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116742.447757 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146211.29 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 14621.13 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131363.577196 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131590.16 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 13159.02 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144522.593692 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118431.15 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 11843.11 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156365.708538 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106588.03 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 10658.80 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167024.511899 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95929.23 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 9592.92 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176617.434925 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86336.31 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 8633.63 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185251.065647 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77702.68 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 7770.27 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193021.333298 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69932.41 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 6993.24 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200014.574183 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62939.17 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 6293.92 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206308.490980 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56645.25 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 5664.53 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211973.016098 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50980.73 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 5098.07 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217071.088703 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45882.65 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 4588.27 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221659.354048 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41294.39 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 4129.44 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225788.792858 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37164.95 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3716.49 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229505.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33448.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233105.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29848.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236705.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26248.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240305.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22648.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243905.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19048.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247505.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15448.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251105.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11848.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254705.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8248.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258305.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4648.45 sec) 04:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261905.287788 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048.45 sec) 04:21:00 [GPUGRID] [rr_sim] 2262953.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995199278.02G/439.78G) 04:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851654.65 04:21:00 [---] pick_jobs_to_run() start 04:21:00 [---] pick_jobs_to_run() end 04:21:00 [---] [rr_sim] end 04:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:21:00 [---] [cpu_sched_debug] preliminary job list: 04:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:21:00 [---] [cpu_sched_debug] final job list: 04:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:21:00 [---] [cpu_sched_debug] enforce_run_list: end 04:21:00 [---] choose_project(): 15660.000000 04:21:00 [iThena.Measurements] doing RPC to get work 04:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419939.09 cpu 2164232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:21:00 [---] Got 0 tasks 04:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:22:00 [---] [rr_sim] doing sim: CPU sched 04:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995199278.02G/439.78G) 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262953.74 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 226295.37 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226295.374215 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036658.37 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 203665.84 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429961.211009 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832992.53 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 183299.25 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613260.464123 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649693.28 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 164969.33 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778229.791926 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484723.95 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 148472.40 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926702.186949 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336251.56 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 133625.16 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060327.342469 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202626.40 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 120262.64 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180589.982438 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082363.76 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 108236.38 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288826.358409 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974127.38 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 97412.74 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386239.096783 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876714.65 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 87671.46 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473910.561320 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789043.18 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 78904.32 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552814.879404 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710138.86 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 71013.89 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623828.765678 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639124.98 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 63912.50 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687741.263326 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575212.48 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 57521.25 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745262.511208 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517691.23 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 51769.12 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797031.634303 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465922.11 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 46592.21 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843623.845088 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419329.90 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 41932.99 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885556.834794 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377396.91 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 37739.69 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923296.525530 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339657.22 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 33965.72 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957262.247192 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305691.49 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 30569.15 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987831.396688 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275122.35 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 27512.23 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015343.631235 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247610.11 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 24761.01 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040104.642327 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222849.10 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 22284.91 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062389.552309 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200564.19 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 20056.42 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082445.971293 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180507.77 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 18050.78 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100496.748379 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162456.99 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 16245.70 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116742.447757 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146211.29 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 14621.13 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131363.577196 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131590.16 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 13159.02 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144522.593692 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118431.15 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 11843.11 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156365.708538 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106588.03 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 10658.80 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167024.511899 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95929.23 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 9592.92 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176617.434925 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86336.31 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 8633.63 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185251.065647 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77702.68 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 7770.27 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193021.333298 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69932.41 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 6993.24 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200014.574183 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62939.17 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 6293.92 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206308.490980 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56645.25 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 5664.53 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211973.016098 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50980.73 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 5098.07 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217071.088703 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45882.65 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 4588.27 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221659.354048 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41294.39 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 4129.44 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225788.792858 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37164.95 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3716.49 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229505.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33448.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233105.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29848.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236705.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26248.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240305.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22648.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243905.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19048.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247505.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15448.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251105.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11848.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254705.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8248.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258305.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4648.45 sec) 04:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261905.287788 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048.45 sec) 04:22:00 [GPUGRID] [rr_sim] 2262953.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995199278.02G/439.78G) 04:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851714.65 04:22:00 [---] pick_jobs_to_run() start 04:22:00 [---] pick_jobs_to_run() end 04:22:00 [---] [rr_sim] end 04:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:22:00 [---] [cpu_sched_debug] preliminary job list: 04:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:22:00 [---] [cpu_sched_debug] final job list: 04:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:22:00 [---] [cpu_sched_debug] enforce_run_list: end 04:22:00 [---] choose_project(): 15720.000000 04:22:00 [iThena.Measurements] doing RPC to get work 04:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419879.09 cpu 2164232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:22:00 [---] Got 0 tasks 04:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:23:00 [---] [rr_sim] doing sim: CPU sched 04:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995171687.66G/439.78G) 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262891.01 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 226289.10 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226289.100527 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036601.90 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 203660.19 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429949.291001 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832941.71 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 183294.17 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613243.462428 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649647.54 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 164964.75 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778208.216712 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484682.79 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 148468.28 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926676.495568 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336214.51 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 133621.45 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060297.946538 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202593.06 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 120259.31 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180557.252412 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082333.75 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 108233.38 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288790.627698 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974100.38 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 97410.04 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386200.665455 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876690.34 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 87669.03 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473869.699436 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789021.31 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 78902.13 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552771.830020 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710119.18 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 71011.92 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623783.747545 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639107.26 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 63910.73 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687694.473317 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575196.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 57519.65 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745214.126513 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517676.88 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 51767.69 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796981.814388 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465909.19 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 46590.92 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843572.733477 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419318.27 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 41931.83 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885504.560656 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377386.44 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 37738.64 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923243.205117 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339647.80 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 33964.78 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957207.985133 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305683.02 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 30568.30 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987776.287146 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275114.72 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 27511.47 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015287.758959 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247603.25 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 24760.32 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040048.083590 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222842.92 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 22284.29 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062332.375758 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200558.63 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 20055.86 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082388.238709 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180502.77 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 18050.28 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100438.515365 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162452.49 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 16245.25 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116683.764356 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146207.24 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 14620.72 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131304.488447 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131586.52 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 13158.65 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144463.140130 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118427.87 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 11842.79 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156305.926644 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106585.08 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 10658.51 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166964.434506 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95926.57 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 9592.66 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176557.091583 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86333.91 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 8633.39 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185190.482952 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77700.52 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 7770.05 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192960.535183 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69930.47 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 6993.05 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199953.582192 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62937.42 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 6293.74 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206247.324500 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56643.68 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 5664.37 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211911.692577 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50979.31 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 5097.93 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217009.623846 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45881.38 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 4588.14 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221597.761989 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41293.24 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 4129.32 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225727.086317 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37163.92 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3716.39 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229443.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33447.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233043.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29847.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236643.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26247.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240243.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22647.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243843.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19047.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247443.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15447.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251043.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11847.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254643.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8247.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258243.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4647.53 sec) 04:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261843.478212 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1047.53 sec) 04:23:00 [GPUGRID] [rr_sim] 2262891.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995171687.66G/439.78G) 04:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851711.92 04:23:00 [---] pick_jobs_to_run() start 04:23:00 [---] pick_jobs_to_run() end 04:23:00 [---] [rr_sim] end 04:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:23:00 [---] [cpu_sched_debug] preliminary job list: 04:23:00 [---] [cpu_sched_debug] final job list: 04:23:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:23:00 [---] [work_fetch] Request work fetch: CPUs idle 04:23:00 [---] [cpu_sched_debug] enforce_run_list: end 04:23:00 [---] choose_project(): 15780.000000 04:23:00 [iThena.Measurements] doing RPC to get work 04:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419819.09 cpu 2164172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:23:00 [---] Got 0 tasks 04:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:24:00 [---] [rr_sim] doing sim: CPU sched 04:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995171687.66G/439.78G) 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262891.01 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 226289.10 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226289.100527 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036601.90 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 203660.19 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429949.291001 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832941.71 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 183294.17 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613243.462428 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649647.54 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 164964.75 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778208.216712 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484682.79 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 148468.28 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926676.495568 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336214.51 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 133621.45 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060297.946538 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202593.06 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 120259.31 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180557.252412 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082333.75 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 108233.38 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288790.627698 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974100.38 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 97410.04 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386200.665455 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876690.34 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 87669.03 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473869.699436 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789021.31 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 78902.13 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552771.830020 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710119.18 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 71011.92 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623783.747545 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639107.26 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 63910.73 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687694.473317 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575196.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 57519.65 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745214.126513 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517676.88 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 51767.69 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796981.814388 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465909.19 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 46590.92 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843572.733477 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419318.27 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 41931.83 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885504.560656 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377386.44 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 37738.64 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923243.205117 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339647.80 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 33964.78 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957207.985133 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305683.02 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 30568.30 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987776.287146 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275114.72 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 27511.47 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015287.758959 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247603.25 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 24760.32 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040048.083590 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222842.92 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 22284.29 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062332.375758 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200558.63 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 20055.86 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082388.238709 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180502.77 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 18050.28 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100438.515365 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162452.49 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 16245.25 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116683.764356 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146207.24 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 14620.72 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131304.488447 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131586.52 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 13158.65 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144463.140130 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118427.87 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 11842.79 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156305.926644 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106585.08 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 10658.51 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166964.434506 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95926.57 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 9592.66 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176557.091583 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86333.91 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 8633.39 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185190.482952 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77700.52 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 7770.05 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192960.535183 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69930.47 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 6993.05 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199953.582192 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62937.42 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 6293.74 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206247.324500 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56643.68 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 5664.37 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211911.692577 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50979.31 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 5097.93 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217009.623846 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45881.38 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 4588.14 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221597.761989 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41293.24 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 4129.32 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225727.086317 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37163.92 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3716.39 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229443.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33447.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233043.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29847.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236643.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26247.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240243.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22647.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243843.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19047.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247443.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15447.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251043.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11847.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254643.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8247.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258243.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4647.53 sec) 04:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261843.478212 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1047.53 sec) 04:24:00 [GPUGRID] [rr_sim] 2262891.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995171687.66G/439.78G) 04:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851771.92 04:24:00 [---] pick_jobs_to_run() start 04:24:00 [---] pick_jobs_to_run() end 04:24:00 [---] [rr_sim] end 04:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:24:00 [---] [cpu_sched_debug] preliminary job list: 04:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:24:00 [---] [cpu_sched_debug] final job list: 04:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:24:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 995171687.66G 04:24:00 [---] [cpu_sched_debug] enforce_run_list: end 04:24:00 [---] choose_project(): 15840.000000 04:24:00 [iThena.Measurements] doing RPC to get work 04:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419759.09 cpu 2164172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:24:00 [---] Got 0 tasks 04:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:25:00 [---] [rr_sim] doing sim: CPU sched 04:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995144097.31G/439.78G) 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262828.27 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 226282.83 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226282.826839 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036545.44 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 203654.54 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429937.370994 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832890.90 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 183289.09 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613226.460733 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649601.81 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 164960.18 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778186.641499 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484641.63 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 148464.16 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926650.804188 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336177.46 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 133617.75 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060268.550608 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202559.72 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 120255.97 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180524.522386 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082303.75 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 108230.37 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288754.896986 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974073.37 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 97407.34 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386162.234126 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876666.03 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 87666.60 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473828.837552 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788999.43 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 78899.94 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552728.780636 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710099.49 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 71009.95 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623738.729411 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639089.54 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 63908.95 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687647.683309 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575180.59 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 57518.06 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745165.741817 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517662.53 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 51766.25 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796931.994474 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465896.27 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 46589.63 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843521.621865 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419306.65 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 41930.66 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885452.286518 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377375.98 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 37737.60 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923189.884705 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339638.38 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 33963.84 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957153.723073 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305674.55 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 30567.45 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987721.177605 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275107.09 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 27510.71 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015231.886683 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247596.38 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 24759.64 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039991.524853 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222836.74 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 22283.67 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062275.199207 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200553.07 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 20055.31 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082330.506125 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180497.76 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 18049.78 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100380.282351 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162447.99 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 16244.80 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116625.080955 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146203.19 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 14620.32 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131245.399698 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131582.87 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 13158.29 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144403.686567 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118424.58 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 11842.46 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156246.144749 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106582.12 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 10658.21 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166904.357113 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95923.91 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 9592.39 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176496.748241 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86331.52 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 8633.15 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185129.900256 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77698.37 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 7769.84 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192899.737069 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69928.53 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 6992.85 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199892.590201 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62935.68 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 6293.57 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206186.158019 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56642.11 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 5664.21 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211850.369056 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50977.90 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 5097.79 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216948.158990 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45880.11 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 4588.01 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221536.169929 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41292.10 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 4129.21 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225665.379775 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37162.89 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3716.29 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229381.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33446.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232981.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29846.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236581.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26246.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240181.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22646.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243781.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19046.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247381.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15446.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250981.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11846.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254581.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8246.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258181.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4646.60 sec) 04:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261781.668637 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1046.60 sec) 04:25:00 [GPUGRID] [rr_sim] 2262828.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995144097.31G/439.78G) 04:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851769.18 04:25:00 [---] pick_jobs_to_run() start 04:25:00 [---] pick_jobs_to_run() end 04:25:00 [---] [rr_sim] end 04:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:25:00 [---] [cpu_sched_debug] preliminary job list: 04:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:25:00 [---] [cpu_sched_debug] final job list: 04:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:25:00 [---] [cpu_sched_debug] enforce_run_list: end 04:25:00 [---] choose_project(): 15900.000000 04:25:00 [iThena.Measurements] doing RPC to get work 04:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419699.09 cpu 2164112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:25:00 [---] Got 0 tasks 04:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:26:00 [---] [rr_sim] doing sim: CPU sched 04:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995116506.95G/439.78G) 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262765.53 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 226276.55 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226276.553151 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036488.98 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 203648.90 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429925.450986 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832840.08 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 183284.01 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613209.459038 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649556.07 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 164955.61 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778165.066285 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484600.47 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 148460.05 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926625.112807 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336140.42 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 133614.04 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060239.154677 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202526.38 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 120252.64 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180491.792360 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082273.74 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 108227.37 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288719.166274 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974046.37 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 97404.64 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386123.802798 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876641.73 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 87664.17 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473787.975668 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788977.56 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 78897.76 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552685.731252 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710079.80 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 71007.98 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623693.711278 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639071.82 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 63907.18 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687600.893300 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575164.64 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 57516.46 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745117.357121 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517648.17 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 51764.82 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796882.174559 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465883.36 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 46588.34 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843470.510254 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419295.02 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 41929.50 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885400.012379 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377365.52 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 37736.55 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923136.564292 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339628.97 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 33962.90 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957099.461013 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305666.07 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 30566.61 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987666.068063 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275099.46 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 27509.95 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015176.014407 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247589.52 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 24758.95 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039934.966117 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222830.57 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 22283.06 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062218.022656 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200547.51 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 20054.75 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082272.773541 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180492.76 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 18049.28 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100322.049337 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162443.48 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 16244.35 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116566.397554 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146199.13 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 14619.91 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131186.310949 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131579.22 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 13157.92 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144344.233005 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118421.30 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 11842.13 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156186.362855 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106579.17 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 10657.92 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166844.279720 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95921.25 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 9592.13 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176436.404899 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86329.13 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 8632.91 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185069.317560 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77696.21 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 7769.62 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192838.938954 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69926.59 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 6992.66 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199831.598209 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62933.93 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 6293.39 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206124.991539 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56640.54 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 5664.05 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211789.045536 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50976.49 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 5097.65 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216886.694133 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45878.84 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 4587.88 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221474.577870 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41290.95 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 4129.10 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225603.673234 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37161.86 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3716.19 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229319.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33445.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232919.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29845.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236519.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26245.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240119.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22645.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243719.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19045.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247319.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15445.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250919.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11845.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254519.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8245.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258119.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4645.67 sec) 04:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261719.859061 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1045.67 sec) 04:26:00 [GPUGRID] [rr_sim] 2262765.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995116506.95G/439.78G) 04:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851766.44 04:26:00 [---] pick_jobs_to_run() start 04:26:00 [---] pick_jobs_to_run() end 04:26:00 [---] [rr_sim] end 04:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:26:00 [---] [cpu_sched_debug] preliminary job list: 04:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:26:00 [---] [cpu_sched_debug] final job list: 04:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:26:00 [---] [cpu_sched_debug] enforce_run_list: end 04:26:00 [---] choose_project(): 15960.000000 04:26:00 [iThena.Measurements] doing RPC to get work 04:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419639.09 cpu 2164052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2164052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2164052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:26:00 [---] Got 0 tasks 04:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:27:00 [---] [rr_sim] doing sim: CPU sched 04:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995088916.59G/439.78G) 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262702.79 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 226270.28 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226270.279462 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036432.52 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 203643.25 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429913.530979 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832789.26 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 183278.93 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613192.457343 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649510.34 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 164951.03 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778143.491071 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484559.30 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 148455.93 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926599.421426 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336103.37 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 133610.34 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060209.758746 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202493.04 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 120249.30 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180459.062334 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082243.73 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 108224.37 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288683.435563 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974019.36 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 97401.94 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386085.371469 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876617.42 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 87661.74 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473747.113784 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788955.68 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 78895.57 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552642.681868 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710060.11 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 71006.01 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623648.693144 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639054.10 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 63905.41 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687554.103292 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575148.69 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 57514.87 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745068.972425 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517633.82 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 51763.38 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796832.354645 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465870.44 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 46587.04 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843419.398643 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419283.40 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 41928.34 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885347.738241 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377355.06 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 37735.51 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923083.243879 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339619.55 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 33961.96 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957045.198954 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305657.60 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 30565.76 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987610.958521 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275091.84 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 27509.18 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015120.142131 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247582.65 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 24758.27 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039878.407380 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222824.39 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 22282.44 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062160.846105 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200541.95 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 20054.19 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082215.040957 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180487.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 18048.78 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100263.816323 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162438.98 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 16243.90 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116507.714153 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146195.08 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 14619.51 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131127.222200 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131575.57 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 13157.56 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144284.779443 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118418.02 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 11841.80 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156126.580961 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106576.21 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 10657.62 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166784.202327 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95918.59 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 9591.86 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176376.061557 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86326.73 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 8632.67 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185008.734864 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77694.06 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 7769.41 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192778.140840 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69924.65 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 6992.47 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199770.606218 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62932.19 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 6293.22 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206063.825059 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56638.97 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 5663.90 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211727.722015 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50975.07 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 5097.51 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216825.229276 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45877.57 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 4587.76 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221412.985811 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41289.81 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 4128.98 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225541.966692 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37160.83 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3716.08 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229258.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33444.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232858.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29844.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236458.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26244.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240058.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22644.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243658.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19044.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247258.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15444.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250858.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11844.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254458.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8244.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258058.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4644.75 sec) 04:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261658.049485 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1044.75 sec) 04:27:00 [GPUGRID] [rr_sim] 2262702.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995088916.59G/439.78G) 04:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851763.71 04:27:00 [---] pick_jobs_to_run() start 04:27:00 [---] pick_jobs_to_run() end 04:27:00 [---] [rr_sim] end 04:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:27:00 [---] [cpu_sched_debug] preliminary job list: 04:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:27:00 [---] [cpu_sched_debug] final job list: 04:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:27:00 [---] [cpu_sched_debug] enforce_run_list: end 04:27:00 [---] choose_project(): 16020.000000 04:27:00 [iThena.Measurements] doing RPC to get work 04:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419579.09 cpu 2163992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:27:00 [---] Got 0 tasks 04:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:28:00 [---] [rr_sim] doing sim: CPU sched 04:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995061326.24G/439.78G) 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262640.06 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 226264.01 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226264.005774 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036376.05 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 203637.61 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429901.610971 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832738.45 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 183273.84 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613175.455648 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649464.60 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 164946.46 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778121.915857 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484518.14 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 148451.81 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926573.730046 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336066.33 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 133606.63 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060180.362815 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202459.69 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 120245.97 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180426.332308 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082213.73 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 108221.37 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288647.704851 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973992.35 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 97399.24 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386046.940140 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876593.12 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 87659.31 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473706.251901 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788933.81 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 78893.38 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552599.632485 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710040.43 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 71004.04 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623603.675010 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639036.38 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 63903.64 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687507.313283 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575132.74 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 57513.27 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745020.587729 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517619.47 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 51761.95 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796782.534731 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465857.52 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 46585.75 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843368.287032 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419271.77 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 41927.18 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885295.464103 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377344.59 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 37734.46 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923029.923467 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339610.13 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 33961.01 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956990.936894 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305649.12 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 30564.91 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987555.848979 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275084.21 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 27508.42 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015064.269855 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247575.79 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 24757.58 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039821.848644 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222818.21 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 22281.82 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062103.669554 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200536.39 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 20053.64 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082157.308372 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180482.75 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 18048.27 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100205.583309 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162434.47 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 16243.45 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116449.030753 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146191.03 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 14619.10 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131068.133451 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131571.92 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 13157.19 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144225.325880 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118414.73 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 11841.47 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156066.799067 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106573.26 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 10657.33 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166724.124934 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95915.93 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 9591.59 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176315.718215 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86324.34 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 8632.43 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184948.152168 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77691.91 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 7769.19 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192717.342725 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69922.72 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 6992.27 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199709.614227 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62930.44 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 6293.04 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206002.658578 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56637.40 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 5663.74 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211666.398494 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50973.66 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 5097.37 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216763.764419 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45876.29 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 4587.63 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221351.393751 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41288.66 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 4128.87 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225480.260150 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37159.80 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3715.98 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229196.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33443.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232796.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29843.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236396.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26243.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239996.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22643.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243596.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19043.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247196.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15443.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250796.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11843.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254396.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8243.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257996.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4643.82 sec) 04:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261596.239910 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1043.82 sec) 04:28:00 [GPUGRID] [rr_sim] 2262640.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995061326.24G/439.78G) 04:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851760.97 04:28:00 [---] pick_jobs_to_run() start 04:28:00 [---] pick_jobs_to_run() end 04:28:00 [---] [rr_sim] end 04:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:28:00 [---] [cpu_sched_debug] preliminary job list: 04:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:28:00 [---] [cpu_sched_debug] final job list: 04:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:28:00 [---] [cpu_sched_debug] enforce_run_list: end 04:28:00 [---] choose_project(): 16080.000000 04:28:00 [iThena.Measurements] doing RPC to get work 04:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419519.09 cpu 2163932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:28:00 [---] Got 0 tasks 04:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:29:00 [---] [rr_sim] doing sim: CPU sched 04:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995033735.88G/439.78G) 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262577.32 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 226257.73 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226257.732086 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036319.59 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 203631.96 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429889.690963 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832687.63 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 183268.76 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613158.453953 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649418.87 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 164941.89 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778100.340644 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484476.98 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 148447.70 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926548.038665 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1336029.28 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 133602.93 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060150.966885 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202426.35 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 120242.64 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180393.602282 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082183.72 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 108218.37 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288611.974140 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973965.35 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 97396.53 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1386008.508812 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876568.81 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 87656.88 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473665.390017 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788911.93 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 78891.19 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552556.583101 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710020.74 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 71002.07 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623558.656877 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639018.66 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 63901.87 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687460.523275 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575116.80 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 57511.68 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744972.203033 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517605.12 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 51760.51 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796732.714816 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465844.61 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 46584.46 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843317.175420 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419260.15 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 41926.01 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885243.189964 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377334.13 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 37733.41 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922976.603054 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339600.72 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 33960.07 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956936.674834 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305640.65 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 30564.06 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987500.739437 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275076.58 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 27507.66 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015008.397579 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247568.92 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 24756.89 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039765.289907 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222812.03 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 22281.20 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062046.493002 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200530.83 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 20053.08 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082099.575788 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180477.75 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 18047.77 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100147.350295 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162429.97 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 16243.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116390.347352 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146186.97 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 14618.70 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131009.044702 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131568.28 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 13156.83 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144165.872318 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118411.45 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 11841.14 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156007.017172 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106570.30 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 10657.03 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166664.047541 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95913.27 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 9591.33 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176255.374873 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86321.95 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 8632.19 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184887.569472 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77689.75 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 7768.98 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192656.544610 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69920.78 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 6992.08 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199648.622235 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62928.70 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 6292.87 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205941.492098 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56635.83 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 5663.58 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211605.074974 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50972.25 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 5097.22 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216702.299562 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45875.02 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 4587.50 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221289.801692 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41287.52 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 4128.75 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225418.553609 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37158.77 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3715.88 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229134.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33442.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232734.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29842.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236334.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26242.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239934.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22642.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243534.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19042.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247134.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15442.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250734.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11842.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254334.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8242.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257934.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4642.89 sec) 04:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261534.430334 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1042.89 sec) 04:29:00 [GPUGRID] [rr_sim] 2262577.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995033735.88G/439.78G) 04:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851758.23 04:29:00 [---] pick_jobs_to_run() start 04:29:00 [---] pick_jobs_to_run() end 04:29:00 [---] [rr_sim] end 04:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:29:00 [---] [cpu_sched_debug] preliminary job list: 04:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:29:00 [---] [cpu_sched_debug] final job list: 04:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:29:00 [---] [cpu_sched_debug] enforce_run_list: end 04:29:00 [---] choose_project(): 16140.000000 04:29:00 [iThena.Measurements] doing RPC to get work 04:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419459.09 cpu 2163872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:29:00 [---] Got 0 tasks 04:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:30:00 [---] [rr_sim] doing sim: CPU sched 04:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (995006145.53G/439.78G) 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262514.58 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 226251.46 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226251.458398 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036263.13 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 203626.31 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429877.770956 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832636.81 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 183263.68 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613141.452258 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649373.13 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 164937.31 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778078.765430 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484435.82 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 148443.58 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926522.347285 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335992.24 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 133599.22 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060121.570954 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202393.01 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 120239.30 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180360.872256 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082153.71 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 108215.37 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288576.243428 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973938.34 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 97393.83 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385970.077483 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876544.51 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 87654.45 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473624.528133 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788890.06 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 78889.01 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552513.533717 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710001.05 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 71000.11 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623513.638743 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639000.95 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 63900.09 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687413.733267 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575100.85 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 57510.09 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744923.818338 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517590.77 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 51759.08 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796682.894902 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465831.69 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 46583.17 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843266.063809 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419248.52 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 41924.85 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885190.915826 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377323.67 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 37732.37 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922923.282641 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339591.30 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 33959.13 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956882.412775 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305632.17 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 30563.22 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987445.629895 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275068.95 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 27506.90 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014952.525303 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247562.06 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 24756.21 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039708.731171 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222805.85 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 22280.59 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061989.316451 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200525.27 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 20052.53 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082041.843204 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180472.74 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 18047.27 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100089.117281 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162425.47 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 16242.55 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116331.663951 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146182.92 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 14618.29 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130949.955954 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131564.63 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 13156.46 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144106.418756 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118408.17 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 11840.82 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155947.235278 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106567.35 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 10656.73 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166603.970148 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95910.61 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 9591.06 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176195.031531 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86319.55 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 8631.96 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184826.986776 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77687.60 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 7768.76 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192595.746496 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69918.84 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 6991.88 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199587.630244 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62926.95 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 6292.70 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205880.325617 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56634.26 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 5663.43 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211543.751453 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50970.83 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 5097.08 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216640.834706 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45873.75 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 4587.37 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221228.209633 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41286.37 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 4128.64 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225356.847067 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37157.74 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3715.77 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229072.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33441.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232672.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29841.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236272.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26241.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239872.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22641.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243472.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19041.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247072.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15441.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250672.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11841.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254272.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8241.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257872.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4641.96 sec) 04:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261472.620758 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1041.96 sec) 04:30:00 [GPUGRID] [rr_sim] 2262514.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (995006145.53G/439.78G) 04:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851755.50 04:30:00 [---] pick_jobs_to_run() start 04:30:00 [---] pick_jobs_to_run() end 04:30:00 [---] [rr_sim] end 04:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:30:00 [---] [cpu_sched_debug] preliminary job list: 04:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:30:00 [---] [cpu_sched_debug] final job list: 04:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:30:00 [---] [cpu_sched_debug] enforce_run_list: end 04:30:00 [---] choose_project(): 16200.000000 04:30:00 [iThena.Measurements] doing RPC to get work 04:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419399.09 cpu 2163812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:30:00 [---] Got 0 tasks 04:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:31:00 [---] [rr_sim] doing sim: CPU sched 04:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994978555.17G/439.78G) 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262451.85 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 226245.18 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226245.184710 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036206.66 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 203620.67 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429865.850948 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832586.00 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 183258.60 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613124.450563 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649327.40 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 164932.74 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778057.190216 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484394.66 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 148439.47 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926496.655904 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335955.19 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 133595.52 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060092.175023 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202359.67 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 120235.97 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180328.142230 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082123.70 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 108212.37 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288540.512717 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973911.33 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 97391.13 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385931.646155 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876520.20 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 87652.02 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473583.666249 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788868.18 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 78886.82 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552470.484333 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709981.36 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 70998.14 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623468.620610 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638983.23 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 63898.32 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687366.943258 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575084.90 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 57508.49 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744875.433642 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517576.41 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 51757.64 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796633.074987 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465818.77 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 46581.88 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843214.952198 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419236.89 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 41923.69 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885138.641688 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377313.21 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 37731.32 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922869.962228 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339581.88 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 33958.19 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956828.150715 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305623.70 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 30562.37 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987390.520353 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275061.33 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 27506.13 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014896.653027 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247555.19 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 24755.52 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039652.172434 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222799.67 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 22279.97 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061932.139900 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200519.71 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 20051.97 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081984.110620 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180467.74 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 18046.77 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100030.884267 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162420.96 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 16242.10 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116272.980550 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146178.87 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 14617.89 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130890.867205 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131560.98 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 13156.10 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144046.965194 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118404.88 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 11840.49 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155887.453384 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106564.39 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 10656.44 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166543.892755 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95907.95 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 9590.80 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176134.688189 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86317.16 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 8631.72 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184766.404080 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77685.44 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 7768.54 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192534.948381 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69916.90 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 6991.69 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199526.638253 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62925.21 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 6292.52 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205819.159137 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56632.69 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 5663.27 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211482.427933 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50969.42 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 5096.94 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216579.369849 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45872.48 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 4587.25 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221166.617574 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41285.23 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 4128.52 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225295.140526 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37156.71 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3715.67 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229010.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33441.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232610.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29841.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236210.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26241.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239810.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22641.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243410.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19041.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247010.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15441.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250610.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11841.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254210.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8241.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257810.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4641.04 sec) 04:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261410.811183 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1041.04 sec) 04:31:00 [GPUGRID] [rr_sim] 2262451.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994978555.17G/439.78G) 04:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851752.76 04:31:00 [---] pick_jobs_to_run() start 04:31:00 [---] pick_jobs_to_run() end 04:31:00 [---] [rr_sim] end 04:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:31:00 [---] [cpu_sched_debug] preliminary job list: 04:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:31:00 [---] [cpu_sched_debug] final job list: 04:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:31:00 [---] [cpu_sched_debug] enforce_run_list: end 04:31:00 [---] choose_project(): 16260.000000 04:31:00 [iThena.Measurements] doing RPC to get work 04:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419339.09 cpu 2163752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:31:00 [---] Got 0 tasks 04:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:32:00 [---] [rr_sim] doing sim: CPU sched 04:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994950964.81G/439.78G) 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262389.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 226238.91 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226238.911021 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036150.20 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 203615.02 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429853.930940 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832535.18 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 183253.52 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613107.448868 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649281.66 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 164928.17 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778035.615002 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484353.50 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 148435.35 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926470.964523 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335918.15 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 133591.81 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060062.779092 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202326.33 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 120232.63 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180295.412204 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082093.70 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 108209.37 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288504.782005 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973884.33 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 97388.43 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385893.214826 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876495.90 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 87649.59 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473542.804365 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788846.31 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 78884.63 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552427.434950 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709961.68 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 70996.17 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623423.602476 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638965.51 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 63896.55 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687320.153250 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575068.96 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 57506.90 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744827.048946 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517562.06 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 51756.21 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796583.255073 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465805.86 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 46580.59 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843163.840587 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419225.27 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 41922.53 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885086.367549 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377302.74 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 37730.27 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922816.641816 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339572.47 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 33957.25 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956773.888655 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305615.22 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 30561.52 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987335.410811 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275053.70 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 27505.37 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014840.780751 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247548.33 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 24754.83 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039595.613698 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222793.50 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 22279.35 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061874.963349 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200514.15 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 20051.41 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081926.378035 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180462.73 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 18046.27 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099972.651253 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162416.46 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 16241.65 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116214.297149 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146174.81 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 14617.48 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130831.778456 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131557.33 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 13155.73 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143987.511631 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118401.60 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 11840.16 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155827.671489 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106561.44 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 10656.14 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166483.815362 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95905.29 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 9590.53 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176074.344847 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86314.77 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 8631.48 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184705.821384 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77683.29 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 7768.33 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192474.150267 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69914.96 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 6991.50 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199465.646261 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62923.46 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 6292.35 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205757.992656 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56631.12 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 5663.11 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211421.104412 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50968.01 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 5096.80 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216517.904992 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45871.21 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 4587.12 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221105.025514 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41284.08 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 4128.41 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225233.433984 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37155.68 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3715.57 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228949.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33440.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232549.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29840.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236149.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26240.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239749.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22640.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243349.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19040.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246949.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15440.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250549.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11840.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254149.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8240.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257749.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4640.11 sec) 04:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261349.001607 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1040.11 sec) 04:32:00 [GPUGRID] [rr_sim] 2262389.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994950964.81G/439.78G) 04:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851750.02 04:32:00 [---] pick_jobs_to_run() start 04:32:00 [---] pick_jobs_to_run() end 04:32:00 [---] [rr_sim] end 04:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:32:00 [---] [cpu_sched_debug] preliminary job list: 04:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:32:00 [---] [cpu_sched_debug] final job list: 04:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:32:00 [---] [cpu_sched_debug] enforce_run_list: end 04:32:00 [---] choose_project(): 16320.000000 04:32:00 [iThena.Measurements] doing RPC to get work 04:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419279.09 cpu 2163692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:32:00 [---] Got 0 tasks 04:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:33:00 [---] [rr_sim] doing sim: CPU sched 04:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994923374.46G/439.78G) 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262326.37 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 226232.64 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226232.637333 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036093.74 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 203609.37 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429842.010933 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832484.36 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 183248.44 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613090.447173 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649235.93 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 164923.59 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 778014.039788 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484312.33 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 148431.23 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926445.273143 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335881.10 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 133588.11 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060033.383162 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202292.99 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 120229.30 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180262.682178 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082063.69 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 108206.37 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288469.051294 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973857.32 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 97385.73 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385854.783497 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876471.59 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 87647.16 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473501.942481 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788824.43 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 78882.44 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552384.385566 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709941.99 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 70994.20 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623378.584342 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638947.79 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 63894.78 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687273.363241 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575053.01 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 57505.30 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744778.664250 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517547.71 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 51754.77 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796533.435158 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465792.94 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 46579.29 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843112.728975 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419213.64 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 41921.36 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885034.093411 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377292.28 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 37729.23 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922763.321403 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339563.05 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 33956.31 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956719.626596 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305606.75 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 30560.67 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987280.301269 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275046.07 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 27504.61 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014784.908475 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247541.46 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 24754.15 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039539.054961 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222787.32 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 22278.73 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061817.786798 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200508.59 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 20050.86 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081868.645451 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180457.73 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 18045.77 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099914.418239 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162411.96 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 16241.20 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116155.613748 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146170.76 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 14617.08 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130772.689707 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131553.68 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 13155.37 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143928.058069 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118398.32 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 11839.83 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155767.889595 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106558.48 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 10655.85 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166423.737969 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95902.64 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 9590.26 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176014.001505 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86312.37 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 8631.24 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184645.238688 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77681.13 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 7768.11 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192413.352152 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69913.02 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 6991.30 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199404.654270 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62921.72 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 6292.17 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205696.826176 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56629.55 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 5662.95 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211359.780891 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50966.59 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 5096.66 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216456.440135 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45869.93 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 4586.99 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221043.433455 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41282.94 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 4128.29 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225171.727442 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37154.65 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3715.46 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228887.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33439.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232487.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29839.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236087.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26239.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239687.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22639.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243287.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19039.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246887.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15439.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250487.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11839.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254087.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8239.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257687.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4639.18 sec) 04:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261287.192031 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1039.18 sec) 04:33:00 [GPUGRID] [rr_sim] 2262326.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994923374.46G/439.78G) 04:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851747.29 04:33:00 [---] pick_jobs_to_run() start 04:33:00 [---] pick_jobs_to_run() end 04:33:00 [---] [rr_sim] end 04:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:33:00 [---] [cpu_sched_debug] preliminary job list: 04:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:33:00 [---] [cpu_sched_debug] final job list: 04:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:33:00 [---] [cpu_sched_debug] enforce_run_list: end 04:33:00 [---] choose_project(): 16380.000000 04:33:00 [iThena.Measurements] doing RPC to get work 04:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 592019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419219.09 cpu 2163632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:33:00 [---] Got 0 tasks 04:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:34:00 [---] [rr_sim] doing sim: CPU sched 04:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994895784.10G/439.78G) 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262263.64 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 226226.36 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226226.363645 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2036037.27 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 203603.73 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429830.090925 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832433.55 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 183243.35 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613073.445478 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649190.19 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 164919.02 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777992.464575 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484271.17 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 148427.12 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926419.581762 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335844.05 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 133584.41 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1060003.987231 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202259.65 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 120225.96 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180229.952153 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082033.68 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 108203.37 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288433.320582 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973830.32 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 97383.03 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385816.352169 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876447.28 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 87644.73 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473461.080597 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788802.56 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 78880.26 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552341.336182 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709922.30 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 70992.23 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623333.566209 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638930.07 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 63893.01 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687226.573233 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575037.06 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 57503.71 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744730.279554 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517533.36 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 51753.34 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796483.615244 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465780.02 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 46578.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843061.617364 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419202.02 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 41920.20 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884981.819273 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377281.82 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 37728.18 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922710.000990 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339553.64 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 33955.36 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956665.364536 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305598.27 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 30559.83 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987225.191727 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275038.44 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 27503.84 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014729.036199 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247534.60 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 24753.46 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039482.496224 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222781.14 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 22278.11 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061760.610247 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200503.03 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 20050.30 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081810.912867 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180452.72 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 18045.27 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099856.185225 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162407.45 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 16240.75 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116096.930348 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146166.71 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 14616.67 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130713.600958 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131550.04 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 13155.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143868.604507 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118395.03 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 11839.50 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155708.107701 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106555.53 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 10655.55 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166363.660576 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95899.98 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 9590.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175953.658163 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86309.98 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 8631.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184584.655992 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77678.98 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 7767.90 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192352.554037 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69911.08 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 6991.11 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199343.662278 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62919.97 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 6292.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205635.659695 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56627.98 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 5662.80 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211298.457371 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50965.18 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 5096.52 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216394.975279 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45868.66 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 4586.87 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220981.841396 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41281.80 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 4128.18 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225110.020901 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37153.62 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3715.36 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228825.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33438.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232425.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29838.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236025.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26238.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239625.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22638.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243225.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19038.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246825.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15438.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250425.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11838.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254025.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8238.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257625.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4638.25 sec) 04:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261225.382456 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1038.25 sec) 04:34:00 [GPUGRID] [rr_sim] 2262263.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994895784.10G/439.78G) 04:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851744.55 04:34:00 [---] pick_jobs_to_run() start 04:34:00 [---] pick_jobs_to_run() end 04:34:00 [---] [rr_sim] end 04:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:34:00 [---] [cpu_sched_debug] preliminary job list: 04:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:34:00 [---] [cpu_sched_debug] final job list: 04:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:34:00 [---] [cpu_sched_debug] enforce_run_list: end 04:34:00 [---] choose_project(): 16440.000000 04:34:00 [iThena.Measurements] doing RPC to get work 04:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419159.09 cpu 2163572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:34:00 [---] Got 0 tasks 04:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:35:00 [---] [rr_sim] doing sim: CPU sched 04:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994868193.74G/439.78G) 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262200.90 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 226220.09 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226220.089957 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035980.81 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 203598.08 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429818.170918 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832382.73 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 183238.27 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613056.443783 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649144.46 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 164914.45 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777970.889361 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484230.01 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 148423.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926393.890381 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335807.01 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 133580.70 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059974.591300 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202226.31 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 120222.63 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180197.222127 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1082003.68 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 108200.37 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288397.589871 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973803.31 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 97380.33 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385777.920840 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876422.98 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 87642.30 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473420.218713 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788780.68 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 78878.07 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552298.286798 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709902.61 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 70990.26 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623288.548075 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638912.35 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 63891.24 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687179.783224 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575021.12 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 57502.11 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744681.894858 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517519.00 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 51751.90 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796433.795329 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465767.10 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 46576.71 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843010.505753 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419190.39 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 41919.04 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884929.545134 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377271.35 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 37727.14 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922656.680578 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339544.22 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 33954.42 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956611.102476 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305589.80 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 30558.98 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987170.082185 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275030.82 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 27503.08 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014673.163924 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247527.74 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 24752.77 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039425.937488 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222774.96 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 22277.50 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061703.433696 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200497.47 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 20049.75 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081753.180283 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180447.72 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 18044.77 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099797.952211 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162402.95 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 16240.29 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116038.246947 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146162.65 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 14616.27 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130654.512209 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131546.39 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 13154.64 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143809.150944 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118391.75 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 11839.17 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155648.325807 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106552.57 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 10655.26 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166303.583183 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95897.32 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 9589.73 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175893.314821 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86307.58 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 8630.76 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184524.073296 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77676.83 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 7767.68 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192291.755923 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69909.14 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 6990.91 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199282.670287 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62918.23 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 6291.82 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205574.493215 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56626.41 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 5662.64 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211237.133850 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50963.77 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 5096.38 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216333.510422 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45867.39 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 4586.74 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220920.249336 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41280.65 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 4128.07 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225048.314359 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37152.59 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3715.26 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228763.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33437.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232363.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29837.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235963.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26237.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239563.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22637.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243163.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19037.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246763.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15437.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250363.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11837.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253963.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8237.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257563.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4637.33 sec) 04:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261163.572880 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1037.33 sec) 04:35:00 [GPUGRID] [rr_sim] 2262200.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994868193.74G/439.78G) 04:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851741.81 04:35:00 [---] pick_jobs_to_run() start 04:35:00 [---] pick_jobs_to_run() end 04:35:00 [---] [rr_sim] end 04:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:35:00 [---] [cpu_sched_debug] preliminary job list: 04:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:35:00 [---] [cpu_sched_debug] final job list: 04:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:35:00 [---] [cpu_sched_debug] enforce_run_list: end 04:35:00 [---] choose_project(): 16500.000000 04:35:00 [iThena.Measurements] doing RPC to get work 04:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419099.09 cpu 2163512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:35:00 [---] Got 0 tasks 04:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:36:00 [---] [rr_sim] doing sim: CPU sched 04:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994840603.39G/439.78G) 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262138.16 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 226213.82 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226213.816268 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035924.35 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 203592.43 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429806.250910 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832331.91 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 183233.19 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613039.442087 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649098.72 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 164909.87 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777949.314147 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484188.85 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 148418.88 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926368.199001 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335769.96 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 133577.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059945.195369 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202192.97 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 120219.30 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180164.492101 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081973.67 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 108197.37 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288361.859159 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973776.30 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 97377.63 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385739.489512 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876398.67 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 87639.87 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473379.356829 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788758.81 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 78875.88 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552255.237414 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709882.93 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 70988.29 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623243.529941 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638894.63 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 63889.46 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687132.993216 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575005.17 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 57500.52 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744633.510163 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517504.65 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 51750.47 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796383.975415 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465754.19 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 46575.42 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842959.394142 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419178.77 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 41917.88 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884877.270996 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377260.89 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 37726.09 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922603.360165 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339534.80 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 33953.48 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956556.840417 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305581.32 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 30558.13 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987114.972644 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275023.19 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 27502.32 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014617.291648 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247520.87 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 24752.09 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039369.378751 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222768.78 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 22276.88 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061646.257145 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200491.91 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 20049.19 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081695.447699 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180442.71 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 18044.27 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099739.719197 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162398.44 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 16239.84 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115979.563546 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146158.60 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 14615.86 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130595.423460 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131542.74 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 13154.27 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143749.697382 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118388.47 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 11838.85 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155588.543912 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106549.62 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 10654.96 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166243.505790 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95894.66 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 9589.47 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175832.971479 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86305.19 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 8630.52 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184463.490600 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77674.67 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 7767.47 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192230.957808 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69907.20 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 6990.72 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199221.678296 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62916.48 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 6291.65 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205513.326735 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56624.84 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 5662.48 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211175.810330 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50962.35 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 5096.24 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216272.045565 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45866.12 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 4586.61 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220858.657277 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41279.51 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 4127.95 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224986.607818 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37151.55 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3715.16 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228701.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33436.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232301.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29836.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235901.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26236.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239501.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22636.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243101.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19036.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246701.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15436.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250301.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11836.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253901.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8236.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257501.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4636.40 sec) 04:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261101.763304 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1036.40 sec) 04:36:00 [GPUGRID] [rr_sim] 2262138.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994840603.39G/439.78G) 04:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851739.08 04:36:00 [---] pick_jobs_to_run() start 04:36:00 [---] pick_jobs_to_run() end 04:36:00 [---] [rr_sim] end 04:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:36:00 [---] [cpu_sched_debug] preliminary job list: 04:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:36:00 [---] [cpu_sched_debug] final job list: 04:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:36:00 [---] [cpu_sched_debug] enforce_run_list: end 04:36:00 [---] choose_project(): 16560.000000 04:36:00 [iThena.Measurements] doing RPC to get work 04:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 419039.09 cpu 2163452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:36:00 [---] Got 0 tasks 04:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:37:00 [---] [rr_sim] doing sim: CPU sched 04:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994813013.03G/439.78G) 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262075.43 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 226207.54 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226207.542580 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035867.88 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 203586.79 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429794.330902 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832281.09 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 183228.11 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613022.440392 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649052.99 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 164905.30 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777927.738933 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484147.69 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 148414.77 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926342.507620 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335732.92 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 133573.29 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059915.799438 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202159.63 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 120215.96 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180131.762075 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081943.66 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 108194.37 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288326.128448 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973749.30 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 97374.93 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385701.058183 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876374.37 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 87637.44 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473338.494945 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788736.93 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 78873.69 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552212.188031 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709863.24 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 70986.32 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623198.511808 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638876.91 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 63887.69 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687086.203207 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574989.22 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 57498.92 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744585.125467 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517490.30 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 51749.03 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796334.155500 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465741.27 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 46574.13 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842908.282530 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419167.14 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 41916.71 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884824.996858 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377250.43 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 37725.04 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922550.039752 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339525.39 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 33952.54 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956502.578357 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305572.85 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 30557.28 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987059.863102 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275015.56 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 27501.56 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014561.419372 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247514.01 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 24751.40 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039312.820015 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222762.61 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 22276.26 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061589.080593 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200486.35 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 20048.63 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081637.715114 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180437.71 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 18043.77 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099681.486183 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162393.94 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 16239.39 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115920.880145 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146154.55 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 14615.45 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130536.334711 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131539.09 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 13153.91 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143690.243820 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118385.18 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 11838.52 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155528.762018 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106546.66 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 10654.67 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166183.428397 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95892.00 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 9589.20 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175772.628137 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86302.80 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 8630.28 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184402.907904 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77672.52 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 7767.25 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192170.159693 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69905.27 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 6990.53 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199160.686304 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62914.74 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 6291.47 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205452.160254 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56623.27 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 5662.33 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211114.486809 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50960.94 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 5096.09 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216210.580708 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45864.85 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 4586.48 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220797.065218 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41278.36 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 4127.84 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224924.901276 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37150.52 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3715.05 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228639.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33435.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232239.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29835.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235839.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26235.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239439.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22635.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243039.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19035.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246639.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15435.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250239.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11835.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253839.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8235.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257439.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4635.47 sec) 04:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2261039.953729 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1035.47 sec) 04:37:00 [GPUGRID] [rr_sim] 2262075.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994813013.03G/439.78G) 04:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851736.34 04:37:00 [---] pick_jobs_to_run() start 04:37:00 [---] pick_jobs_to_run() end 04:37:00 [---] [rr_sim] end 04:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:37:00 [---] [cpu_sched_debug] preliminary job list: 04:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:37:00 [---] [cpu_sched_debug] final job list: 04:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:37:00 [---] [cpu_sched_debug] enforce_run_list: end 04:37:00 [---] choose_project(): 16620.000000 04:37:00 [iThena.Measurements] doing RPC to get work 04:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418979.09 cpu 2163392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:37:00 [---] Got 0 tasks 04:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:38:00 [---] [rr_sim] doing sim: CPU sched 04:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994785422.68G/439.78G) 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2262012.69 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 226201.27 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226201.268892 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035811.42 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 203581.14 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429782.410895 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832230.28 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 183223.03 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 613005.438697 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1649007.25 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 164900.73 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777906.163720 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484106.53 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 148410.65 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926316.816240 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335695.87 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 133569.59 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059886.403508 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202126.29 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 120212.63 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180099.032049 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081913.66 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 108191.37 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288290.397736 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973722.29 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 97372.23 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385662.626854 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876350.06 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 87635.01 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473297.633061 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788715.06 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 78871.51 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552169.138647 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709843.55 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 70984.36 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623153.493674 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638859.20 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 63885.92 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1687039.413199 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574973.28 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 57497.33 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744536.740771 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517475.95 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 51747.59 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796284.335586 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465728.35 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 46572.84 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842857.170919 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419155.52 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 41915.55 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884772.722719 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377239.97 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 37724.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922496.719339 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339515.97 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 33951.60 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956448.316297 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305564.37 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 30556.44 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987004.753560 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275007.94 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 27500.79 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014505.547096 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247507.14 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 24750.71 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039256.261278 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222756.43 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 22275.64 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061531.904042 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200480.78 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 20048.08 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081579.982530 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180432.71 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 18043.27 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099623.253169 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162389.44 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 16238.94 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115862.196744 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146150.49 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 14615.05 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130477.245962 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131535.44 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 13153.54 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143630.790258 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118381.90 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 11838.19 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155468.980124 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106543.71 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 10654.37 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166123.351003 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95889.34 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 9588.93 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175712.284795 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86300.40 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 8630.04 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184342.325208 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77670.36 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 7767.04 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192109.361579 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69903.33 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 6990.33 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199099.694313 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62912.99 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 6291.30 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205390.993774 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56621.70 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 5662.17 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211053.163288 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50959.53 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 5095.95 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216149.115851 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45863.57 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 4586.36 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220735.473158 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41277.22 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 4127.72 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224863.194735 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37149.49 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3714.95 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228578.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33434.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232178.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29834.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235778.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26234.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239378.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22634.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242978.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19034.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246578.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15434.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250178.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11834.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253778.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8234.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257378.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4634.54 sec) 04:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260978.144153 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1034.54 sec) 04:38:00 [GPUGRID] [rr_sim] 2262012.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994785422.68G/439.78G) 04:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851733.60 04:38:00 [---] pick_jobs_to_run() start 04:38:00 [---] pick_jobs_to_run() end 04:38:00 [---] [rr_sim] end 04:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:38:00 [---] [cpu_sched_debug] preliminary job list: 04:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:38:00 [---] [cpu_sched_debug] final job list: 04:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:38:00 [---] [cpu_sched_debug] enforce_run_list: end 04:38:00 [---] choose_project(): 16680.000000 04:38:00 [iThena.Measurements] doing RPC to get work 04:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418919.09 cpu 2163332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:38:00 [---] Got 0 tasks 04:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:39:00 [---] [rr_sim] doing sim: CPU sched 04:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994757832.32G/439.78G) 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261949.95 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 226195.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226194.995204 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035754.96 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 203575.50 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429770.490887 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832179.46 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 183217.95 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612988.437002 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648961.52 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 164896.15 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777884.588506 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484065.36 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 148406.54 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926291.124859 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335658.83 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 133565.88 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059857.007577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202092.94 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 120209.29 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180066.302023 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081883.65 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 108188.37 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288254.667024 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973695.29 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 97369.53 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385624.195526 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876325.76 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 87632.58 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473256.771177 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788693.18 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 78869.32 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552126.089263 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709823.86 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 70982.39 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623108.475541 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638841.48 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 63884.15 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686992.623190 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574957.33 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 57495.73 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744488.356075 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517461.60 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 51746.16 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796234.515671 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465715.44 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 46571.54 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842806.059308 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419143.89 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 41914.39 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884720.448581 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377229.50 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 37722.95 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922443.398927 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339506.55 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 33950.66 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956394.054238 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305555.90 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 30555.59 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986949.644018 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275000.31 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 27500.03 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014449.674820 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247500.28 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 24750.03 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039199.702542 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222750.25 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 22275.02 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061474.727491 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200475.22 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 20047.52 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081522.249946 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180427.70 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 18042.77 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099565.020155 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162384.93 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 16238.49 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115803.513343 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146146.44 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 14614.64 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130418.157213 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131531.79 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 13153.18 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143571.336695 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118378.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 11837.86 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155409.198230 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106540.75 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 10654.08 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166063.273610 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95886.68 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 9588.67 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175651.941453 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86298.01 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 8629.80 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184281.742512 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77668.21 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 7766.82 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192048.563464 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69901.39 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 6990.14 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199038.702322 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62911.25 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 6291.12 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205329.827293 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56620.12 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 5662.01 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210991.839768 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50958.11 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 5095.81 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216087.650995 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45862.30 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 4586.23 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220673.881099 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41276.07 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 4127.61 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224801.488193 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37148.46 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3714.85 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228516.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33433.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232116.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29833.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235716.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26233.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239316.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22633.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242916.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19033.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246516.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15433.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250116.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11833.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253716.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8233.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257316.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4633.62 sec) 04:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260916.334577 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1033.62 sec) 04:39:00 [GPUGRID] [rr_sim] 2261949.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994757832.32G/439.78G) 04:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851730.86 04:39:00 [---] pick_jobs_to_run() start 04:39:00 [---] pick_jobs_to_run() end 04:39:00 [---] [rr_sim] end 04:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:39:00 [---] [cpu_sched_debug] preliminary job list: 04:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:39:00 [---] [cpu_sched_debug] final job list: 04:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:39:00 [---] [cpu_sched_debug] enforce_run_list: end 04:39:00 [---] choose_project(): 16740.000000 04:39:00 [iThena.Measurements] doing RPC to get work 04:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418859.09 cpu 2163272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:39:00 [---] Got 0 tasks 04:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:40:00 [---] [rr_sim] doing sim: CPU sched 04:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994730241.96G/439.78G) 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261887.22 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 226188.72 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226188.721516 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035698.49 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 203569.85 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429758.570880 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832128.64 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 183212.86 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612971.435307 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648915.78 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 164891.58 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777863.013292 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1484024.20 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 148402.42 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926265.433478 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335621.78 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 133562.18 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059827.611646 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202059.60 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 120205.96 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180033.571997 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081853.64 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 108185.36 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288218.936313 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973668.28 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 97366.83 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385585.764197 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876301.45 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 87630.15 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473215.909293 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788671.31 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 78867.13 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552083.039879 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709804.18 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 70980.42 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623063.457407 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638823.76 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 63882.38 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686945.833182 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574941.38 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 57494.14 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744439.971379 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517447.24 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 51744.72 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796184.695757 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465702.52 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 46570.25 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842754.947697 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419132.27 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 41913.23 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884668.174443 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377219.04 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 37721.90 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922390.078514 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339497.14 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 33949.71 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956339.792178 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305547.42 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 30554.74 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986894.534476 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274992.68 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 27499.27 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014393.802544 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247493.41 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 24749.34 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039143.143805 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222744.07 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 22274.41 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061417.550940 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200469.66 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 20046.97 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081464.517362 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180422.70 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 18042.27 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099506.787141 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162380.43 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 16238.04 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115744.829943 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146142.39 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 14614.24 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130359.068464 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131528.15 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 13152.81 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143511.883133 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118375.33 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 11837.53 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155349.416335 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106537.80 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 10653.78 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166003.196217 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95884.02 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 9588.40 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175591.598111 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86295.62 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 8629.56 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184221.159816 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77666.06 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 7766.61 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191987.765350 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69899.45 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 6989.94 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198977.710330 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62909.50 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 6290.95 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205268.660813 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56618.55 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 5661.86 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210930.516247 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50956.70 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 5095.67 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216026.186138 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45861.03 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 4586.10 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220612.289040 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41274.93 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 4127.49 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224739.781651 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37147.43 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3714.74 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228454.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33432.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232054.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29832.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235654.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26232.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239254.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22632.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242854.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19032.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246454.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15432.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250054.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11832.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253654.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8232.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257254.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4632.69 sec) 04:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260854.525002 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1032.69 sec) 04:40:00 [GPUGRID] [rr_sim] 2261887.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994730241.96G/439.78G) 04:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851728.13 04:40:00 [---] pick_jobs_to_run() start 04:40:00 [---] pick_jobs_to_run() end 04:40:00 [---] [rr_sim] end 04:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:40:00 [---] [cpu_sched_debug] preliminary job list: 04:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:40:00 [---] [cpu_sched_debug] final job list: 04:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:40:00 [---] [cpu_sched_debug] enforce_run_list: end 04:40:00 [---] choose_project(): 16800.000000 04:40:00 [iThena.Measurements] doing RPC to get work 04:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418799.09 cpu 2163212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:40:00 [---] Got 0 tasks 04:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:41:00 [---] [rr_sim] doing sim: CPU sched 04:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994702651.61G/439.78G) 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261824.48 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 226182.45 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226182.447827 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035642.03 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 203564.20 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429746.650872 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832077.83 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 183207.78 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612954.433612 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648870.04 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 164887.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777841.438078 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483983.04 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 148398.30 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926239.742098 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335584.74 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 133558.47 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059798.215715 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1202026.26 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 120202.63 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1180000.841971 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081823.64 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 108182.36 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288183.205601 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973641.27 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 97364.13 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385547.332869 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876277.15 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 87627.71 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473175.047409 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788649.43 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 78864.94 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1552039.990496 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709784.49 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 70978.45 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1623018.439273 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638806.04 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 63880.60 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686899.043173 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574925.44 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 57492.54 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744391.586683 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517432.89 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 51743.29 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796134.875842 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465689.60 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 46568.96 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842703.836086 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419120.64 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 41912.06 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884615.900304 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377208.58 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 37720.86 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922336.758101 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339487.72 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 33948.77 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956285.530118 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305538.95 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 30553.89 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986839.424934 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274985.05 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 27498.51 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014337.930268 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247486.55 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 24748.65 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039086.585068 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222737.89 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 22273.79 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061360.374389 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200464.10 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 20046.41 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081406.784777 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180417.69 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 18041.77 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099448.554127 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162375.92 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 16237.59 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115686.146542 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146138.33 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 14613.83 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130299.979715 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131524.50 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 13152.45 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143452.429571 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118372.05 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 11837.20 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155289.634441 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106534.84 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 10653.48 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165943.118824 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95881.36 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 9588.14 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175531.254769 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86293.22 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 8629.32 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184160.577120 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77663.90 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 7766.39 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191926.967235 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69897.51 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 6989.75 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198916.718339 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62907.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 6290.78 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205207.494332 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56616.98 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 5661.70 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210869.192726 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50955.29 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 5095.53 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215964.721281 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45859.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 4585.98 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220550.696980 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41273.78 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 4127.38 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224678.075110 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37146.40 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3714.64 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228392.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33431.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231992.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29831.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235592.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26231.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239192.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22631.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242792.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19031.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246392.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15431.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249992.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11831.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253592.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8231.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257192.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4631.76 sec) 04:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260792.715426 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1031.76 sec) 04:41:00 [GPUGRID] [rr_sim] 2261824.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994702651.61G/439.78G) 04:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851725.39 04:41:00 [---] pick_jobs_to_run() start 04:41:00 [---] pick_jobs_to_run() end 04:41:00 [---] [rr_sim] end 04:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:41:00 [---] [cpu_sched_debug] preliminary job list: 04:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:41:00 [---] [cpu_sched_debug] final job list: 04:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:41:00 [---] [cpu_sched_debug] enforce_run_list: end 04:41:00 [---] choose_project(): 16860.000000 04:41:00 [iThena.Measurements] doing RPC to get work 04:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418739.09 cpu 2163152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:41:00 [---] Got 0 tasks 04:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:42:00 [---] [rr_sim] doing sim: CPU sched 04:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994675061.25G/439.78G) 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261761.74 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 226176.17 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226176.174139 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035585.57 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 203558.56 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429734.730864 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1832027.01 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 183202.70 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612937.431917 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648824.31 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 164882.43 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777819.862864 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483941.88 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 148394.19 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926214.050717 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335547.69 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 133554.77 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059768.819785 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201992.92 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 120199.29 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179968.111945 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081793.63 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 108179.36 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288147.474890 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973614.27 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 97361.43 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385508.901540 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876252.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 87625.28 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473134.185525 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788627.56 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 78862.76 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551996.941112 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709764.80 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 70976.48 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622973.421140 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638788.32 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 63878.83 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686852.253165 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574909.49 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 57490.95 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744343.201988 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517418.54 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 51741.85 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796085.055928 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465676.69 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 46567.67 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842652.724474 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419109.02 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 41910.90 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884563.626166 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377198.12 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 37719.81 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922283.437689 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339478.30 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 33947.83 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956231.268059 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305530.47 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 30553.05 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986784.315392 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274977.43 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 27497.74 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014282.057992 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247479.68 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 24747.97 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039030.026332 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222731.72 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 22273.17 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061303.197838 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200458.54 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 20045.85 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081349.052193 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180412.69 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 18041.27 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099390.321113 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162371.42 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 16237.14 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115627.463141 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146134.28 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 14613.43 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130240.890966 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131520.85 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 13152.09 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143392.976008 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118368.77 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 11836.88 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155229.852547 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106531.89 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 10653.19 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165883.041431 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95878.70 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 9587.87 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175470.911427 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86290.83 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 8629.08 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184099.994424 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77661.75 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 7766.17 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191866.169120 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69895.57 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 6989.56 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198855.726348 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62906.02 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 6290.60 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205146.327852 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56615.41 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 5661.54 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210807.869206 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50953.87 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 5095.39 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215903.256424 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45858.48 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 4585.85 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220489.104921 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41272.64 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 4127.26 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224616.368568 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37145.37 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3714.54 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228330.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33430.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231930.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29830.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235530.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26230.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239130.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22630.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242730.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19030.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246330.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15430.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249930.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11830.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253530.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8230.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257130.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4630.84 sec) 04:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260730.905850 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1030.84 sec) 04:42:00 [GPUGRID] [rr_sim] 2261761.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994675061.25G/439.78G) 04:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851722.65 04:42:00 [---] pick_jobs_to_run() start 04:42:00 [---] pick_jobs_to_run() end 04:42:00 [---] [rr_sim] end 04:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:42:00 [---] [cpu_sched_debug] preliminary job list: 04:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:42:00 [---] [cpu_sched_debug] final job list: 04:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:42:00 [---] [cpu_sched_debug] enforce_run_list: end 04:42:00 [---] choose_project(): 16920.000000 04:42:00 [iThena.Measurements] doing RPC to get work 04:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418679.09 cpu 2163092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:42:00 [---] Got 0 tasks 04:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:43:00 [---] [rr_sim] doing sim: CPU sched 04:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994647470.90G/439.78G) 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261699.00 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 226169.90 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226169.900451 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035529.10 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 203552.91 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429722.810857 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831976.19 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 183197.62 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612920.430222 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648778.57 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 164877.86 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777798.287651 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483900.72 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 148390.07 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926188.359337 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335510.65 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 133551.06 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059739.423854 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201959.58 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 120195.96 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179935.381919 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081763.62 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 108176.36 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288111.744178 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973587.26 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 97358.73 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385470.470211 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876228.53 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 87622.85 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473093.323641 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788605.68 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 78860.57 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551953.891728 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709745.11 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 70974.51 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622928.403006 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638770.60 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 63877.06 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686805.463156 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574893.54 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 57489.35 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744294.817292 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517404.19 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 51740.42 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796035.236013 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465663.77 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 46566.38 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842601.612863 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419097.39 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 41909.74 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884511.352028 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377187.65 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 37718.77 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922230.117276 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339468.89 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 33946.89 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956177.005999 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305522.00 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 30552.20 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986729.205850 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274969.80 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 27496.98 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014226.185716 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247472.82 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 24747.28 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038973.467595 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222725.54 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 22272.55 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061246.021287 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200452.98 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 20045.30 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081291.319609 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180407.68 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 18040.77 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099332.088099 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162366.92 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 16236.69 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115568.779740 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146130.22 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 14613.02 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130181.802217 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131517.20 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 13151.72 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143333.522446 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118365.48 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 11836.55 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155170.070652 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106528.93 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 10652.89 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165822.964038 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95876.04 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 9587.60 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175410.568085 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86288.44 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 8628.84 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184039.411728 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77659.59 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 7765.96 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191805.371006 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69893.63 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 6989.36 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198794.734356 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62904.27 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 6290.43 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205085.161371 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56613.84 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 5661.38 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210746.545685 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50952.46 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 5095.25 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215841.791568 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45857.21 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 4585.72 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220427.512862 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41271.49 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 4127.15 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224554.662027 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37144.34 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3714.43 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228269.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33429.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231869.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29829.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235469.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26229.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239069.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22629.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242669.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19029.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246269.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15429.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249869.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11829.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253469.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8229.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257069.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4629.91 sec) 04:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260669.096275 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1029.91 sec) 04:43:00 [GPUGRID] [rr_sim] 2261699.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994647470.90G/439.78G) 04:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851719.92 04:43:00 [---] pick_jobs_to_run() start 04:43:00 [---] pick_jobs_to_run() end 04:43:00 [---] [rr_sim] end 04:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:43:00 [---] [cpu_sched_debug] preliminary job list: 04:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:43:00 [---] [cpu_sched_debug] final job list: 04:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:43:00 [---] [cpu_sched_debug] enforce_run_list: end 04:43:00 [---] choose_project(): 16980.000000 04:43:00 [iThena.Measurements] doing RPC to get work 04:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418619.09 cpu 2163032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:43:00 [---] Got 0 tasks 04:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:44:00 [---] [rr_sim] doing sim: CPU sched 04:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994647470.90G/439.78G) 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261699.00 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 226169.90 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226169.900451 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035529.10 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 203552.91 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429722.810857 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831976.19 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 183197.62 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612920.430222 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648778.57 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 164877.86 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777798.287651 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483900.72 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 148390.07 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926188.359337 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335510.65 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 133551.06 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059739.423854 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201959.58 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 120195.96 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179935.381919 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081763.62 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 108176.36 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288111.744178 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973587.26 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 97358.73 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385470.470211 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876228.53 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 87622.85 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473093.323641 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788605.68 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 78860.57 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551953.891728 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709745.11 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 70974.51 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622928.403006 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638770.60 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 63877.06 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686805.463156 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574893.54 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 57489.35 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744294.817292 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517404.19 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 51740.42 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796035.236013 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465663.77 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 46566.38 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842601.612863 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419097.39 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 41909.74 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884511.352028 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377187.65 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 37718.77 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922230.117276 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339468.89 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 33946.89 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956177.005999 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305522.00 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 30552.20 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986729.205850 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274969.80 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 27496.98 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014226.185716 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247472.82 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 24747.28 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038973.467595 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222725.54 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 22272.55 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061246.021287 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200452.98 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 20045.30 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081291.319609 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180407.68 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 18040.77 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099332.088099 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162366.92 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 16236.69 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115568.779740 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146130.22 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 14613.02 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130181.802217 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131517.20 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 13151.72 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143333.522446 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118365.48 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 11836.55 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155170.070652 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106528.93 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 10652.89 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165822.964038 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95876.04 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 9587.60 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175410.568085 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86288.44 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 8628.84 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184039.411728 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77659.59 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 7765.96 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191805.371006 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69893.63 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 6989.36 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198794.734356 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62904.27 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 6290.43 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205085.161371 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56613.84 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 5661.38 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210746.545685 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50952.46 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 5095.25 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215841.791568 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45857.21 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 4585.72 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220427.512862 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41271.49 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 4127.15 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224554.662027 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37144.34 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3714.43 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228269.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33429.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231869.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29829.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235469.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26229.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239069.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22629.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242669.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19029.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246269.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15429.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249869.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11829.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253469.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8229.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257069.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4629.91 sec) 04:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260669.096275 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1029.91 sec) 04:44:00 [GPUGRID] [rr_sim] 2261699.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994647470.90G/439.78G) 04:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851779.92 04:44:00 [---] pick_jobs_to_run() start 04:44:00 [---] pick_jobs_to_run() end 04:44:00 [---] [rr_sim] end 04:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:44:00 [---] [cpu_sched_debug] preliminary job list: 04:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:44:00 [---] [cpu_sched_debug] final job list: 04:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:44:00 [---] [cpu_sched_debug] enforce_run_list: end 04:44:00 [---] choose_project(): 17040.000000 04:44:00 [iThena.Measurements] doing RPC to get work 04:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418559.09 cpu 2163032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2163032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2163032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:44:00 [---] Got 0 tasks 04:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:45:00 [---] [rr_sim] doing sim: CPU sched 04:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994619880.54G/439.78G) 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261636.27 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 226163.63 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226163.626763 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035472.64 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 203547.26 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429710.890849 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831925.38 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 183192.54 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612903.428527 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648732.84 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 164873.28 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777776.712437 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483859.56 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 148385.96 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926162.667956 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335473.60 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 133547.36 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059710.027923 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201926.24 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 120192.62 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179902.651893 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081733.62 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 108173.36 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288076.013467 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973560.25 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 97356.03 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385432.038883 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876204.23 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 87620.42 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473052.461757 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788583.81 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 78858.38 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551910.842344 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709725.43 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 70972.54 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622883.384873 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638752.88 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 63875.29 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686758.673148 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574877.59 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 57487.76 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744246.432596 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517389.84 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 51738.98 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795985.416099 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465650.85 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 46565.09 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842550.501252 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419085.77 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 41908.58 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884459.077889 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377177.19 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 37717.72 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922176.796863 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339459.47 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 33945.95 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956122.743939 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305513.52 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 30551.35 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986674.096308 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274962.17 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 27496.22 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014170.313440 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247465.95 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 24746.60 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038916.908859 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222719.36 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 22271.94 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061188.844736 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200447.42 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 20044.74 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081233.587025 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180402.68 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 18040.27 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099273.855085 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162362.41 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 16236.24 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115510.096339 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146126.17 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 14612.62 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130122.713468 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131513.55 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 13151.36 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143274.068884 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118362.20 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 11836.22 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155110.288758 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106525.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 10652.60 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165762.886645 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95873.38 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 9587.34 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175350.224743 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86286.04 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 8628.60 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183978.829032 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77657.44 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 7765.74 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191744.572891 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69891.69 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 6989.17 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198733.742365 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62902.53 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 6290.25 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205023.994891 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56612.27 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 5661.23 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210685.222165 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50951.05 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 5095.10 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215780.326711 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45855.94 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 4585.59 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220365.920802 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41270.35 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 4127.03 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224492.955485 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37143.31 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3714.33 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228207.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33428.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231807.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29828.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235407.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26228.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239007.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22628.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242607.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19028.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246207.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15428.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249807.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11828.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253407.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8228.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257007.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4628.98 sec) 04:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260607.286699 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1028.98 sec) 04:45:00 [GPUGRID] [rr_sim] 2261636.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994619880.54G/439.78G) 04:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851777.18 04:45:00 [---] pick_jobs_to_run() start 04:45:00 [---] pick_jobs_to_run() end 04:45:00 [---] [rr_sim] end 04:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:45:00 [---] [cpu_sched_debug] preliminary job list: 04:45:00 [---] [cpu_sched_debug] final job list: 04:45:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:45:00 [---] [work_fetch] Request work fetch: CPUs idle 04:45:00 [---] [cpu_sched_debug] enforce_run_list: end 04:45:00 [---] choose_project(): 17100.000000 04:45:00 [iThena.Measurements] doing RPC to get work 04:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418499.09 cpu 2162972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:45:00 [---] Got 0 tasks 04:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:46:00 [---] [rr_sim] doing sim: CPU sched 04:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994619880.54G/439.78G) 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261636.27 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 226163.63 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226163.626763 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035472.64 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 203547.26 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429710.890849 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831925.38 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 183192.54 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612903.428527 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648732.84 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 164873.28 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777776.712437 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483859.56 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 148385.96 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926162.667956 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335473.60 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 133547.36 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059710.027923 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201926.24 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 120192.62 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179902.651893 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081733.62 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 108173.36 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288076.013467 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973560.25 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 97356.03 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385432.038883 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876204.23 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 87620.42 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473052.461757 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788583.81 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 78858.38 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551910.842344 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709725.43 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 70972.54 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622883.384873 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638752.88 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 63875.29 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686758.673148 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574877.59 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 57487.76 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744246.432596 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517389.84 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 51738.98 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795985.416099 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465650.85 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 46565.09 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842550.501252 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419085.77 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 41908.58 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884459.077889 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377177.19 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 37717.72 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922176.796863 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339459.47 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 33945.95 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956122.743939 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305513.52 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 30551.35 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986674.096308 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274962.17 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 27496.22 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014170.313440 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247465.95 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 24746.60 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038916.908859 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222719.36 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 22271.94 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061188.844736 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200447.42 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 20044.74 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081233.587025 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180402.68 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 18040.27 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099273.855085 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162362.41 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 16236.24 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115510.096339 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146126.17 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 14612.62 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130122.713468 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131513.55 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 13151.36 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143274.068884 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118362.20 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 11836.22 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155110.288758 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106525.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 10652.60 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165762.886645 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95873.38 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 9587.34 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175350.224743 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86286.04 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 8628.60 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183978.829032 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77657.44 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 7765.74 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191744.572891 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69891.69 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 6989.17 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198733.742365 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62902.53 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 6290.25 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205023.994891 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56612.27 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 5661.23 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210685.222165 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50951.05 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 5095.10 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215780.326711 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45855.94 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 4585.59 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220365.920802 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41270.35 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 4127.03 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224492.955485 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37143.31 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3714.33 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228207.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33428.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231807.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29828.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235407.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26228.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239007.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22628.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242607.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19028.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246207.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15428.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249807.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11828.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253407.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8228.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257007.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4628.98 sec) 04:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260607.286699 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1028.98 sec) 04:46:00 [GPUGRID] [rr_sim] 2261636.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994619880.54G/439.78G) 04:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851837.18 04:46:00 [---] pick_jobs_to_run() start 04:46:00 [---] pick_jobs_to_run() end 04:46:00 [---] [rr_sim] end 04:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:46:00 [---] [cpu_sched_debug] preliminary job list: 04:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:46:00 [---] [cpu_sched_debug] final job list: 04:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:46:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 994619880.54G 04:46:00 [---] [cpu_sched_debug] enforce_run_list: end 04:46:00 [---] choose_project(): 17160.000000 04:46:00 [iThena.Measurements] doing RPC to get work 04:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418439.09 cpu 2162972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:46:00 [---] Got 0 tasks 04:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:47:00 [---] [rr_sim] doing sim: CPU sched 04:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994592290.18G/439.78G) 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261573.53 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 226157.35 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226157.353074 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035416.18 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 203541.62 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429698.970842 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831874.56 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 183187.46 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612886.426832 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648687.10 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 164868.71 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777755.137223 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483818.39 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 148381.84 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926136.976575 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335436.55 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 133543.66 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059680.631992 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201892.90 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 120189.29 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179869.921868 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081703.61 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 108170.36 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288040.282755 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973533.25 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 97353.32 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385393.607554 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876179.92 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 87617.99 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473011.599873 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788561.93 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 78856.19 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551867.792960 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709705.74 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 70970.57 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622838.366739 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638735.16 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 63873.52 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686711.883139 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574861.65 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 57486.16 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744198.047900 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517375.48 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 51737.55 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795935.596185 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465637.93 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 46563.79 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842499.389641 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419074.14 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 41907.41 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884406.803751 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377166.73 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 37716.67 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922123.476450 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339450.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 33945.01 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956068.481880 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305505.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 30550.50 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986618.986766 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274954.54 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 27495.45 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014114.441164 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247459.09 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 24745.91 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038860.350122 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222713.18 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 22271.32 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061131.668185 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200441.86 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 20044.19 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081175.854441 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180397.68 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 18039.77 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099215.622071 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162357.91 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 16235.79 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115451.412938 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146122.12 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 14612.21 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130063.624719 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131509.91 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 13150.99 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143214.615322 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118358.92 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 11835.89 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155050.506864 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106523.02 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 10652.30 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165702.809252 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95870.72 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 9587.07 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175289.881401 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86283.65 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 8628.36 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183918.246336 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77655.28 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 7765.53 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191683.774777 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69889.76 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 6988.98 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198672.750373 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62900.78 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 6290.08 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204962.828411 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56610.70 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 5661.07 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210623.898644 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50949.63 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 5094.96 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215718.861854 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45854.67 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 4585.47 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220304.328743 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41269.20 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 4126.92 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224431.248943 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37142.28 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3714.23 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228145.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33428.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231745.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29828.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235345.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26228.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238945.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22628.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242545.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19028.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246145.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15428.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249745.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11828.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253345.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8228.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256945.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4628.05 sec) 04:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260545.477123 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1028.05 sec) 04:47:00 [GPUGRID] [rr_sim] 2261573.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994592290.18G/439.78G) 04:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851834.44 04:47:00 [---] pick_jobs_to_run() start 04:47:00 [---] pick_jobs_to_run() end 04:47:00 [---] [rr_sim] end 04:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:47:00 [---] [cpu_sched_debug] preliminary job list: 04:47:00 [---] [cpu_sched_debug] final job list: 04:47:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 04:47:00 [---] [work_fetch] Request work fetch: CPUs idle 04:47:00 [---] [cpu_sched_debug] enforce_run_list: end 04:47:00 [---] choose_project(): 17220.000000 04:47:00 [iThena.Measurements] doing RPC to get work 04:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418379.09 cpu 2162912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:47:00 [---] Got 0 tasks 04:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:48:00 [---] [rr_sim] doing sim: CPU sched 04:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994592290.18G/439.78G) 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261573.53 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 226157.35 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226157.353074 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035416.18 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 203541.62 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429698.970842 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831874.56 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 183187.46 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612886.426832 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648687.10 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 164868.71 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777755.137223 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483818.39 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 148381.84 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926136.976575 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335436.55 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 133543.66 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059680.631992 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201892.90 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 120189.29 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179869.921868 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081703.61 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 108170.36 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288040.282755 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973533.25 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 97353.32 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385393.607554 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876179.92 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 87617.99 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1473011.599873 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788561.93 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 78856.19 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551867.792960 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709705.74 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 70970.57 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622838.366739 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638735.16 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 63873.52 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686711.883139 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574861.65 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 57486.16 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744198.047900 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517375.48 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 51737.55 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795935.596185 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465637.93 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 46563.79 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842499.389641 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419074.14 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 41907.41 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884406.803751 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377166.73 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 37716.67 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922123.476450 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339450.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 33945.01 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956068.481880 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305505.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 30550.50 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986618.986766 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274954.54 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 27495.45 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014114.441164 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247459.09 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 24745.91 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038860.350122 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222713.18 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 22271.32 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061131.668185 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200441.86 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 20044.19 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081175.854441 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180397.68 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 18039.77 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099215.622071 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162357.91 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 16235.79 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115451.412938 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146122.12 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 14612.21 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130063.624719 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131509.91 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 13150.99 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143214.615322 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118358.92 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 11835.89 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155050.506864 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106523.02 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 10652.30 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165702.809252 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95870.72 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 9587.07 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175289.881401 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86283.65 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 8628.36 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183918.246336 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77655.28 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 7765.53 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191683.774777 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69889.76 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 6988.98 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198672.750373 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62900.78 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 6290.08 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204962.828411 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56610.70 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 5661.07 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210623.898644 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50949.63 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 5094.96 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215718.861854 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45854.67 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 4585.47 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220304.328743 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41269.20 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 4126.92 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224431.248943 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37142.28 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3714.23 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228145.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33428.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231745.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29828.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235345.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26228.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238945.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22628.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242545.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19028.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246145.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15428.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249745.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11828.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253345.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8228.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256945.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4628.05 sec) 04:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260545.477123 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1028.05 sec) 04:48:00 [GPUGRID] [rr_sim] 2261573.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994592290.18G/439.78G) 04:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851894.44 04:48:00 [---] pick_jobs_to_run() start 04:48:00 [---] pick_jobs_to_run() end 04:48:00 [---] [rr_sim] end 04:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:48:00 [---] [cpu_sched_debug] preliminary job list: 04:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:48:00 [---] [cpu_sched_debug] final job list: 04:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:48:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 994592290.18G 04:48:00 [---] [cpu_sched_debug] enforce_run_list: end 04:48:00 [---] choose_project(): 17280.000000 04:48:00 [iThena.Measurements] doing RPC to get work 04:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418319.09 cpu 2162912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:48:00 [---] Got 0 tasks 04:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:49:00 [---] [rr_sim] doing sim: CPU sched 04:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994564699.83G/439.78G) 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261510.79 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 226151.08 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226151.079386 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035359.71 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 203535.97 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429687.050834 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831823.74 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 183182.37 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612869.425137 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648641.37 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 164864.14 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777733.562009 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483777.23 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 148377.72 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926111.285195 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335399.51 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 133539.95 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059651.236062 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201859.56 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 120185.96 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179837.191842 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081673.60 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 108167.36 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1288004.552044 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973506.24 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 97350.62 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385355.176226 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876155.62 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 87615.56 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472970.737989 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788540.06 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 78854.01 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551824.743577 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709686.05 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 70968.61 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622793.348605 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638717.45 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 63871.74 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686665.093131 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574845.70 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 57484.57 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744149.663204 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517361.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 51736.11 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795885.776270 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465625.02 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 46562.50 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842448.278029 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419062.52 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 41906.25 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884354.529613 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377156.26 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 37715.63 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922070.156038 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339440.64 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 33944.06 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956014.219820 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305496.57 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 30549.66 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986563.877224 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274946.92 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 27494.69 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014058.568888 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247452.22 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 24745.22 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038803.791386 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222707.00 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 22270.70 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061074.491633 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200436.30 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 20043.63 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081118.121856 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180392.67 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 18039.27 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099157.389057 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162353.40 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 16235.34 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115392.729538 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146118.06 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 14611.81 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130004.535970 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131506.26 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 13150.63 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143155.161759 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118355.63 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 11835.56 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154990.724970 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106520.07 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 10652.01 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165642.731859 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95868.06 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 9586.81 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175229.538059 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86281.26 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 8628.13 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183857.663640 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77653.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 7765.31 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191622.976662 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69887.82 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 6988.78 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198611.758382 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62899.04 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 6289.90 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204901.661930 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56609.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 5660.91 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210562.575123 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50948.22 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 5094.82 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215657.396997 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45853.40 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 4585.34 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220242.736684 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41268.06 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 4126.81 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224369.542402 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37141.25 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3714.13 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228083.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33427.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231683.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29827.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235283.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26227.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238883.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22627.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242483.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19027.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246083.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15427.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249683.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11827.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253283.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8227.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256883.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4627.13 sec) 04:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260483.667548 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1027.13 sec) 04:49:00 [GPUGRID] [rr_sim] 2261510.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994564699.83G/439.78G) 04:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851891.71 04:49:00 [---] pick_jobs_to_run() start 04:49:00 [---] pick_jobs_to_run() end 04:49:00 [---] [rr_sim] end 04:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:49:00 [---] [cpu_sched_debug] preliminary job list: 04:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 04:49:00 [---] [cpu_sched_debug] final job list: 04:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:49:00 [---] [cpu_sched_debug] enforce_run_list: end 04:49:00 [---] choose_project(): 17340.000000 04:49:00 [iThena.Measurements] doing RPC to get work 04:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 591059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418259.09 cpu 2162852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:49:00 [---] Got 0 tasks 04:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:50:00 [---] [rr_sim] doing sim: CPU sched 04:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994537109.47G/439.78G) 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261448.06 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 226144.81 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226144.805698 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035303.25 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 203530.33 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429675.130826 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831772.93 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 183177.29 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612852.423442 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648595.63 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 164859.56 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777711.986796 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483736.07 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 148373.61 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926085.593814 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335362.46 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 133536.25 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059621.840131 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201826.22 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 120182.62 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179804.461816 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081643.60 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 108164.36 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287968.821332 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973479.24 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 97347.92 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385316.744897 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876131.31 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 87613.13 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472929.876105 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788518.18 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 78851.82 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551781.694193 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709666.36 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 70966.64 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622748.330472 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638699.73 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 63869.97 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686618.303123 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574829.75 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 57482.98 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744101.278508 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517346.78 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 51734.68 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795835.956356 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465612.10 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 46561.21 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842397.166418 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419050.89 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 41905.09 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884302.255474 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377145.80 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 37714.58 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922016.835625 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339431.22 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 33943.12 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955959.957760 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305488.10 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 30548.81 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986508.767683 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274939.29 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 27493.93 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014002.696612 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247445.36 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 24744.54 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038747.232649 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222700.82 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 22270.08 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061017.315082 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200430.74 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 20043.07 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081060.389272 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180387.67 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 18038.77 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099099.156043 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162348.90 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 16234.89 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115334.046137 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146114.01 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 14611.40 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129945.447221 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131502.61 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 13150.26 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143095.708197 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118352.35 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 11835.23 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154930.943075 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106517.11 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 10651.71 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165582.654466 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95865.40 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 9586.54 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175169.194717 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86278.86 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 8627.89 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183797.080944 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77650.98 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 7765.10 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191562.178547 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69885.88 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 6988.59 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198550.766391 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62897.29 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 6289.73 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204840.495450 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56607.56 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 5660.76 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210501.251603 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50946.81 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 5094.68 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215595.932141 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45852.12 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 4585.21 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220181.144625 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41266.91 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 4126.69 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224307.835860 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37140.22 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3714.02 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228021.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33426.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231621.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29826.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235221.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26226.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238821.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22626.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242421.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19026.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246021.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15426.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249621.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11826.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253221.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8226.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256821.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4626.20 sec) 04:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260421.857972 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1026.20 sec) 04:50:00 [GPUGRID] [rr_sim] 2261448.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994537109.47G/439.78G) 04:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851888.97 04:50:00 [---] pick_jobs_to_run() start 04:50:00 [---] pick_jobs_to_run() end 04:50:00 [---] [rr_sim] end 04:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:50:00 [---] [cpu_sched_debug] preliminary job list: 04:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:50:00 [---] [cpu_sched_debug] final job list: 04:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:50:00 [---] [cpu_sched_debug] enforce_run_list: end 04:50:00 [---] choose_project(): 17400.000000 04:50:00 [iThena.Measurements] doing RPC to get work 04:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418199.09 cpu 2162792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:50:00 [---] Got 0 tasks 04:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:51:00 [---] [rr_sim] doing sim: CPU sched 04:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994509519.11G/439.78G) 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261385.32 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 226138.53 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226138.532010 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035246.79 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 203524.68 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429663.210819 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831722.11 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 183172.21 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612835.421747 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648549.90 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 164854.99 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777690.411582 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483694.91 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 148369.49 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926059.902433 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335325.42 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 133532.54 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059592.444200 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201792.88 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 120179.29 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179771.731790 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081613.59 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 108161.36 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287933.090621 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973452.23 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 97345.22 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385278.313568 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876107.01 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 87610.70 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472889.014221 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788496.31 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 78849.63 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551738.644809 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709646.68 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 70964.67 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622703.312338 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638682.01 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 63868.20 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686571.513114 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574813.81 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 57481.38 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744052.893813 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517332.43 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 51733.24 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795786.136441 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465599.18 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 46559.92 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842346.054807 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419039.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 41903.93 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884249.981336 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377135.34 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 37713.53 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921963.515212 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339421.80 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 33942.18 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955905.695701 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305479.62 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 30547.96 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986453.658141 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274931.66 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 27493.17 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013946.824336 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247438.50 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 24743.85 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038690.673913 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222694.65 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 22269.46 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060960.138531 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200425.18 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 20042.52 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081002.656688 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180382.66 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 18038.27 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099040.923029 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162344.40 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 16234.44 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115275.362736 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146109.96 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 14611.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129886.358472 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131498.96 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 13149.90 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143036.254635 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118349.07 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 11834.91 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154871.161181 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106514.16 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 10651.42 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165522.577073 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95862.74 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 9586.27 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175108.851375 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86276.47 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 8627.65 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183736.498248 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77648.82 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 7764.88 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191501.380433 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69883.94 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 6988.39 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198489.774399 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62895.55 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 6289.55 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204779.328969 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56605.99 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 5660.60 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210439.928082 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50945.39 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 5094.54 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215534.467284 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45850.85 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 4585.09 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220119.552565 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41265.77 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 4126.58 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224246.129319 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37139.19 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3713.92 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227960.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33425.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231560.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29825.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235160.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26225.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238760.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22625.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242360.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19025.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245960.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15425.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249560.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11825.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253160.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8225.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256760.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4625.27 sec) 04:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260360.048397 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1025.27 sec) 04:51:00 [GPUGRID] [rr_sim] 2261385.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994509519.11G/439.78G) 04:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851886.23 04:51:00 [---] pick_jobs_to_run() start 04:51:00 [---] pick_jobs_to_run() end 04:51:00 [---] [rr_sim] end 04:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:51:00 [---] [cpu_sched_debug] preliminary job list: 04:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:51:00 [---] [cpu_sched_debug] final job list: 04:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:51:00 [---] [cpu_sched_debug] enforce_run_list: end 04:51:00 [---] choose_project(): 17460.000000 04:51:00 [iThena.Measurements] doing RPC to get work 04:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418139.09 cpu 2162732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:51:00 [---] Got 0 tasks 04:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:52:00 [---] [rr_sim] doing sim: CPU sched 04:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994481928.76G/439.78G) 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261322.58 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 226132.26 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226132.258322 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035190.32 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 203519.03 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429651.290811 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831671.29 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 183167.13 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612818.420052 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648504.16 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 164850.42 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777668.836368 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483653.75 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 148365.37 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926034.211053 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335288.37 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 133528.84 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059563.048269 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201759.53 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 120175.95 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179739.001764 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081583.58 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 108158.36 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287897.359909 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973425.22 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 97342.52 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385239.882240 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876082.70 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 87608.27 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472848.152337 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788474.43 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 78847.44 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551695.595425 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709626.99 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 70962.70 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622658.294204 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638664.29 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 63866.43 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686524.723106 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574797.86 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 57479.79 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744004.509117 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517318.07 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 51731.81 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795736.316527 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465586.27 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 46558.63 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842294.943196 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419027.64 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 41902.76 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884197.707198 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377124.88 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 37712.49 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921910.194800 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339412.39 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 33941.24 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955851.433641 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305471.15 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 30547.11 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986398.548599 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274924.03 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 27492.40 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013890.952060 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247431.63 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 24743.16 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038634.115176 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222688.47 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 22268.85 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060902.961980 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200419.62 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 20041.96 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080944.924104 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180377.66 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 18037.77 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098982.690015 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162339.89 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 16233.99 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115216.679335 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146105.90 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 14610.59 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129827.269723 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131495.31 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 13149.53 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142976.801072 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118345.78 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 11834.58 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154811.379287 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106511.20 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 10651.12 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165462.499680 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95860.08 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 9586.01 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175048.508033 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86274.08 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 8627.41 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183675.915552 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77646.67 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 7764.67 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191440.582318 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69882.00 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 6988.20 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198428.782408 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62893.80 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 6289.38 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204718.162489 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56604.42 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 5660.44 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210378.604562 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50943.98 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 5094.40 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215473.002427 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45849.58 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 4584.96 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220057.960506 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41264.62 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 4126.46 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224184.422777 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37138.16 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3713.82 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227898.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33424.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231498.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29824.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235098.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26224.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238698.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22624.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242298.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19024.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245898.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15424.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249498.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11824.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253098.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8224.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256698.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4624.34 sec) 04:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260298.238821 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1024.34 sec) 04:52:00 [GPUGRID] [rr_sim] 2261322.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994481928.76G/439.78G) 04:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851883.50 04:52:00 [---] pick_jobs_to_run() start 04:52:00 [---] pick_jobs_to_run() end 04:52:00 [---] [rr_sim] end 04:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:52:00 [---] [cpu_sched_debug] preliminary job list: 04:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:52:00 [---] [cpu_sched_debug] final job list: 04:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:52:00 [---] [cpu_sched_debug] enforce_run_list: end 04:52:00 [---] choose_project(): 17520.000000 04:52:00 [iThena.Measurements] doing RPC to get work 04:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418079.09 cpu 2162672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:52:00 [---] Got 0 tasks 04:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:53:00 [---] [rr_sim] doing sim: CPU sched 04:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994454338.40G/439.78G) 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261259.85 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 226125.98 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226125.984633 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035133.86 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 203513.39 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429639.370803 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831620.48 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 183162.05 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612801.418357 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648458.43 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 164845.84 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777647.261154 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483612.59 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 148361.26 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 926008.519672 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335251.33 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 133525.13 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059533.652338 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201726.19 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 120172.62 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179706.271738 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081553.57 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 108155.36 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287861.629198 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973398.22 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 97339.82 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385201.450911 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876058.40 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 87605.84 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472807.290454 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788452.56 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 78845.26 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551652.546042 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709607.30 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 70960.73 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622613.276071 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638646.57 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 63864.66 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686477.933097 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574781.91 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 57478.19 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743956.124421 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517303.72 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 51730.37 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795686.496612 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465573.35 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 46557.33 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842243.831584 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419016.01 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 41901.60 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884145.433059 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377114.41 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 37711.44 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921856.874387 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339402.97 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 33940.30 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955797.171582 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305462.67 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 30546.27 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986343.439057 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274916.41 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 27491.64 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013835.079784 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247424.77 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 24742.48 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038577.556439 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222682.29 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 22268.23 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060845.785429 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200414.06 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 20041.41 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080887.191519 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180372.65 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 18037.27 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098924.457001 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162335.39 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 16233.54 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115157.995934 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146101.85 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 14610.19 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129768.180974 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131491.67 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 13149.17 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142917.347510 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118342.50 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 11834.25 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154751.597393 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106508.25 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 10650.82 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165402.422287 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95857.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 9585.74 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174988.164691 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86271.68 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 8627.17 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183615.332856 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77644.51 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 7764.45 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191379.784204 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69880.06 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 6988.01 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198367.790417 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62892.06 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 6289.21 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204656.996008 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56602.85 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 5660.29 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210317.281041 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50942.57 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 5094.26 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215411.537570 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45848.31 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 4584.83 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219996.368447 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41263.48 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 4126.35 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224122.716235 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37137.13 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3713.71 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227836.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33423.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231436.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29823.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235036.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26223.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238636.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22623.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242236.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19023.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245836.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15423.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249436.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11823.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253036.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8223.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256636.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4623.42 sec) 04:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260236.429245 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1023.42 sec) 04:53:00 [GPUGRID] [rr_sim] 2261259.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994454338.40G/439.78G) 04:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851880.76 04:53:00 [---] pick_jobs_to_run() start 04:53:00 [---] pick_jobs_to_run() end 04:53:00 [---] [rr_sim] end 04:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:53:00 [---] [cpu_sched_debug] preliminary job list: 04:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:53:00 [---] [cpu_sched_debug] final job list: 04:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:53:00 [---] [cpu_sched_debug] enforce_run_list: end 04:53:00 [---] choose_project(): 17580.000000 04:53:00 [iThena.Measurements] doing RPC to get work 04:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 418019.09 cpu 2162612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:53:00 [---] Got 0 tasks 04:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:54:00 [---] [rr_sim] doing sim: CPU sched 04:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994426748.05G/439.78G) 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261197.11 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 226119.71 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226119.710945 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035077.40 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 203507.74 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429627.450796 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831569.66 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 183156.97 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612784.416661 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648412.69 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 164841.27 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777625.685941 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483571.42 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 148357.14 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925982.828292 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335214.28 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 133521.43 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059504.256408 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201692.85 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 120169.29 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179673.541712 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081523.57 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 108152.36 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287825.898486 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973371.21 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 97337.12 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385163.019583 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876034.09 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 87603.41 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472766.428570 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788430.68 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 78843.07 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551609.496658 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709587.61 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 70958.76 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622568.257937 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638628.85 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 63862.89 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686431.143089 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574765.97 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 57476.60 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743907.739725 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517289.37 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 51728.94 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795636.676698 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465560.43 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 46556.04 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842192.719973 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419004.39 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 41900.44 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884093.158921 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377103.95 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 37710.40 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921803.553974 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339393.56 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 33939.36 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955742.909522 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305454.20 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 30545.42 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986288.329515 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274908.78 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 27490.88 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013779.207509 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247417.90 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 24741.79 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038520.997703 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222676.11 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 22267.61 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060788.608878 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200408.50 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 20040.85 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080829.458935 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180367.65 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 18036.77 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098866.223987 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162330.89 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 16233.09 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115099.312533 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146097.80 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 14609.78 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129709.092225 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131488.02 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 13148.80 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142857.893948 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118339.22 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 11833.92 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154691.815498 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106505.29 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 10650.53 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165342.344894 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95854.76 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 9585.48 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174927.821349 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86269.29 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 8626.93 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183554.750160 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77642.36 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 7764.24 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191318.986089 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69878.12 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 6987.81 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198306.798425 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62890.31 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 6289.03 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204595.829528 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56601.28 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 5660.13 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210255.957520 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50941.15 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 5094.12 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215350.072713 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45847.04 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 4584.70 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219934.776387 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41262.33 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 4126.23 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224061.009694 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37136.10 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3713.61 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227774.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33422.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231374.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29822.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234974.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26222.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238574.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22622.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242174.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19022.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245774.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15422.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249374.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11822.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252974.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8222.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256574.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4622.49 sec) 04:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260174.619670 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1022.49 sec) 04:54:00 [GPUGRID] [rr_sim] 2261197.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994426748.05G/439.78G) 04:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851878.02 04:54:00 [---] pick_jobs_to_run() start 04:54:00 [---] pick_jobs_to_run() end 04:54:00 [---] [rr_sim] end 04:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:54:00 [---] [cpu_sched_debug] preliminary job list: 04:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:54:00 [---] [cpu_sched_debug] final job list: 04:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:54:00 [---] [cpu_sched_debug] enforce_run_list: end 04:54:00 [---] choose_project(): 17640.000000 04:54:00 [iThena.Measurements] doing RPC to get work 04:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417959.09 cpu 2162552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:54:00 [---] Got 0 tasks 04:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:55:00 [---] [rr_sim] doing sim: CPU sched 04:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994399157.69G/439.78G) 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261134.37 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 226113.44 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226113.437257 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2035020.94 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 203502.09 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429615.530788 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831518.84 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 183151.88 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612767.414966 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648366.96 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 164836.70 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777604.110727 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483530.26 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 148353.03 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925957.136911 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335177.24 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 133517.72 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059474.860477 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201659.51 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 120165.95 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179640.811686 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081493.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 108149.36 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287790.167775 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973344.20 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 97334.42 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385124.588254 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876009.78 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 87600.98 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472725.566686 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788408.81 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 78840.88 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551566.447274 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709567.93 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 70956.79 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622523.239804 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638611.13 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 63861.11 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686384.353080 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574750.02 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 57475.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743859.355029 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517275.02 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 51727.50 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795586.856783 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465547.52 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 46554.75 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842141.608362 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418992.76 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 41899.28 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884040.884783 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377093.49 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 37709.35 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921750.233561 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339384.14 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 33938.41 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955688.647462 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305445.73 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 30544.57 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986233.219973 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274901.15 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 27490.12 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013723.335233 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247411.04 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 24741.10 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038464.438966 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222669.93 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 22266.99 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060731.432327 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200402.94 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 20040.29 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080771.726351 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180362.65 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 18036.26 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098807.990973 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162326.38 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 16232.64 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115040.629133 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146093.74 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 14609.37 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129650.003476 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131484.37 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 13148.44 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142798.440386 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118335.93 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 11833.59 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154632.033604 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106502.34 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 10650.23 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165282.267501 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95852.11 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 9585.21 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174867.478007 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86266.89 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 8626.69 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183494.167464 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77640.21 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 7764.02 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191258.187974 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69876.18 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 6987.62 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198245.806434 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62888.57 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 6288.86 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204534.663047 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56599.71 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 5659.97 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210194.634000 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50939.74 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 5093.97 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215288.607857 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45845.76 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 4584.58 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219873.184328 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41261.19 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 4126.12 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223999.303152 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37135.07 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3713.51 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227712.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33421.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231312.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29821.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234912.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26221.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238512.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22621.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242112.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19021.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245712.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15421.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249312.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11821.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252912.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8221.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256512.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4621.56 sec) 04:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260112.810094 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1021.56 sec) 04:55:00 [GPUGRID] [rr_sim] 2261134.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994399157.69G/439.78G) 04:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851875.29 04:55:00 [---] pick_jobs_to_run() start 04:55:00 [---] pick_jobs_to_run() end 04:55:00 [---] [rr_sim] end 04:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:55:00 [---] [cpu_sched_debug] preliminary job list: 04:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:55:00 [---] [cpu_sched_debug] final job list: 04:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:55:00 [---] [cpu_sched_debug] enforce_run_list: end 04:55:00 [---] choose_project(): 17700.000000 04:55:00 [iThena.Measurements] doing RPC to get work 04:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417899.09 cpu 2162492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:55:00 [---] Got 0 tasks 04:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:56:00 [---] [rr_sim] doing sim: CPU sched 04:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994371567.33G/439.78G) 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261071.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 226107.16 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226107.163569 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034964.47 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 203496.45 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429603.610781 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831468.02 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 183146.80 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612750.413271 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648321.22 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 164832.12 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777582.535513 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483489.10 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 148348.91 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925931.445530 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335140.19 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 133514.02 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059445.464546 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201626.17 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 120162.62 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179608.081660 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081463.55 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 108146.36 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287754.437063 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973317.20 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 97331.72 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385086.156925 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875985.48 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 87598.55 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472684.704802 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788386.93 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 78838.69 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551523.397890 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709548.24 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 70954.82 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622478.221670 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638593.41 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 63859.34 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686337.563072 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574734.07 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 57473.41 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743810.970333 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517260.67 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 51726.07 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795537.036869 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465534.60 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 46553.46 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842090.496751 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418981.14 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 41898.11 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883988.610644 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377083.03 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 37708.30 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921696.913149 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339374.72 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 33937.47 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955634.385403 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305437.25 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 30543.73 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986178.110431 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274893.53 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 27489.35 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013667.462957 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247404.17 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 24740.42 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038407.880230 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222663.76 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 22266.38 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060674.255776 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200397.38 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 20039.74 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080713.993767 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180357.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 18035.76 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098749.757959 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162321.88 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 16232.19 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114981.945732 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146089.69 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 14608.97 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129590.914727 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131480.72 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 13148.07 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142738.986823 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118332.65 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 11833.26 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154572.251710 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106499.38 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 10649.94 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165222.190108 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95849.45 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 9584.94 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174807.134666 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86264.50 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 8626.45 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183433.584768 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77638.05 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 7763.81 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191197.389860 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69874.25 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 6987.42 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198184.814442 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62886.82 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 6288.68 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204473.496567 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56598.14 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 5659.81 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210133.310479 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50938.33 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 5093.83 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215227.143000 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45844.49 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 4584.45 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219811.592269 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41260.04 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 4126.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223937.596611 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37134.04 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3713.40 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227651.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33420.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231251.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29820.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234851.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26220.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238451.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22620.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242051.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19020.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245651.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15420.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249251.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11820.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252851.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8220.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256451.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4620.64 sec) 04:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2260051.000518 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1020.64 sec) 04:56:00 [GPUGRID] [rr_sim] 2261071.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994371567.33G/439.78G) 04:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851872.55 04:56:00 [---] pick_jobs_to_run() start 04:56:00 [---] pick_jobs_to_run() end 04:56:00 [---] [rr_sim] end 04:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:56:00 [---] [cpu_sched_debug] preliminary job list: 04:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:56:00 [---] [cpu_sched_debug] final job list: 04:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:56:00 [---] [cpu_sched_debug] enforce_run_list: end 04:56:00 [---] choose_project(): 17760.000000 04:56:00 [iThena.Measurements] doing RPC to get work 04:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417839.09 cpu 2162432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:56:00 [---] Got 0 tasks 04:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:57:00 [---] [rr_sim] doing sim: CPU sched 04:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994343976.98G/439.78G) 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2261008.90 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 226100.89 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226100.889881 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034908.01 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 203490.80 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429591.690773 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831417.21 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 183141.72 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612733.411576 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648275.49 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 164827.55 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777560.960299 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483447.94 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 148344.79 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925905.754150 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335103.14 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 133510.31 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059416.068615 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201592.83 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 120159.28 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179575.351634 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081433.55 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 108143.35 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287718.706351 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973290.19 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 97329.02 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385047.725597 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875961.17 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 87596.12 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472643.842918 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788365.06 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 78836.51 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551480.348506 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709528.55 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 70952.86 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622433.203536 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638575.70 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 63857.57 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686290.773063 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574718.13 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 57471.81 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743762.585637 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517246.31 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 51724.63 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795487.216954 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465521.68 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 46552.17 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842039.385139 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418969.51 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 41896.95 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883936.336506 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377072.56 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 37707.26 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921643.592736 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339365.31 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 33936.53 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955580.123343 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305428.78 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 30542.88 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986123.000889 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274885.90 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 27488.59 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013611.590681 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247397.31 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 24739.73 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038351.321493 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222657.58 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 22265.76 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060617.079224 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200391.82 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 20039.18 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080656.261183 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180352.64 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 18035.26 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098691.524945 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162317.37 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 16231.74 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114923.262331 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146085.64 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 14608.56 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129531.825978 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131477.07 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 13147.71 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142679.533261 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118329.37 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 11832.94 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154512.469815 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106496.43 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 10649.64 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165162.112714 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95846.79 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 9584.68 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174746.791324 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86262.11 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 8626.21 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183373.002072 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77635.90 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 7763.59 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191136.591745 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69872.31 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 6987.23 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198123.822451 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62885.08 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 6288.51 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204412.330087 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56596.57 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 5659.66 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210071.986958 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50936.91 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 5093.69 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215165.678143 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45843.22 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 4584.32 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219750.000209 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41258.90 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 4125.89 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223875.890069 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37133.01 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3713.30 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227589.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33419.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231189.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29819.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234789.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26219.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238389.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22619.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241989.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19019.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245589.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15419.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249189.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11819.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252789.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8219.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256389.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4619.71 sec) 04:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259989.190943 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1019.71 sec) 04:57:00 [GPUGRID] [rr_sim] 2261008.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994343976.98G/439.78G) 04:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851869.81 04:57:00 [---] pick_jobs_to_run() start 04:57:00 [---] pick_jobs_to_run() end 04:57:00 [---] [rr_sim] end 04:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:57:00 [---] [cpu_sched_debug] preliminary job list: 04:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:57:00 [---] [cpu_sched_debug] final job list: 04:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:57:00 [---] [cpu_sched_debug] enforce_run_list: end 04:57:00 [---] choose_project(): 17820.000000 04:57:00 [iThena.Measurements] doing RPC to get work 04:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417779.09 cpu 2162372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:57:00 [---] Got 0 tasks 04:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:58:00 [---] [rr_sim] doing sim: CPU sched 04:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994316386.62G/439.78G) 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260946.16 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 226094.62 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226094.616192 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034851.55 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 203485.15 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429579.770765 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831366.39 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 183136.64 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612716.409881 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648229.75 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 164822.98 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777539.385085 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483406.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 148340.68 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925880.062769 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335066.10 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 133506.61 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059386.672685 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201559.49 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 120155.95 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179542.621608 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081403.54 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 108140.35 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287682.975640 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973263.19 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 97326.32 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1385009.294268 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875936.87 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 87593.69 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472602.981034 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788343.18 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 78834.32 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551437.299123 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709508.86 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 70950.89 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622388.185403 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638557.98 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 63855.80 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686243.983055 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574702.18 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 57470.22 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743714.200942 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517231.96 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 51723.20 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795437.397040 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465508.76 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 46550.88 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841988.273528 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418957.89 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 41895.79 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883884.062368 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377062.10 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 37706.21 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921590.272323 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339355.89 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 33935.59 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955525.861283 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305420.30 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 30542.03 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986067.891347 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274878.27 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 27487.83 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013555.718405 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247390.44 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 24739.04 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038294.762757 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222651.40 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 22265.14 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060559.902673 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200386.26 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 20038.63 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080598.528598 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180347.63 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 18034.76 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098633.291931 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162312.87 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 16231.29 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114864.578930 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146081.58 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 14608.16 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129472.737229 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131473.42 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 13147.34 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142620.079699 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118326.08 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 11832.61 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154452.687921 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106493.47 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 10649.35 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165102.035321 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95844.13 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 9584.41 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174686.447982 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86259.71 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 8625.97 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183312.419376 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77633.74 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 7763.37 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191075.793630 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69870.37 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 6987.04 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198062.830460 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62883.33 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 6288.33 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204351.163606 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56595.00 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 5659.50 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210010.663438 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50935.50 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 5093.55 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215104.213286 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45841.95 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 4584.19 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219688.408150 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41257.75 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 4125.78 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223814.183527 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37131.98 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3713.20 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227527.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33418.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231127.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29818.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234727.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26218.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238327.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22618.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241927.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19018.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245527.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15418.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249127.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11818.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252727.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8218.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256327.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4618.78 sec) 04:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259927.381367 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1018.78 sec) 04:58:00 [GPUGRID] [rr_sim] 2260946.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994316386.62G/439.78G) 04:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851867.07 04:58:00 [---] pick_jobs_to_run() start 04:58:00 [---] pick_jobs_to_run() end 04:58:00 [---] [rr_sim] end 04:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:58:00 [---] [cpu_sched_debug] preliminary job list: 04:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:58:00 [---] [cpu_sched_debug] final job list: 04:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:58:00 [---] [cpu_sched_debug] enforce_run_list: end 04:58:00 [---] choose_project(): 17880.000000 04:58:00 [iThena.Measurements] doing RPC to get work 04:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417719.09 cpu 2162312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:58:00 [---] Got 0 tasks 04:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 04:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 04:59:00 [---] [rr_sim] doing sim: CPU sched 04:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994288796.26G/439.78G) 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260883.43 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 226088.34 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226088.342504 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034795.08 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 203479.51 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429567.850758 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831315.57 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 183131.56 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612699.408186 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648184.02 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 164818.40 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777517.809872 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483365.62 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 148336.56 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925854.371389 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1335029.05 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 133502.91 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059357.276754 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201526.15 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 120152.61 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179509.891583 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081373.53 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 108137.35 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287647.244928 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973236.18 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 97323.62 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384970.862940 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875912.56 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 87591.26 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472562.119150 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788321.31 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 78832.13 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551394.249739 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709489.18 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 70948.92 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622343.167269 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638540.26 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 63854.03 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686197.193046 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574686.23 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 57468.62 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743665.816246 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517217.61 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 51721.76 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795387.577125 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465495.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 46549.58 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841937.161917 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418946.26 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 41894.63 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883831.788229 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377051.64 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 37705.16 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921536.951910 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339346.47 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 33934.65 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955471.599224 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305411.83 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 30541.18 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986012.781805 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274870.64 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 27487.06 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013499.846129 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247383.58 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 24738.36 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038238.204020 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222645.22 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 22264.52 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060502.726122 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200380.70 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 20038.07 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080540.796014 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180342.63 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 18034.26 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098575.058917 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162308.37 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 16230.84 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114805.895529 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146077.53 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 14607.75 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129413.648480 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131469.78 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 13146.98 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142560.626136 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118322.80 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 11832.28 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154392.906027 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106490.52 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 10649.05 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165041.957928 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95841.47 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 9584.15 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174626.104640 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86257.32 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 8625.73 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183251.836680 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77631.59 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 7763.16 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191014.995516 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69868.43 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 6986.84 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198001.838468 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62881.59 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 6288.16 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204289.997126 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56593.43 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 5659.34 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209949.339917 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50934.09 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 5093.41 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215042.748430 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45840.68 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 4584.07 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219626.816091 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41256.61 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 4125.66 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223752.476986 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37130.95 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3713.09 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227465.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33417.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231065.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29817.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234665.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26217.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238265.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22617.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241865.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19017.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245465.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15417.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249065.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11817.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252665.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8217.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256265.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4617.85 sec) 04:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259865.571791 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1017.85 sec) 04:59:00 [GPUGRID] [rr_sim] 2260883.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994288796.26G/439.78G) 04:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851864.34 04:59:00 [---] pick_jobs_to_run() start 04:59:00 [---] pick_jobs_to_run() end 04:59:00 [---] [rr_sim] end 04:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 04:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 04:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 04:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 04:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 04:59:00 [---] [cpu_sched_debug] preliminary job list: 04:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:59:00 [---] [cpu_sched_debug] final job list: 04:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 04:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 04:59:00 [---] [cpu_sched_debug] enforce_run_list: end 04:59:00 [---] choose_project(): 17940.000000 04:59:00 [iThena.Measurements] doing RPC to get work 04:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417659.09 cpu 2162252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 04:59:00 [---] Got 0 tasks 05:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:00:00 [---] [rr_sim] doing sim: CPU sched 05:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994261205.91G/439.78G) 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260820.69 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 226082.07 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226082.068816 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034738.62 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 203473.86 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429555.930750 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831264.76 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 183126.48 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612682.406491 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648138.28 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 164813.83 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777496.234658 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483324.45 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 148332.45 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925828.680008 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334992.01 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 133499.20 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059327.880823 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201492.81 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 120149.28 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179477.161557 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081343.53 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 108134.35 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287611.514217 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973209.17 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 97320.92 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384932.431611 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875888.26 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 87588.83 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472521.257266 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788299.43 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 78829.94 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551351.200355 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709469.49 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 70946.95 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622298.149136 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638522.54 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 63852.25 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686150.403038 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574670.29 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 57467.03 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743617.431550 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517203.26 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 51720.33 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795337.757211 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465482.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 46548.29 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841886.050306 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418934.64 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 41893.46 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883779.514091 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377041.17 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 37704.12 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921483.631498 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339337.06 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 33933.71 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955417.337164 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305403.35 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 30540.34 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985957.672263 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274863.02 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 27486.30 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013443.973853 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247376.71 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 24737.67 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038181.645284 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222639.04 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 22263.90 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060445.549571 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200375.14 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 20037.51 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080483.063430 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180337.62 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 18033.76 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098516.825903 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162303.86 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 16230.39 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114747.212128 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146073.48 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 14607.35 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129354.559731 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131466.13 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 13146.61 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142501.172574 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118319.52 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 11831.95 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154333.124133 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106487.56 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 10648.76 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164981.880535 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95838.81 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 9583.88 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174565.761298 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86254.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 8625.49 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183191.253984 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77629.43 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 7762.94 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190954.197401 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69866.49 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 6986.65 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197940.846477 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62879.84 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 6287.98 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204228.830645 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56591.86 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 5659.19 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209888.016397 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50932.67 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 5093.27 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214981.283573 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45839.40 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 4583.94 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219565.224031 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41255.46 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 4125.55 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223690.770444 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37129.92 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3712.99 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227403.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33416.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231003.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29816.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234603.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26216.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238203.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22616.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241803.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19016.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245403.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15416.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249003.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11816.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252603.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8216.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256203.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4616.93 sec) 05:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259803.762216 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1016.93 sec) 05:00:00 [GPUGRID] [rr_sim] 2260820.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994261205.91G/439.78G) 05:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851861.60 05:00:00 [---] pick_jobs_to_run() start 05:00:00 [---] pick_jobs_to_run() end 05:00:00 [---] [rr_sim] end 05:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:00:00 [---] [cpu_sched_debug] preliminary job list: 05:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:00:00 [---] [cpu_sched_debug] final job list: 05:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:00:00 [---] [cpu_sched_debug] enforce_run_list: end 05:00:00 [---] choose_project(): 18000.000000 05:00:00 [iThena.Measurements] doing RPC to get work 05:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417599.09 cpu 2162192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:00:00 [---] Got 0 tasks 05:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:01:00 [---] [rr_sim] doing sim: CPU sched 05:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994233615.55G/439.78G) 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260757.95 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 226075.80 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226075.795128 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034682.16 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 203468.22 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429544.010743 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831213.94 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 183121.39 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612665.404796 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648092.55 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 164809.25 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777474.659444 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483283.29 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 148328.33 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925802.988627 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334954.96 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 133495.50 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059298.484892 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201459.47 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 120145.95 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179444.431531 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081313.52 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 108131.35 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287575.783505 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973182.17 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 97318.22 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384894.000282 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875863.95 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 87586.40 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472480.395382 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788277.56 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 78827.76 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551308.150971 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709449.80 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 70944.98 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622253.131002 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638504.82 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 63850.48 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686103.613029 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574654.34 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 57465.43 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743569.046854 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517188.90 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 51718.89 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795287.937296 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465470.01 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 46547.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841834.938694 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418923.01 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 41892.30 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883727.239953 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377030.71 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 37703.07 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921430.311085 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339327.64 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 33932.76 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955363.075104 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305394.88 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 30539.49 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985902.562721 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274855.39 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 27485.54 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013388.101577 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247369.85 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 24736.98 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038125.086547 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222632.86 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 22263.29 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060388.373020 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200369.58 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 20036.96 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080425.330846 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180332.62 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 18033.26 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098458.592889 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162299.36 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 16229.94 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114688.528728 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146069.42 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 14606.94 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129295.470982 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131462.48 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 13146.25 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142441.719012 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118316.23 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 11831.62 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154273.342238 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106484.61 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 10648.46 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164921.803142 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95836.15 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 9583.61 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174505.417956 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86252.53 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 8625.25 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183130.671288 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77627.28 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 7762.73 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190893.399287 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69864.55 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 6986.46 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197879.854486 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62878.10 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 6287.81 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204167.664165 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56590.29 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 5659.03 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209826.692876 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50931.26 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 5093.13 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214919.818716 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45838.13 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 4583.81 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219503.631972 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41254.32 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 4125.43 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223629.063903 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37128.89 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3712.89 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227341.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33416.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230941.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29816.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234541.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26216.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238141.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22616.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241741.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19016.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245341.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15416.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248941.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11816.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252541.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8216.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256141.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4616.00 sec) 05:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259741.952640 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1016.00 sec) 05:01:00 [GPUGRID] [rr_sim] 2260757.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994233615.55G/439.78G) 05:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851858.86 05:01:00 [---] pick_jobs_to_run() start 05:01:00 [---] pick_jobs_to_run() end 05:01:00 [---] [rr_sim] end 05:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:01:00 [---] [cpu_sched_debug] preliminary job list: 05:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:01:00 [---] [cpu_sched_debug] final job list: 05:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:01:00 [---] [cpu_sched_debug] enforce_run_list: end 05:01:00 [---] choose_project(): 18060.000000 05:01:00 [iThena.Measurements] doing RPC to get work 05:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417539.09 cpu 2162132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:01:00 [---] Got 0 tasks 05:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:02:00 [---] [rr_sim] doing sim: CPU sched 05:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994206025.20G/439.78G) 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260695.21 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 226069.52 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226069.521439 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034625.69 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 203462.57 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429532.090735 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831163.12 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 183116.31 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612648.403101 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648046.81 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 164804.68 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777453.084230 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483242.13 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 148324.21 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925777.297247 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334917.92 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 133491.79 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059269.088962 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201426.13 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 120142.61 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179411.701505 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081283.51 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 108128.35 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287540.052794 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973155.16 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 97315.52 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384855.568954 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875839.65 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 87583.96 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472439.533498 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788255.68 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 78825.57 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551265.101588 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709430.11 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 70943.01 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622208.112868 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638487.10 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 63848.71 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686056.823021 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574638.39 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 57463.84 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743520.662158 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517174.55 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 51717.46 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795238.117382 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465457.10 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 46545.71 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841783.827083 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418911.39 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 41891.14 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883674.965814 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377020.25 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 37702.02 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921376.990672 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339318.22 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 33931.82 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955308.813045 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305386.40 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 30538.64 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985847.453180 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274847.76 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 27484.78 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013332.229301 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247362.99 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 24736.30 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038068.527810 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222626.69 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 22262.67 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060331.196469 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200364.02 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 20036.40 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080367.598261 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180327.62 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 18032.76 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098400.359875 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162294.85 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 16229.49 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114629.845327 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146065.37 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 14606.54 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129236.382233 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131458.83 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 13145.88 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142382.265450 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118312.95 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 11831.29 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154213.560344 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106481.65 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 10648.17 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164861.725749 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95833.49 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 9583.35 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174445.074614 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86250.14 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 8625.01 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183070.088592 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77625.13 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 7762.51 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190832.601172 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69862.61 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 6986.26 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197818.862494 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62876.35 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 6287.64 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204106.497684 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56588.72 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 5658.87 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209765.369355 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50929.85 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 5092.98 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214858.353859 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45836.86 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 4583.69 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219442.039913 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41253.17 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 4125.32 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223567.357361 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37127.86 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3712.79 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227280.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33415.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230880.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29815.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234480.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26215.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238080.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22615.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241680.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19015.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245280.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15415.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248880.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11815.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252480.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8215.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256080.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4615.07 sec) 05:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259680.143064 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1015.07 sec) 05:02:00 [GPUGRID] [rr_sim] 2260695.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994206025.20G/439.78G) 05:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851856.13 05:02:00 [---] pick_jobs_to_run() start 05:02:00 [---] pick_jobs_to_run() end 05:02:00 [---] [rr_sim] end 05:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:02:00 [---] [cpu_sched_debug] preliminary job list: 05:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:02:00 [---] [cpu_sched_debug] final job list: 05:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:02:00 [---] [cpu_sched_debug] enforce_run_list: end 05:02:00 [---] choose_project(): 18120.000000 05:02:00 [iThena.Measurements] doing RPC to get work 05:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417479.09 cpu 2162072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:02:00 [---] Got 0 tasks 05:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:03:00 [---] [rr_sim] doing sim: CPU sched 05:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994178434.84G/439.78G) 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260632.48 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 226063.25 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226063.247751 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034569.23 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 203456.92 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429520.170727 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831112.31 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 183111.23 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612631.401406 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1648001.08 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 164800.11 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777431.509017 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483200.97 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 148320.10 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925751.605866 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334880.87 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 133488.09 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059239.693031 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201392.78 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 120139.28 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179378.971479 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081253.51 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 108125.35 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287504.322082 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973128.16 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 97312.82 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384817.137625 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875815.34 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 87581.53 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472398.671614 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788233.81 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 78823.38 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551222.052204 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709410.43 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 70941.04 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622163.094735 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638469.38 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 63846.94 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1686010.033012 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574622.44 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 57462.24 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743472.277462 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517160.20 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 51716.02 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795188.297467 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465444.18 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 46544.42 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841732.715472 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418899.76 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 41889.98 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883622.691676 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377009.79 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 37700.98 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921323.670260 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339308.81 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 33930.88 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955254.550985 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305377.93 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 30537.79 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985792.343638 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274840.13 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 27484.01 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013276.357025 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247356.12 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 24735.61 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038011.969074 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222620.51 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 22262.05 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060274.019918 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200358.46 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 20035.85 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080309.865677 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180322.61 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 18032.26 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098342.126861 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162290.35 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 16229.04 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114571.161926 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146061.32 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 14606.13 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129177.293485 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131455.18 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 13145.52 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142322.811887 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118309.67 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 11830.97 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154153.778450 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106478.70 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 10647.87 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164801.648356 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95830.83 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 9583.08 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174384.731272 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86247.75 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 8624.77 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183009.505896 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77622.97 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 7762.30 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190771.803057 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69860.67 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 6986.07 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197757.870503 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62874.61 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 6287.46 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204045.331204 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56587.15 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 5658.71 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209704.045835 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50928.43 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 5092.84 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214796.889002 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45835.59 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 4583.56 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219380.447853 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41252.03 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 4125.20 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223505.650819 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37126.83 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3712.68 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227218.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33414.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230818.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29814.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234418.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26214.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238018.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22614.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241618.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19014.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245218.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15414.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248818.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11814.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252418.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8214.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256018.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4614.14 sec) 05:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259618.333489 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1014.14 sec) 05:03:00 [GPUGRID] [rr_sim] 2260632.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994178434.84G/439.78G) 05:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851853.39 05:03:00 [---] pick_jobs_to_run() start 05:03:00 [---] pick_jobs_to_run() end 05:03:00 [---] [rr_sim] end 05:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:03:00 [---] [cpu_sched_debug] preliminary job list: 05:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:03:00 [---] [cpu_sched_debug] final job list: 05:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:03:00 [---] [cpu_sched_debug] enforce_run_list: end 05:03:00 [---] choose_project(): 18180.000000 05:03:00 [iThena.Measurements] doing RPC to get work 05:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417419.09 cpu 2162012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2162012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2162012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:03:00 [---] Got 0 tasks 05:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:04:00 [---] [rr_sim] doing sim: CPU sched 05:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994150844.48G/439.78G) 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260569.74 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 226056.97 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226056.974063 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034512.77 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 203451.28 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429508.250720 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831061.49 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 183106.15 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612614.399711 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647955.34 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 164795.53 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777409.933803 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483159.81 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 148315.98 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925725.914485 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334843.83 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 133484.38 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059210.297100 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201359.44 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 120135.94 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179346.241453 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081223.50 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 108122.35 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287468.591371 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973101.15 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 97310.11 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384778.706297 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875791.03 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 87579.10 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472357.809730 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788211.93 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 78821.19 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551179.002820 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709390.74 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 70939.07 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622118.076601 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638451.66 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 63845.17 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685963.243004 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574606.50 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 57460.65 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743423.892767 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517145.85 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 51714.58 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795138.477553 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465431.26 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 46543.13 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841681.603861 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418888.14 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 41888.81 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883570.417538 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376999.32 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 37699.93 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921270.349847 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339299.39 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 33929.94 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955200.288925 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305369.45 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 30536.95 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985737.234096 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274832.51 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 27483.25 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013220.484749 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247349.26 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 24734.93 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037955.410337 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222614.33 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 22261.43 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060216.843367 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200352.90 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 20035.29 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080252.133093 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180317.61 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 18031.76 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098283.893847 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162285.85 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 16228.58 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114512.478525 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146057.26 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 14605.73 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129118.204736 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131451.54 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 13145.15 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142263.358325 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118306.38 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 11830.64 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154093.996556 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106475.74 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 10647.57 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164741.570963 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95828.17 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 9582.82 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174324.387930 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86245.35 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 8624.54 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182948.923200 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77620.82 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 7762.08 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190711.004943 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69858.74 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 6985.87 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197696.878512 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62872.86 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 6287.29 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203984.164723 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56585.58 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 5658.56 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209642.722314 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50927.02 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 5092.70 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214735.424146 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45834.32 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 4583.43 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219318.855794 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41250.88 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 4125.09 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223443.944278 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37125.80 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3712.58 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227156.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33413.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230756.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29813.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234356.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26213.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237956.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22613.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241556.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19013.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245156.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15413.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248756.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11813.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252356.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8213.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255956.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4613.22 sec) 05:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259556.523913 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1013.22 sec) 05:04:00 [GPUGRID] [rr_sim] 2260569.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994150844.48G/439.78G) 05:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851850.65 05:04:00 [---] pick_jobs_to_run() start 05:04:00 [---] pick_jobs_to_run() end 05:04:00 [---] [rr_sim] end 05:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:04:00 [---] [cpu_sched_debug] preliminary job list: 05:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:04:00 [---] [cpu_sched_debug] final job list: 05:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:04:00 [---] [cpu_sched_debug] enforce_run_list: end 05:04:00 [---] choose_project(): 18240.000000 05:04:00 [iThena.Measurements] doing RPC to get work 05:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417359.09 cpu 2161952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:04:00 [---] Got 0 tasks 05:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:06:00 [---] [rr_sim] doing sim: CPU sched 05:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994123254.13G/439.78G) 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260507.00 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 226050.70 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226050.700375 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034456.30 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 203445.63 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429496.330712 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1831010.67 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 183101.07 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612597.398016 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647909.61 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 164790.96 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777388.358589 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483118.65 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 148311.86 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925700.223105 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334806.78 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 133480.68 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059180.901169 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201326.10 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 120132.61 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179313.511427 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081193.49 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 108119.35 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287432.860659 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973074.14 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 97307.41 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384740.274968 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875766.73 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 87576.67 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472316.947846 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788190.06 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 78819.01 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551135.953436 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709371.05 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 70937.11 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622073.058467 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638433.95 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 63843.39 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685916.452996 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574590.55 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 57459.06 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743375.508071 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517131.50 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 51713.15 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795088.657638 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465418.35 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 46541.83 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841630.492249 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418876.51 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 41887.65 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883518.143399 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376988.86 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 37698.89 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921217.029434 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339289.97 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 33929.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955146.026866 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305360.98 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 30536.10 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985682.124554 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274824.88 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 27482.49 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013164.612473 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247342.39 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 24734.24 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037898.851601 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222608.15 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 22260.82 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060159.666815 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200347.34 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 20034.73 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080194.400509 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180312.60 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 18031.26 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098225.660833 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162281.34 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 16228.13 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114453.795124 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146053.21 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 14605.32 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129059.115987 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131447.89 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 13144.79 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142203.904763 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118303.10 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 11830.31 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154034.214661 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106472.79 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 10647.28 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164681.493570 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95825.51 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 9582.55 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174264.044588 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86242.96 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 8624.30 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182888.340504 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77618.66 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 7761.87 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190650.206828 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69856.80 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 6985.68 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197635.886520 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62871.12 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 6287.11 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203922.998243 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56584.01 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 5658.40 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209581.398793 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50925.60 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 5092.56 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214673.959289 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45833.04 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 4583.30 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219257.263735 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41249.74 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 4124.97 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223382.237736 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37124.77 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3712.48 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227094.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33412.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230694.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29812.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234294.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26212.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237894.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22612.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241494.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19012.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245094.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15412.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248694.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11812.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252294.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8212.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255894.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4612.29 sec) 05:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259494.714337 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1012.29 sec) 05:06:00 [GPUGRID] [rr_sim] 2260507.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994123254.13G/439.78G) 05:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851907.92 05:06:00 [---] pick_jobs_to_run() start 05:06:00 [---] pick_jobs_to_run() end 05:06:00 [---] [rr_sim] end 05:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:06:00 [---] [cpu_sched_debug] preliminary job list: 05:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:06:00 [---] [cpu_sched_debug] final job list: 05:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:06:00 [---] [cpu_sched_debug] enforce_run_list: end 05:06:00 [---] choose_project(): 18360.000000 05:06:00 [iThena.Measurements] doing RPC to get work 05:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 590039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417239.09 cpu 2161892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:06:00 [---] Got 0 tasks 05:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:07:00 [---] [rr_sim] doing sim: CPU sched 05:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994095663.77G/439.78G) 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260444.27 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 226044.43 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226044.426687 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034399.84 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 203439.98 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429484.410705 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830959.86 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 183095.99 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612580.396321 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647863.87 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 164786.39 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777366.783375 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483077.48 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 148307.75 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925674.531724 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334769.74 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 133476.97 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059151.505238 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201292.76 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 120129.28 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179280.781401 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081163.49 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 108116.35 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287397.129948 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973047.14 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 97304.71 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384701.843639 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875742.42 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 87574.24 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472276.085962 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788168.18 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 78816.82 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551092.904052 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709351.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 70935.14 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1622028.040334 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638416.23 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 63841.62 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685869.662987 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574574.60 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 57457.46 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743327.123375 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517117.14 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 51711.71 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795038.837724 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465405.43 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 46540.54 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841579.380638 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418864.89 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 41886.49 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883465.869261 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376978.40 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 37697.84 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921163.709021 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339280.56 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 33928.06 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955091.764806 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305352.50 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 30535.25 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985627.015012 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274817.25 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 27481.73 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013108.740197 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247335.53 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 24733.55 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037842.292864 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222601.97 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 22260.20 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060102.490264 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200341.78 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 20034.18 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080136.667925 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180307.60 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 18030.76 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098167.427819 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162276.84 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 16227.68 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114395.111723 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146049.16 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 14604.92 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129000.027238 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131444.24 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 13144.42 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142144.451200 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118299.82 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 11829.98 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153974.432767 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106469.83 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 10646.98 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164621.416177 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95822.85 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 9582.29 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174203.701246 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86240.57 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 8624.06 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182827.757808 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77616.51 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 7761.65 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190589.408714 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69854.86 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 6985.49 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197574.894529 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62869.37 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 6286.94 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203861.831763 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56582.44 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 5658.24 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209520.075273 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50924.19 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 5092.42 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214612.494432 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45831.77 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 4583.18 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219195.671676 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41248.60 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 4124.86 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223320.531195 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37123.74 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3712.37 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227032.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33411.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230632.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29811.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234232.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26211.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237832.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22611.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241432.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19011.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245032.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15411.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248632.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11811.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252232.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8211.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255832.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4611.36 sec) 05:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259432.904762 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1011.36 sec) 05:07:00 [GPUGRID] [rr_sim] 2260444.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994095663.77G/439.78G) 05:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851905.18 05:07:00 [---] pick_jobs_to_run() start 05:07:00 [---] pick_jobs_to_run() end 05:07:00 [---] [rr_sim] end 05:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:07:00 [---] [cpu_sched_debug] preliminary job list: 05:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:07:00 [---] [cpu_sched_debug] final job list: 05:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:07:00 [---] [cpu_sched_debug] enforce_run_list: end 05:07:00 [---] choose_project(): 18420.000000 05:07:00 [iThena.Measurements] doing RPC to get work 05:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417179.09 cpu 2161832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:07:00 [---] Got 0 tasks 05:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:08:00 [---] [rr_sim] doing sim: CPU sched 05:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994068073.41G/439.78G) 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260381.53 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 226038.15 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226038.152998 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034343.38 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 203434.34 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429472.490697 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830909.04 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 183090.90 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612563.394626 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647818.14 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 164781.81 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777345.208161 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1483036.32 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 148303.63 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925648.840344 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334732.69 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 133473.27 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059122.109308 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201259.42 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 120125.94 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179248.051375 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081133.48 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 108113.35 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287361.399236 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973020.13 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 97302.01 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384663.412311 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875718.12 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 87571.81 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472235.224078 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788146.31 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 78814.63 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551049.854669 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709331.68 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 70933.17 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621983.022200 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638398.51 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 63839.85 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685822.872979 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574558.66 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 57455.87 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743278.738679 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517102.79 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 51710.28 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794989.017810 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465392.51 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 46539.25 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841528.269027 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418853.26 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 41885.33 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883413.595123 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376967.93 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 37696.79 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921110.388609 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339271.14 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 33927.11 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955037.502746 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305344.03 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 30534.40 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985571.905470 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274809.62 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 27480.96 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013052.867921 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247328.66 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 24732.87 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037785.734128 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222595.80 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 22259.58 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060045.313713 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200336.22 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 20033.62 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080078.935340 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180302.59 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 18030.26 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098109.194805 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162272.34 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 16227.23 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114336.428323 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146045.10 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 14604.51 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128940.938489 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131440.59 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 13144.06 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142084.997638 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118296.53 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 11829.65 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153914.650873 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106466.88 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 10646.69 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164561.338784 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95820.19 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 9582.02 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174143.357904 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86238.17 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 8623.82 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182767.175112 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77614.35 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 7761.44 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190528.610599 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69852.92 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 6985.29 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197513.902537 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62867.63 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 6286.76 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203800.665282 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56580.86 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 5658.09 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209458.751752 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50922.78 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 5092.28 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214551.029575 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45830.50 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 4583.05 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219134.079616 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41247.45 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 4124.75 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223258.824653 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37122.71 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3712.27 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226971.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33410.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230571.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29810.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234171.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26210.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237771.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22610.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241371.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19010.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244971.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15410.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248571.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11810.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252171.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8210.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255771.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4610.43 sec) 05:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259371.095186 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1010.43 sec) 05:08:00 [GPUGRID] [rr_sim] 2260381.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994068073.41G/439.78G) 05:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851902.44 05:08:00 [---] pick_jobs_to_run() start 05:08:00 [---] pick_jobs_to_run() end 05:08:00 [---] [rr_sim] end 05:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:08:00 [---] [cpu_sched_debug] preliminary job list: 05:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:08:00 [---] [cpu_sched_debug] final job list: 05:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:08:00 [---] [cpu_sched_debug] enforce_run_list: end 05:08:00 [---] choose_project(): 18480.000000 05:08:00 [iThena.Measurements] doing RPC to get work 05:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417119.09 cpu 2161772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:08:00 [---] Got 0 tasks 05:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:09:00 [---] [rr_sim] doing sim: CPU sched 05:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994040483.06G/439.78G) 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260318.79 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 226031.88 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226031.879310 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034286.91 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 203428.69 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429460.570689 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830858.22 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 183085.82 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612546.392931 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647772.40 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 164777.24 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777323.632948 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482995.16 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 148299.52 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925623.148963 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334695.64 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 133469.56 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059092.713377 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201226.08 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 120122.61 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179215.321349 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081103.47 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 108110.35 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287325.668525 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972993.12 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 97299.31 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384624.980982 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875693.81 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 87569.38 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472194.362194 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788124.43 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 78812.44 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1551006.805285 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709311.99 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 70931.20 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621938.004067 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638380.79 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 63838.08 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685776.082970 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574542.71 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 57454.27 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743230.353983 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517088.44 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 51708.84 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794939.197895 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465379.60 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 46537.96 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841477.157416 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418841.64 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 41884.16 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883361.320984 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376957.47 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 37695.75 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921057.068196 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339261.72 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 33926.17 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954983.240687 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305335.55 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 30533.56 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985516.795928 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274802.00 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 27480.20 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012996.995645 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247321.80 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 24732.18 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037729.175391 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222589.62 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 22258.96 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059988.137162 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200330.66 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 20033.07 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080021.202756 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180297.59 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 18029.76 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098050.961791 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162267.83 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 16226.78 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114277.744922 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146041.05 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 14604.10 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128881.849740 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131436.94 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 13143.69 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142025.544076 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118293.25 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 11829.32 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153854.868978 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106463.92 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 10646.39 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164501.261391 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95817.53 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 9581.75 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174083.014562 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86235.78 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 8623.58 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182706.592416 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77612.20 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 7761.22 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190467.812484 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69850.98 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 6985.10 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197452.910546 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62865.88 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 6286.59 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203739.498802 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56579.29 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 5657.93 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209397.428232 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50921.36 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 5092.14 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214489.564719 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45829.23 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 4582.92 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219072.487557 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41246.31 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 4124.63 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223197.118111 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37121.67 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3712.17 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226909.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33409.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230509.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29809.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234109.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26209.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237709.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22609.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241309.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19009.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244909.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15409.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248509.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11809.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252109.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8209.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255709.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4609.51 sec) 05:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259309.285610 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1009.51 sec) 05:09:00 [GPUGRID] [rr_sim] 2260318.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994040483.06G/439.78G) 05:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851899.71 05:09:00 [---] pick_jobs_to_run() start 05:09:00 [---] pick_jobs_to_run() end 05:09:00 [---] [rr_sim] end 05:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:09:00 [---] [cpu_sched_debug] preliminary job list: 05:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:09:00 [---] [cpu_sched_debug] final job list: 05:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:09:00 [---] [cpu_sched_debug] enforce_run_list: end 05:09:00 [---] choose_project(): 18540.000000 05:09:00 [iThena.Measurements] doing RPC to get work 05:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 417059.09 cpu 2161712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:09:00 [---] Got 0 tasks 05:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:10:00 [---] [rr_sim] doing sim: CPU sched 05:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (994012892.70G/439.78G) 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260256.06 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 226025.61 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226025.605622 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034230.45 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 203423.05 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429448.650682 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830807.41 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 183080.74 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612529.391235 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647726.66 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 164772.67 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777302.057734 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482954.00 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 148295.40 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925597.457582 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334658.60 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 133465.86 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059063.317446 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201192.74 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 120119.27 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179182.591323 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081073.46 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 108107.35 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287289.937813 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972966.12 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 97296.61 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384586.549654 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875669.51 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 87566.95 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472153.500310 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788102.56 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 78810.26 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550963.755901 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709292.30 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 70929.23 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621892.985933 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638363.07 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 63836.31 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685729.292962 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574526.76 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 57452.68 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743181.969287 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517074.09 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 51707.41 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794889.377981 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465366.68 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 46536.67 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841426.045804 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418830.01 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 41883.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883309.046846 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376947.01 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 37694.70 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921003.747783 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339252.31 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 33925.23 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954928.978627 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305327.08 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 30532.71 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985461.686386 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274794.37 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 27479.44 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012941.123370 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247314.93 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 24731.49 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037672.616654 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222583.44 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 22258.34 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059930.960611 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200325.10 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 20032.51 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079963.470172 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180292.59 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 18029.26 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097992.728777 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162263.33 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 16226.33 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114219.061521 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146036.99 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 14603.70 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128822.760991 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131433.30 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 13143.33 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141966.090514 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118289.97 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 11829.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153795.087084 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106460.97 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 10646.10 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164441.183998 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95814.87 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 9581.49 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174022.671220 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86233.38 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 8623.34 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182646.009720 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77610.05 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 7761.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190407.014370 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69849.04 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 6984.90 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197391.918555 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62864.14 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 6286.41 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203678.332321 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56577.72 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 5657.77 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209336.104711 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50919.95 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 5092.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214428.099862 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45827.96 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 4582.80 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219010.895498 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41245.16 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 4124.52 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223135.411570 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37120.64 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3712.06 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226847.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33408.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230447.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29808.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234047.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26208.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237647.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22608.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241247.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19008.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244847.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15408.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248447.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11808.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252047.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8208.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255647.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4608.58 sec) 05:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259247.476035 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1008.58 sec) 05:10:00 [GPUGRID] [rr_sim] 2260256.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (994012892.70G/439.78G) 05:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851896.97 05:10:00 [---] pick_jobs_to_run() start 05:10:00 [---] pick_jobs_to_run() end 05:10:00 [---] [rr_sim] end 05:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:10:00 [---] [cpu_sched_debug] preliminary job list: 05:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:10:00 [---] [cpu_sched_debug] final job list: 05:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:10:00 [---] [cpu_sched_debug] enforce_run_list: end 05:10:00 [---] choose_project(): 18600.000000 05:10:00 [iThena.Measurements] doing RPC to get work 05:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416999.09 cpu 2161652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:10:00 [---] Got 0 tasks 05:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:11:00 [---] [rr_sim] doing sim: CPU sched 05:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993985302.35G/439.78G) 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260193.32 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 226019.33 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226019.331934 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034173.99 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 203417.40 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429436.730674 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830756.59 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 183075.66 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612512.389540 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647680.93 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 164768.09 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777280.482520 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482912.84 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 148291.28 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925571.766202 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334621.55 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 133462.16 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059033.921515 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201159.40 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 120115.94 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179149.861298 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081043.46 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 108104.35 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287254.207102 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972939.11 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 97293.91 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384548.118325 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875645.20 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 87564.52 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472112.638426 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788080.68 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 78808.07 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550920.706517 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709272.61 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 70927.26 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621847.967799 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638345.35 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 63834.54 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685682.502953 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574510.82 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 57451.08 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743133.584592 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517059.73 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 51705.97 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794839.558066 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465353.76 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 46535.38 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841374.934193 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418818.39 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 41881.84 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883256.772708 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376936.55 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 37693.65 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920950.427371 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339242.89 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 33924.29 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954874.716567 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305318.60 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 30531.86 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985406.576844 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274786.74 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 27478.67 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012885.251094 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247308.07 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 24730.81 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037616.057918 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222577.26 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 22257.73 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059873.784060 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200319.54 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 20031.95 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079905.737588 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180287.58 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 18028.76 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097934.495763 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162258.82 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 16225.88 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114160.378120 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146032.94 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 14603.29 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128763.672242 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131429.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 13142.96 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141906.636951 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118286.68 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 11828.67 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153735.305190 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106458.01 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 10645.80 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164381.106605 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95812.21 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 9581.22 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173962.327878 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86230.99 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 8623.10 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182585.427024 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77607.89 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 7760.79 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190346.216255 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69847.10 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 6984.71 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197330.926563 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62862.39 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 6286.24 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203617.165841 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56576.15 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 5657.62 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209274.781190 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50918.54 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 5091.85 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214366.635005 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45826.68 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 4582.67 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218949.303438 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41244.02 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 4124.40 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223073.705028 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37119.61 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3711.96 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226785.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33407.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230385.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29807.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233985.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26207.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237585.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22607.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241185.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19007.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244785.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15407.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248385.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11807.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251985.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8207.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255585.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4607.65 sec) 05:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259185.666459 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1007.65 sec) 05:11:00 [GPUGRID] [rr_sim] 2260193.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993985302.35G/439.78G) 05:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851894.23 05:11:00 [---] pick_jobs_to_run() start 05:11:00 [---] pick_jobs_to_run() end 05:11:00 [---] [rr_sim] end 05:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:11:00 [---] [cpu_sched_debug] preliminary job list: 05:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:11:00 [---] [cpu_sched_debug] final job list: 05:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:11:00 [---] [cpu_sched_debug] enforce_run_list: end 05:11:00 [---] choose_project(): 18660.000000 05:11:00 [iThena.Measurements] doing RPC to get work 05:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416939.09 cpu 2161592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:11:00 [---] Got 0 tasks 05:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:12:00 [---] [rr_sim] doing sim: CPU sched 05:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993957711.99G/439.78G) 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260130.58 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 226013.06 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226013.058246 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034117.52 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 203411.75 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429424.810666 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830705.77 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 183070.58 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612495.387845 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647635.19 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 164763.52 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777258.907306 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482871.68 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 148287.17 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925546.074821 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334584.51 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 133458.45 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1059004.525585 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201126.06 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 120112.61 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179117.131272 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1081013.45 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 108101.35 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287218.476390 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972912.11 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 97291.21 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384509.686996 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875620.90 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 87562.09 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472071.776542 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788058.81 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 78805.88 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550877.657134 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709252.93 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 70925.29 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621802.949666 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638327.63 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 63832.76 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685635.712945 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574494.87 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 57449.49 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743085.199896 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517045.38 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 51704.54 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794789.738152 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465340.84 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 46534.08 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841323.822582 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418806.76 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 41880.68 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883204.498569 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376926.08 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 37692.61 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920897.106958 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339233.48 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 33923.35 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954820.454508 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305310.13 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 30531.01 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985351.467302 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274779.12 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 27477.91 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012829.378818 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247301.20 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 24730.12 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037559.499181 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222571.08 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 22257.11 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059816.607509 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200313.97 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 20031.40 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079848.005003 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180282.58 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 18028.26 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097876.262749 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162254.32 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 16225.43 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114101.694719 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146028.89 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 14602.89 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128704.583493 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131426.00 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 13142.60 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141847.183389 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118283.40 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 11828.34 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153675.523296 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106455.06 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 10645.51 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164321.029212 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95809.55 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 9580.96 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173901.984536 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86228.60 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 8622.86 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182524.844328 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77605.74 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 7760.57 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190285.418141 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69845.16 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 6984.52 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197269.934572 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62860.65 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 6286.06 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203555.999360 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56574.58 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 5657.46 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209213.457670 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50917.12 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 5091.71 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214305.170148 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45825.41 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 4582.54 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218887.711379 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41242.87 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 4124.29 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223011.998487 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37118.58 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3711.86 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226723.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33406.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230323.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29806.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233923.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26206.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237523.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22606.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241123.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19006.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244723.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15406.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248323.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11806.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251923.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8206.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255523.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4606.73 sec) 05:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259123.856883 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1006.73 sec) 05:12:00 [GPUGRID] [rr_sim] 2260130.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993957711.99G/439.78G) 05:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851891.50 05:12:00 [---] pick_jobs_to_run() start 05:12:00 [---] pick_jobs_to_run() end 05:12:00 [---] [rr_sim] end 05:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:12:00 [---] [cpu_sched_debug] preliminary job list: 05:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:12:00 [---] [cpu_sched_debug] final job list: 05:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:12:00 [---] [cpu_sched_debug] enforce_run_list: end 05:12:00 [---] choose_project(): 18720.000000 05:12:00 [iThena.Measurements] doing RPC to get work 05:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416879.09 cpu 2161532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:12:00 [---] Got 0 tasks 05:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:13:00 [---] [rr_sim] doing sim: CPU sched 05:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993930121.63G/439.78G) 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260067.85 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 226006.78 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226006.784557 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034061.06 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 203406.11 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429412.890659 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830654.95 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 183065.50 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612478.386150 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647589.46 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 164758.95 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777237.332093 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482830.51 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 148283.05 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925520.383441 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334547.46 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 133454.75 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058975.129654 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201092.72 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 120109.27 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179084.401246 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080983.44 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 108098.34 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287182.745678 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972885.10 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 97288.51 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384471.255668 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875596.59 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 87559.66 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1472030.914658 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788036.93 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 78803.69 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550834.607750 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709233.24 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 70923.32 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621757.931532 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638309.91 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 63830.99 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685588.922936 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574478.92 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 57447.89 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743036.815200 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517031.03 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 51703.10 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794739.918237 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465327.93 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 46532.79 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841272.710971 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418795.13 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 41879.51 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883152.224431 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376915.62 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 37691.56 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920843.786545 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339224.06 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 33922.41 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954766.192448 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305301.65 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 30530.17 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985296.357760 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274771.49 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 27477.15 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012773.506542 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247294.34 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 24729.43 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037502.940445 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222564.91 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 22256.49 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059759.430958 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200308.41 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 20030.84 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079790.272419 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180277.57 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 18027.76 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097818.029735 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162249.82 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 16224.98 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114043.011318 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146024.83 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 14602.48 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128645.494744 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131422.35 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 13142.24 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141787.729827 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118280.12 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 11828.01 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153615.741401 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106452.10 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 10645.21 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164260.951819 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95806.89 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 9580.69 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173841.641194 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86226.20 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 8622.62 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182464.261632 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77603.58 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 7760.36 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190224.620026 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69843.23 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 6984.32 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197208.942581 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62858.90 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 6285.89 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203494.832880 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56573.01 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 5657.30 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209152.134149 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50915.71 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 5091.57 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214243.705292 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45824.14 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 4582.41 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218826.119320 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41241.73 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 4124.17 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222950.291945 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37117.55 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3711.76 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226662.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33405.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230262.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29805.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233862.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26205.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237462.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22605.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241062.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19005.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244662.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15405.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248262.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11805.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251862.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8205.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255462.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4605.80 sec) 05:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259062.047308 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1005.80 sec) 05:13:00 [GPUGRID] [rr_sim] 2260067.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993930121.63G/439.78G) 05:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851888.76 05:13:00 [---] pick_jobs_to_run() start 05:13:00 [---] pick_jobs_to_run() end 05:13:00 [---] [rr_sim] end 05:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:13:00 [---] [cpu_sched_debug] preliminary job list: 05:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:13:00 [---] [cpu_sched_debug] final job list: 05:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:13:00 [---] [cpu_sched_debug] enforce_run_list: end 05:13:00 [---] choose_project(): 18780.000000 05:13:00 [iThena.Measurements] doing RPC to get work 05:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416819.09 cpu 2161472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:13:00 [---] Got 0 tasks 05:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:14:00 [---] [rr_sim] doing sim: CPU sched 05:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993902531.28G/439.78G) 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2260005.11 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 226000.51 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 226000.510869 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2034004.60 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 203400.46 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429400.970651 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830604.14 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 183060.41 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612461.384455 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647543.72 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 164754.37 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777215.756879 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482789.35 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 148278.94 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925494.692060 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334510.42 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 133451.04 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058945.733723 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201059.37 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 120105.94 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179051.671220 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080953.44 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 108095.34 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287147.014967 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972858.09 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 97285.81 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384432.824339 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875572.28 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 87557.23 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471990.052774 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788015.06 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 78801.51 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550791.558366 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709213.55 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 70921.36 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621712.913399 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638292.20 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 63829.22 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685542.132928 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574462.98 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 57446.30 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742988.430504 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517016.68 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 51701.67 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794690.098323 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465315.01 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 46531.50 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841221.599360 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418783.51 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 41878.35 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883099.950293 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376905.16 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 37690.52 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920790.466132 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339214.64 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 33921.46 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954711.930388 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305293.18 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 30529.32 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985241.248219 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274763.86 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 27476.39 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012717.634266 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247287.47 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 24728.75 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037446.381708 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222558.73 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 22255.87 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059702.254407 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200302.85 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 20030.29 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079732.539835 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180272.57 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 18027.26 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097759.796721 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162245.31 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 16224.53 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113984.327918 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146020.78 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 14602.08 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128586.405995 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131418.70 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 13141.87 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141728.276264 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118276.83 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 11827.68 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153555.959507 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106449.15 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 10644.91 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164200.874425 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95804.23 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 9580.42 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173781.297852 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86223.81 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 8622.38 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182403.678936 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77601.43 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 7760.14 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190163.821911 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69841.29 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 6984.13 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197147.950589 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62857.16 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 6285.72 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203433.666399 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56571.44 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 5657.14 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209090.810629 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50914.30 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 5091.43 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214182.240435 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45822.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 4582.29 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218764.527260 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41240.58 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 4124.06 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222888.585403 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37116.52 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3711.65 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226600.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33404.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230200.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29804.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233800.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26204.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237400.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22604.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241000.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19004.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244600.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15404.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248200.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11804.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251800.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8204.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255400.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4604.87 sec) 05:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2259000.237732 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1004.87 sec) 05:14:00 [GPUGRID] [rr_sim] 2260005.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993902531.28G/439.78G) 05:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851886.02 05:14:00 [---] pick_jobs_to_run() start 05:14:00 [---] pick_jobs_to_run() end 05:14:00 [---] [rr_sim] end 05:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:14:00 [---] [cpu_sched_debug] preliminary job list: 05:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:14:00 [---] [cpu_sched_debug] final job list: 05:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:14:00 [---] [cpu_sched_debug] enforce_run_list: end 05:14:00 [---] choose_project(): 18840.000000 05:14:00 [iThena.Measurements] doing RPC to get work 05:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416759.09 cpu 2161412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:14:00 [---] Got 0 tasks 05:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:15:00 [---] [rr_sim] doing sim: CPU sched 05:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993874940.92G/439.78G) 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259942.37 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 225994.24 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225994.237181 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033948.13 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 203394.81 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429389.050644 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830553.32 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 183055.33 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612444.382760 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647497.99 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 164749.80 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777194.181665 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482748.19 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 148274.82 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925469.000679 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334473.37 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 133447.34 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058916.337792 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1201026.03 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 120102.60 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1179018.941194 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080923.43 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 108092.34 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287111.284255 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972831.09 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 97283.11 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384394.393011 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875547.98 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 87554.80 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471949.190890 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787993.18 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 78799.32 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550748.508982 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709193.86 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 70919.39 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621667.895265 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638274.48 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 63827.45 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685495.342919 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574447.03 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 57444.70 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742940.045808 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517002.33 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 51700.23 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794640.278408 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465302.09 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 46530.21 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841170.487748 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418771.88 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 41877.19 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883047.676154 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376894.70 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 37689.47 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920737.145720 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339205.23 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 33920.52 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954657.668329 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305284.70 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 30528.47 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985186.138677 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274756.23 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 27475.62 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012661.761990 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247280.61 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 24728.06 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037389.822972 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222552.55 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 22255.25 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059645.077855 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200297.29 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 20029.73 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079674.807251 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180267.56 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 18026.76 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097701.563706 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162240.81 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 16224.08 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113925.644517 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146016.73 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 14601.67 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128527.317246 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131415.05 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 13141.51 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141668.822702 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118273.55 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 11827.35 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153496.177613 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106446.19 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 10644.62 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164140.797032 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95801.57 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 9580.16 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173720.954510 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86221.42 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 8622.14 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182343.096240 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77599.28 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 7759.93 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190103.023797 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69839.35 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 6983.93 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197086.958598 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62855.41 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 6285.54 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203372.499919 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56569.87 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 5656.99 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209029.487108 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50912.88 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 5091.29 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214120.775578 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45821.60 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 4582.16 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218702.935201 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41239.44 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 4123.94 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222826.878862 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37115.49 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3711.55 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226538.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33403.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230138.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29803.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233738.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26203.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237338.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22603.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240938.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19003.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244538.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15403.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248138.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11803.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251738.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8203.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255338.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4603.94 sec) 05:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258938.428156 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1003.94 sec) 05:15:00 [GPUGRID] [rr_sim] 2259942.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993874940.92G/439.78G) 05:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851883.28 05:15:00 [---] pick_jobs_to_run() start 05:15:00 [---] pick_jobs_to_run() end 05:15:00 [---] [rr_sim] end 05:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:15:00 [---] [cpu_sched_debug] preliminary job list: 05:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:15:00 [---] [cpu_sched_debug] final job list: 05:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:15:00 [---] [cpu_sched_debug] enforce_run_list: end 05:15:00 [---] choose_project(): 18900.000000 05:15:00 [iThena.Measurements] doing RPC to get work 05:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416699.09 cpu 2161352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:15:00 [---] Got 0 tasks 05:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:16:00 [---] [rr_sim] doing sim: CPU sched 05:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993847350.56G/439.78G) 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259879.63 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 225987.96 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225987.963493 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033891.67 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 203389.17 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429377.130636 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830502.50 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 183050.25 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612427.381065 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647452.25 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 164745.23 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777172.606451 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482707.03 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 148270.70 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925443.309299 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334436.33 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 133443.63 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058886.941861 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200992.69 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 120099.27 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178986.211168 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080893.42 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 108089.34 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287075.553544 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972804.08 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 97280.41 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384355.961682 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875523.67 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 87552.37 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471908.329007 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787971.31 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 78797.13 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550705.459599 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709174.18 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 70917.42 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621622.877131 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638256.76 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 63825.68 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685448.552911 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574431.08 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 57443.11 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742891.661112 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516987.97 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 51698.80 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794590.458494 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465289.18 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 46528.92 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841119.376137 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418760.26 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 41876.03 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882995.402016 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376884.23 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 37688.42 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920683.825307 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339195.81 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 33919.58 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954603.406269 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305276.23 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 30527.62 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985131.029135 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274748.61 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 27474.86 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012605.889714 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247273.75 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 24727.37 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037333.264235 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222546.37 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 22254.64 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059587.901304 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200291.73 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 20029.17 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079617.074666 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180262.56 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 18026.26 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097643.330692 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162236.30 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 16223.63 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113866.961116 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146012.67 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 14601.27 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128468.228497 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131411.41 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 13141.14 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141609.369140 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118270.27 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 11827.03 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153436.395719 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106443.24 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 10644.32 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164080.719639 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95798.92 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 9579.89 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173660.611168 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86219.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 8621.90 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182282.513544 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77597.12 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 7759.71 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190042.225682 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69837.41 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 6983.74 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197025.966607 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62853.67 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 6285.37 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203311.333439 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56568.30 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 5656.83 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208968.163587 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50911.47 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 5091.15 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214059.310721 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45820.32 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 4582.03 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218641.343142 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41238.29 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 4123.83 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222765.172320 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37114.46 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3711.45 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226476.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33403.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230076.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29803.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233676.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26203.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237276.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22603.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240876.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19003.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244476.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15403.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248076.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11803.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251676.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8203.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255276.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4603.02 sec) 05:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258876.618581 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1003.02 sec) 05:16:00 [GPUGRID] [rr_sim] 2259879.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993847350.56G/439.78G) 05:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851880.55 05:16:00 [---] pick_jobs_to_run() start 05:16:00 [---] pick_jobs_to_run() end 05:16:00 [---] [rr_sim] end 05:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:16:00 [---] [cpu_sched_debug] preliminary job list: 05:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:16:00 [---] [cpu_sched_debug] final job list: 05:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:16:00 [---] [cpu_sched_debug] enforce_run_list: end 05:16:00 [---] choose_project(): 18960.000000 05:16:00 [iThena.Measurements] doing RPC to get work 05:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416639.09 cpu 2161292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:16:00 [---] Got 0 tasks 05:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:17:00 [---] [rr_sim] doing sim: CPU sched 05:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993819760.21G/439.78G) 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259816.90 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 225981.69 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225981.689804 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033835.21 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 203383.52 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429365.210628 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830451.69 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 183045.17 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612410.379370 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647406.52 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 164740.65 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777151.031237 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482665.87 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 148266.59 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925417.617918 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334399.28 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 133439.93 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058857.545931 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200959.35 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 120095.94 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178953.481142 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080863.42 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 108086.34 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287039.822832 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972777.08 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 97277.71 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384317.530353 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875499.37 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 87549.94 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471867.467123 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787949.43 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 78794.94 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550662.410215 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709154.49 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 70915.45 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621577.858998 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638239.04 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 63823.90 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685401.762902 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574415.14 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 57441.51 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742843.276417 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516973.62 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 51697.36 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794540.638579 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465276.26 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 46527.63 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841068.264526 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418748.63 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 41874.86 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882943.127878 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376873.77 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 37687.38 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920630.504894 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339186.39 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 33918.64 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954549.144209 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305267.75 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 30526.78 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985075.919593 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274740.98 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 27474.10 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012550.017438 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247266.88 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 24726.69 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037276.705499 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222540.19 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 22254.02 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059530.724753 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200286.17 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 20028.62 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079559.342082 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180257.56 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 18025.76 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097585.097678 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162231.80 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 16223.18 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113808.277715 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146008.62 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 14600.86 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128409.139748 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131407.76 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 13140.78 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141549.915578 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118266.98 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 11826.70 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153376.613824 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106440.28 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 10644.03 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164020.642246 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95796.26 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 9579.63 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173600.267826 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86216.63 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 8621.66 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182221.930848 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77594.97 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 7759.50 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189981.427568 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69835.47 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 6983.55 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196964.974615 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62851.92 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 6285.19 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203250.166958 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56566.73 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 5656.67 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208906.840067 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50910.06 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 5091.01 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213997.845864 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45819.05 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 4581.91 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218579.751082 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41237.15 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 4123.71 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222703.465779 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37113.43 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3711.34 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226414.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33402.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230014.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29802.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233614.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26202.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237214.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22602.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240814.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19002.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244414.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15402.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248014.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11802.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251614.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8202.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255214.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4602.09 sec) 05:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258814.809005 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1002.09 sec) 05:17:00 [GPUGRID] [rr_sim] 2259816.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993819760.21G/439.78G) 05:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851877.81 05:17:00 [---] pick_jobs_to_run() start 05:17:00 [---] pick_jobs_to_run() end 05:17:00 [---] [rr_sim] end 05:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:17:00 [---] [cpu_sched_debug] preliminary job list: 05:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:17:00 [---] [cpu_sched_debug] final job list: 05:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:17:00 [---] [cpu_sched_debug] enforce_run_list: end 05:17:00 [---] choose_project(): 19020.000000 05:17:00 [iThena.Measurements] doing RPC to get work 05:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416579.09 cpu 2161232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:17:00 [---] Got 0 tasks 05:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:18:00 [---] [rr_sim] doing sim: CPU sched 05:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993792169.85G/439.78G) 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259754.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 225975.42 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225975.416116 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033778.75 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 203377.87 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429353.290621 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830400.87 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 183040.09 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612393.377675 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647360.78 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 164736.08 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777129.456024 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482624.71 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 148262.47 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925391.926537 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334362.23 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 133436.22 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058828.150000 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200926.01 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 120092.60 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178920.751116 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080833.41 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 108083.34 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1287004.092121 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972750.07 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 97275.01 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384279.099025 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875475.06 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 87547.51 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471826.605239 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787927.56 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 78792.76 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550619.360831 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709134.80 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 70913.48 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621532.840864 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638221.32 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 63822.13 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685354.972894 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574399.19 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 57439.92 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742794.891721 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516959.27 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 51695.93 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794490.818665 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465263.34 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 46526.33 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841017.152915 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418737.01 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 41873.70 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882890.853739 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376863.31 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 37686.33 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920577.184482 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339176.98 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 33917.70 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954494.882150 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305259.28 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 30525.93 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985020.810051 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274733.35 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 27473.34 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012494.145162 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247260.02 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 24726.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037220.146762 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222534.01 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 22253.40 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059473.548202 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200280.61 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 20028.06 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079501.609498 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180252.55 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 18025.26 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097526.864664 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162227.30 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 16222.73 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113749.594314 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146004.57 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 14600.46 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128350.050999 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131404.11 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 13140.41 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141490.462015 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118263.70 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 11826.37 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153316.831930 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106437.33 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 10643.73 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163960.564853 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95793.60 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 9579.36 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173539.924484 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86214.24 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 8621.42 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182161.348152 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77592.81 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 7759.28 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189920.629453 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69833.53 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 6983.35 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196903.982624 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62850.18 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 6285.02 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203189.000478 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56565.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 5656.52 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208845.516546 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50908.64 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 5090.86 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213936.381008 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45817.78 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 4581.78 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218518.159023 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41236.00 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 4123.60 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222641.759237 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37112.40 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3711.24 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226352.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33401.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229952.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29801.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233552.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26201.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237152.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22601.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240752.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19001.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244352.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15401.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247952.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11801.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251552.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8201.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255152.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4601.16 sec) 05:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258752.999430 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1001.16 sec) 05:18:00 [GPUGRID] [rr_sim] 2259754.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993792169.85G/439.78G) 05:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851875.07 05:18:00 [---] pick_jobs_to_run() start 05:18:00 [---] pick_jobs_to_run() end 05:18:00 [---] [rr_sim] end 05:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:18:00 [---] [cpu_sched_debug] preliminary job list: 05:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:18:00 [---] [cpu_sched_debug] final job list: 05:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:18:00 [---] [cpu_sched_debug] enforce_run_list: end 05:18:00 [---] choose_project(): 19080.000000 05:18:00 [iThena.Measurements] doing RPC to get work 05:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416519.09 cpu 2161172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:18:00 [---] Got 0 tasks 05:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:19:00 [---] [rr_sim] doing sim: CPU sched 05:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993764579.50G/439.78G) 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259691.42 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 225969.14 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225969.142428 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033722.28 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 203372.23 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429341.370613 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830350.05 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 183035.01 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612376.375980 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647315.05 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 164731.50 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777107.880810 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482583.54 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 148258.35 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925366.235157 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334325.19 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 133432.52 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058798.754069 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200892.67 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 120089.27 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178888.021090 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080803.40 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 108080.34 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286968.361409 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972723.06 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 97272.31 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384240.667696 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875450.76 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 87545.08 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471785.743355 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787905.68 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 78790.57 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550576.311447 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709115.11 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 70911.51 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621487.822730 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638203.60 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 63820.36 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685308.182885 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574383.24 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 57438.32 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742746.507025 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516944.92 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 51694.49 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794440.998750 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465250.43 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 46525.04 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840966.041303 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418725.38 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 41872.54 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882838.579601 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376852.84 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 37685.28 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920523.864069 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339167.56 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 33916.76 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954440.620090 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305250.80 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 30525.08 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984965.700509 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274725.72 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 27472.57 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012438.272886 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247253.15 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 24725.32 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037163.588025 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222527.84 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 22252.78 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059416.371651 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200275.05 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 20027.51 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079443.876914 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180247.55 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 18024.75 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097468.631650 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162222.79 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 16222.28 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113690.910913 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146000.51 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 14600.05 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128290.962250 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131400.46 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 13140.05 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141431.008453 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118260.42 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 11826.04 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153257.050036 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106434.37 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 10643.44 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163900.487460 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95790.94 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 9579.09 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173479.581142 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86211.84 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 8621.18 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182100.765456 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77590.66 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 7759.07 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189859.831338 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69831.59 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 6983.16 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196842.990632 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62848.43 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 6284.84 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203127.833997 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56563.59 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 5656.36 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208784.193025 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50907.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 5090.72 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213874.916151 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45816.51 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 4581.65 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218456.566964 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41234.86 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 4123.49 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222580.052695 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37111.37 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3711.14 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226291.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33400.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229891.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29800.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233491.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26200.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237091.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22600.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240691.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19000.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244291.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15400.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247891.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11800.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251491.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8200.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255091.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4600.23 sec) 05:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258691.189854 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1000.23 sec) 05:19:00 [GPUGRID] [rr_sim] 2259691.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993764579.50G/439.78G) 05:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851872.34 05:19:00 [---] pick_jobs_to_run() start 05:19:00 [---] pick_jobs_to_run() end 05:19:00 [---] [rr_sim] end 05:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:19:00 [---] [cpu_sched_debug] preliminary job list: 05:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:19:00 [---] [cpu_sched_debug] final job list: 05:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:19:00 [---] [cpu_sched_debug] enforce_run_list: end 05:19:00 [---] choose_project(): 19140.000000 05:19:00 [iThena.Measurements] doing RPC to get work 05:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416459.09 cpu 2161112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:19:00 [---] Got 0 tasks 05:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:20:00 [---] [rr_sim] doing sim: CPU sched 05:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993736989.14G/439.78G) 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259628.69 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 225962.87 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225962.868740 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033665.82 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 203366.58 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429329.450606 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830299.24 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 183029.92 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612359.374285 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647269.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 164726.93 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777086.305596 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482542.38 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 148254.24 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925340.543776 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334288.14 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 133428.81 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058769.358138 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200859.33 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 120085.93 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178855.291064 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080773.40 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 108077.34 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286932.630698 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972696.06 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 97269.61 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384202.236368 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875426.45 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 87542.65 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471744.881471 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787883.81 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 78788.38 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550533.262063 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709095.43 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 70909.54 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621442.804597 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638185.88 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 63818.59 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685261.392877 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574367.29 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 57436.73 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742698.122329 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516930.57 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 51693.06 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794391.178836 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465237.51 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 46523.75 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840914.929692 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418713.76 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 41871.38 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882786.305463 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376842.38 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 37684.24 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920470.543656 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339158.14 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 33915.81 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954386.358030 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305242.33 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 30524.23 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984910.590967 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274718.10 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 27471.81 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012382.400610 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247246.29 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 24724.63 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037107.029289 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222521.66 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 22252.17 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059359.195100 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200269.49 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 20026.95 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079386.144330 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180242.54 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 18024.25 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097410.398636 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162218.29 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 16221.83 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113632.227513 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145996.46 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 14599.65 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128231.873501 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131396.81 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 13139.68 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141371.554891 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118257.13 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 11825.71 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153197.268141 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106431.42 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 10643.14 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163840.410067 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95788.28 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 9578.83 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173419.237800 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86209.45 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 8620.94 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182040.182760 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77588.50 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 7758.85 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189799.033224 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69829.65 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 6982.97 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196781.998641 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62846.69 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 6284.67 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203066.667517 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56562.02 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 5656.20 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208722.869505 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50905.82 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 5090.58 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213813.451294 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45815.24 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 4581.52 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218394.974905 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41233.71 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 4123.37 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222518.346154 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37110.34 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3711.03 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226229.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33399.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229829.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29799.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233429.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26199.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237029.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22599.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240629.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18999.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244229.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15399.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247829.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11799.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251429.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8199.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255029.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4599.31 sec) 05:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258629.380278 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 999.31 sec) 05:20:00 [GPUGRID] [rr_sim] 2259628.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993736989.14G/439.78G) 05:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851869.60 05:20:00 [---] pick_jobs_to_run() start 05:20:00 [---] pick_jobs_to_run() end 05:20:00 [---] [rr_sim] end 05:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:20:00 [---] [cpu_sched_debug] preliminary job list: 05:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:20:00 [---] [cpu_sched_debug] final job list: 05:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:20:00 [---] [cpu_sched_debug] enforce_run_list: end 05:20:00 [---] choose_project(): 19200.000000 05:20:00 [iThena.Measurements] doing RPC to get work 05:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416399.09 cpu 2161052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2161052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2161052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:20:00 [---] Got 0 tasks 05:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:21:00 [---] [rr_sim] doing sim: CPU sched 05:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993709398.78G/439.78G) 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259565.95 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 225956.60 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225956.595052 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033609.36 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 203360.94 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429317.530598 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830248.42 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 183024.84 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612342.372590 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647223.58 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 164722.36 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777064.730382 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482501.22 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 148250.12 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925314.852396 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334251.10 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 133425.11 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058739.962208 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200825.99 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 120082.60 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178822.561038 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080743.39 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 108074.34 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286896.899986 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972669.05 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 97266.91 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384163.805039 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875402.15 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 87540.21 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471704.019587 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787861.93 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 78786.19 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550490.212680 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709075.74 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 70907.57 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621397.786463 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638168.16 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 63816.82 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685214.602868 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574351.35 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 57435.13 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742649.737633 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516916.21 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 51691.62 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794341.358921 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465224.59 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 46522.46 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840863.818081 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418702.13 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 41870.21 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882734.031324 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376831.92 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 37683.19 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920417.223243 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339148.73 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 33914.87 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954332.095971 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305233.85 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 30523.39 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984855.481425 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274710.47 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 27471.05 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012326.528334 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247239.42 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 24723.94 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037050.470552 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222515.48 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 22251.55 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059302.018549 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200263.93 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 20026.39 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079328.411745 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180237.54 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 18023.75 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097352.165622 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162213.78 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 16221.38 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113573.544112 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145992.41 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 14599.24 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128172.784752 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131393.17 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 13139.32 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141312.101328 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118253.85 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 11825.38 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153137.486247 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106428.46 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 10642.85 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163780.332674 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95785.62 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 9578.56 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173358.894458 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86207.06 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 8620.71 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181979.600064 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77586.35 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 7758.64 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189738.235109 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69827.72 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 6982.77 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196721.006650 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62844.94 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 6284.49 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203005.501036 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56560.45 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 5656.04 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208661.545984 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50904.40 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 5090.44 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213751.986437 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45813.96 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 4581.40 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218333.382845 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41232.57 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 4123.26 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222456.639612 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37109.31 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3710.93 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226167.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33398.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229767.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29798.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233367.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26198.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236967.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22598.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240567.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18998.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244167.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15398.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247767.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11798.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251367.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8198.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254967.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4598.38 sec) 05:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258567.570703 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 998.38 sec) 05:21:00 [GPUGRID] [rr_sim] 2259565.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993709398.78G/439.78G) 05:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851866.86 05:21:00 [---] pick_jobs_to_run() start 05:21:00 [---] pick_jobs_to_run() end 05:21:00 [---] [rr_sim] end 05:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:21:00 [---] [cpu_sched_debug] preliminary job list: 05:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:21:00 [---] [cpu_sched_debug] final job list: 05:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:21:00 [---] [cpu_sched_debug] enforce_run_list: end 05:21:00 [---] choose_project(): 19260.000000 05:21:00 [iThena.Measurements] doing RPC to get work 05:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416339.09 cpu 2160992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:21:00 [---] Got 0 tasks 05:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:22:00 [---] [rr_sim] doing sim: CPU sched 05:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993681808.43G/439.78G) 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259503.21 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 225950.32 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225950.321363 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033552.89 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 203355.29 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429305.610590 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830197.60 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 183019.76 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612325.370895 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647177.84 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 164717.78 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777043.155169 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482460.06 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 148246.01 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925289.161015 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334214.05 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 133421.41 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058710.566277 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200792.65 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 120079.26 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178789.831013 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080713.38 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 108071.34 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286861.169275 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972642.04 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 97264.20 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384125.373710 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875377.84 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 87537.78 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471663.157703 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787840.06 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 78784.01 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550447.163296 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709056.05 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 70905.61 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621352.768330 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638150.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 63815.04 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685167.812860 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574335.40 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 57433.54 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742601.352937 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516901.86 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 51690.19 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794291.539007 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465211.67 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 46521.17 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840812.706470 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418690.51 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 41869.05 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882681.757186 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376821.46 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 37682.15 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920363.902831 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339139.31 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 33913.93 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954277.833911 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305225.38 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 30522.54 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984800.371883 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274702.84 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 27470.28 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012270.656058 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247232.56 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 24723.26 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036993.911816 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222509.30 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 22250.93 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059244.841998 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200258.37 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 20025.84 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079270.679161 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180232.53 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 18023.25 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097293.932608 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162209.28 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 16220.93 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113514.860711 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145988.35 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 14598.84 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128113.696003 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131389.52 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 13138.95 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141252.647766 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118250.57 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 11825.06 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153077.704353 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106425.51 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 10642.55 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163720.255281 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95782.96 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 9578.30 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173298.551116 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86204.66 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 8620.47 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181919.017368 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77584.20 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 7758.42 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189677.436994 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69825.78 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 6982.58 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196660.014658 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62843.20 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 6284.32 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202944.334556 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56558.88 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 5655.89 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208600.222464 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50902.99 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 5090.30 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213690.521581 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45812.69 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 4581.27 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218271.790786 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41231.42 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 4123.14 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222394.933071 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37108.28 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3710.83 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226105.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33397.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229705.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29797.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233305.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26197.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236905.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22597.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240505.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18997.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244105.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15397.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247705.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11797.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251305.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8197.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254905.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4597.45 sec) 05:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258505.761127 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 997.45 sec) 05:22:00 [GPUGRID] [rr_sim] 2259503.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993681808.43G/439.78G) 05:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851864.13 05:22:00 [---] pick_jobs_to_run() start 05:22:00 [---] pick_jobs_to_run() end 05:22:00 [---] [rr_sim] end 05:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:22:00 [---] [cpu_sched_debug] preliminary job list: 05:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:22:00 [---] [cpu_sched_debug] final job list: 05:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:22:00 [---] [cpu_sched_debug] enforce_run_list: end 05:22:00 [---] choose_project(): 19320.000000 05:22:00 [iThena.Measurements] doing RPC to get work 05:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416279.09 cpu 2160932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:22:00 [---] Got 0 tasks 05:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:23:00 [---] [rr_sim] doing sim: CPU sched 05:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993654218.07G/439.78G) 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259440.48 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 225944.05 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225944.047675 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033496.43 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 203349.64 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429293.690583 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830146.79 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 183014.68 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612308.369200 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647132.11 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 164713.21 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777021.579955 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482418.90 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 148241.89 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925263.469634 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334177.01 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 133417.70 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058681.170346 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200759.31 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 120075.93 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178757.100987 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080683.38 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 108068.34 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286825.438563 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972615.04 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 97261.50 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384086.942382 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875353.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 87535.35 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471622.295819 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787818.18 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 78781.82 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550404.113912 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709036.36 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 70903.64 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621307.750196 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638132.73 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 63813.27 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685121.022852 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574319.45 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 57431.95 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742552.968241 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516887.51 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 51688.75 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794241.719092 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465198.76 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 46519.88 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840761.594858 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418678.88 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 41867.89 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882629.483048 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376810.99 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 37681.10 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920310.582418 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339129.89 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 33912.99 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954223.571851 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305216.90 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 30521.69 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984745.262341 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274695.21 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 27469.52 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012214.783782 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247225.69 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 24722.57 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036937.353079 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222503.12 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 22250.31 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059187.665446 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200252.81 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 20025.28 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079212.946577 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180227.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 18022.75 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097235.699594 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162204.78 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 16220.48 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113456.177310 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145984.30 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 14598.43 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128054.607254 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131385.87 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 13138.59 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141193.194204 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118247.28 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 11824.73 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153017.922459 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106422.55 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 10642.26 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163660.177888 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95780.30 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 9578.03 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173238.207774 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86202.27 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 8620.23 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181858.434672 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77582.04 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 7758.20 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189616.638880 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69823.84 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 6982.38 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196599.022667 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62841.45 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 6284.15 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202883.168075 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56557.31 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 5655.73 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208538.898943 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50901.58 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 5090.16 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213629.056724 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45811.42 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 4581.14 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218210.198727 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41230.28 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 4123.03 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222333.226529 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37107.25 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3710.73 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226043.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33396.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229643.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29796.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233243.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26196.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236843.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22596.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240443.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18996.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244043.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15396.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247643.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11796.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251243.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8196.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254843.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4596.53 sec) 05:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258443.951551 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 996.53 sec) 05:23:00 [GPUGRID] [rr_sim] 2259440.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993654218.07G/439.78G) 05:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851861.39 05:23:00 [---] pick_jobs_to_run() start 05:23:00 [---] pick_jobs_to_run() end 05:23:00 [---] [rr_sim] end 05:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:23:00 [---] [cpu_sched_debug] preliminary job list: 05:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:23:00 [---] [cpu_sched_debug] final job list: 05:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:23:00 [---] [cpu_sched_debug] enforce_run_list: end 05:23:00 [---] choose_project(): 19380.000000 05:23:00 [iThena.Measurements] doing RPC to get work 05:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 589019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416219.09 cpu 2160872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:23:00 [---] Got 0 tasks 05:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:24:00 [---] [rr_sim] doing sim: CPU sched 05:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993626627.72G/439.78G) 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259377.74 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 225937.77 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225937.773987 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033439.97 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 203344.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429281.770575 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830095.97 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 183009.60 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612291.367505 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647086.37 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 164708.64 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 777000.004741 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482377.74 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 148237.77 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925237.778254 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334139.96 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 133414.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058651.774415 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200725.97 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 120072.60 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178724.370961 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080653.37 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 108065.34 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286789.707852 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972588.03 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 97258.80 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384048.511053 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875329.23 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 87532.92 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471581.433935 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787796.31 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 78779.63 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550361.064528 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709016.68 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 70901.67 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621262.732062 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638115.01 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 63811.50 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685074.232843 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574303.51 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 57430.35 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742504.583546 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516873.16 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 51687.32 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794191.899178 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465185.84 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 46518.58 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840710.483247 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418667.26 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 41866.73 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882577.208909 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376800.53 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 37680.05 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920257.262005 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339120.48 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 33912.05 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954169.309792 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305208.43 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 30520.84 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984690.152799 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274687.59 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 27468.76 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012158.911506 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247218.83 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 24721.88 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036880.794343 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222496.95 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 22249.69 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059130.488895 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200247.25 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 20024.73 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079155.213993 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180222.53 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 18022.25 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097177.466580 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162200.27 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 16220.03 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113397.493909 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145980.25 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 14598.02 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127995.518505 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131382.22 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 13138.22 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141133.740642 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118244.00 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 11824.40 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152958.140564 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106419.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 10641.96 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163600.100495 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95777.64 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 9577.76 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173177.864432 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86199.88 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 8619.99 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181797.851976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77579.89 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 7757.99 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189555.840765 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69821.90 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 6982.19 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196538.030676 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62839.71 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 6283.97 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202822.001595 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56555.74 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 5655.57 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208477.575422 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50900.16 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 5090.02 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213567.591867 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45810.15 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 4581.01 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218148.606667 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41229.13 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 4122.91 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222271.519987 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37106.22 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3710.62 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225982.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33395.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229582.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29795.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233182.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26195.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236782.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22595.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240382.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18995.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243982.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15395.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247582.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11795.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251182.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8195.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254782.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4595.60 sec) 05:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258382.141976 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 995.60 sec) 05:24:00 [GPUGRID] [rr_sim] 2259377.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993626627.72G/439.78G) 05:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851858.65 05:24:00 [---] pick_jobs_to_run() start 05:24:00 [---] pick_jobs_to_run() end 05:24:00 [---] [rr_sim] end 05:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:24:00 [---] [cpu_sched_debug] preliminary job list: 05:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:24:00 [---] [cpu_sched_debug] final job list: 05:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:24:00 [---] [cpu_sched_debug] enforce_run_list: end 05:24:00 [---] choose_project(): 19440.000000 05:24:00 [iThena.Measurements] doing RPC to get work 05:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416159.09 cpu 2160812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:24:00 [---] Got 0 tasks 05:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:25:00 [---] [rr_sim] doing sim: CPU sched 05:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993599037.36G/439.78G) 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259315.00 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 225931.50 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225931.500299 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033383.50 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 203338.35 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429269.850568 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1830045.15 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 183004.52 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612274.365809 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1647040.64 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 164704.06 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776978.429527 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482336.57 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 148233.66 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925212.086873 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334102.92 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 133410.29 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058622.378485 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200692.62 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 120069.26 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178691.640935 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080623.36 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 108062.34 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286753.977140 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972561.03 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 97256.10 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1384010.079725 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875304.92 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 87530.49 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471540.572051 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787774.43 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 78777.44 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550318.015145 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708996.99 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 70899.70 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621217.713929 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638097.29 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 63809.73 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1685027.442835 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574287.56 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 57428.76 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742456.198850 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516858.80 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 51685.88 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794142.079264 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465172.92 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 46517.29 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840659.371636 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418655.63 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 41865.56 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882524.934771 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376790.07 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 37679.01 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920203.941593 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339111.06 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 33911.11 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954115.047732 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305199.96 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 30520.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984635.043258 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274679.96 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 27468.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012103.039230 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247211.96 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 24721.20 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036824.235606 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222490.77 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 22249.08 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059073.312344 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200241.69 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 20024.17 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079097.481408 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180217.52 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 18021.75 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097119.233566 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162195.77 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 16219.58 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113338.810508 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145976.19 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 14597.62 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127936.429756 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131378.57 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 13137.86 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141074.287079 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118240.72 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 11824.07 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152898.358670 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106416.64 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 10641.66 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163540.023102 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95774.98 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 9577.50 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173117.521090 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86197.48 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 8619.75 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181737.269280 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77577.73 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 7757.77 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189495.042651 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69819.96 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 6982.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196477.038684 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62837.96 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 6283.80 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202760.835115 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56554.17 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 5655.42 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208416.251902 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50898.75 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 5089.88 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213506.127010 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45808.88 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 4580.89 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218087.014608 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41227.99 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 4122.80 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222209.813446 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37105.19 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3710.52 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225920.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33394.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229520.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29794.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233120.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26194.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236720.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22594.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240320.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18994.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243920.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15394.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247520.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11794.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251120.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8194.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254720.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4594.67 sec) 05:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258320.332400 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 994.67 sec) 05:25:00 [GPUGRID] [rr_sim] 2259315.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993599037.36G/439.78G) 05:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851855.92 05:25:00 [---] pick_jobs_to_run() start 05:25:00 [---] pick_jobs_to_run() end 05:25:00 [---] [rr_sim] end 05:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:25:00 [---] [cpu_sched_debug] preliminary job list: 05:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:25:00 [---] [cpu_sched_debug] final job list: 05:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:25:00 [---] [cpu_sched_debug] enforce_run_list: end 05:25:00 [---] choose_project(): 19500.000000 05:25:00 [iThena.Measurements] doing RPC to get work 05:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416099.09 cpu 2160752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:25:00 [---] Got 0 tasks 05:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:26:00 [---] [rr_sim] doing sim: CPU sched 05:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993571447.00G/439.78G) 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259252.27 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 225925.23 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225925.226610 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033327.04 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 203332.70 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429257.930560 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829994.34 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 182999.43 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612257.364114 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646994.90 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 164699.49 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776956.854313 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482295.41 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 148229.54 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925186.395493 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334065.87 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 133406.59 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058592.982554 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200659.28 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 120065.93 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178658.910909 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080593.36 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 108059.34 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286718.246428 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972534.02 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 97253.40 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383971.648396 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875280.62 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 87528.06 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471499.710167 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787752.56 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 78775.26 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550274.965761 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708977.30 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 70897.73 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621172.695795 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638079.57 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 63807.96 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684980.652826 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574271.61 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 57427.16 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742407.814154 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516844.45 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 51684.45 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794092.259349 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465160.01 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 46516.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840608.260025 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418644.01 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 41864.40 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882472.660633 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376779.61 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 37677.96 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920150.621180 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339101.64 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 33910.16 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954060.785672 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305191.48 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 30519.15 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984579.933716 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274672.33 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 27467.23 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012047.166955 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247205.10 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 24720.51 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036767.676870 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222484.59 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 22248.46 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059016.135793 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200236.13 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 20023.61 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079039.748824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180212.52 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 18021.25 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097061.000552 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162191.27 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 16219.13 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113280.127108 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145972.14 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 14597.21 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127877.341007 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131374.93 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 13137.49 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141014.833517 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118237.43 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 11823.74 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152838.576776 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106413.69 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 10641.37 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163479.945709 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95772.32 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 9577.23 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173057.177748 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86195.09 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 8619.51 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181676.686584 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77575.58 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 7757.56 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189434.244536 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69818.02 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 6981.80 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196416.046693 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62836.22 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 6283.62 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202699.668634 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56552.60 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 5655.26 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208354.928381 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50897.34 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 5089.73 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213444.662154 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45807.60 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 4580.76 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218025.422549 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41226.84 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 4122.68 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222148.106904 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37104.16 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3710.42 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225858.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33393.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229458.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29793.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233058.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26193.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236658.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22593.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240258.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18993.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243858.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15393.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247458.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11793.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251058.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8193.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254658.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4593.74 sec) 05:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258258.522824 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 993.74 sec) 05:26:00 [GPUGRID] [rr_sim] 2259252.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993571447.00G/439.78G) 05:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851853.18 05:26:00 [---] pick_jobs_to_run() start 05:26:00 [---] pick_jobs_to_run() end 05:26:00 [---] [rr_sim] end 05:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:26:00 [---] [cpu_sched_debug] preliminary job list: 05:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:26:00 [---] [cpu_sched_debug] final job list: 05:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:26:00 [---] [cpu_sched_debug] enforce_run_list: end 05:26:00 [---] choose_project(): 19560.000000 05:26:00 [iThena.Measurements] doing RPC to get work 05:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 416039.09 cpu 2160692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:26:00 [---] Got 0 tasks 05:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:27:00 [---] [rr_sim] doing sim: CPU sched 05:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993543856.65G/439.78G) 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259189.53 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 225918.95 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225918.952922 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033270.58 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 203327.06 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429246.010552 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829943.52 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 182994.35 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612240.362419 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646949.17 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 164694.92 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776935.279100 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482254.25 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 148225.43 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925160.704112 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1334028.83 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 133402.88 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058563.586623 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200625.94 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 120062.59 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178626.180883 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080563.35 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 108056.33 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286682.515717 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972507.01 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 97250.70 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383933.217068 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875256.31 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 87525.63 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471458.848283 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787730.68 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 78773.07 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550231.916377 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708957.61 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 70895.76 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621127.677662 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638061.85 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 63806.19 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684933.862818 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574255.67 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 57425.57 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742359.429458 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516830.10 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 51683.01 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794042.439435 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465147.09 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 46514.71 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840557.148413 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418632.38 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 41863.24 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882420.386494 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376769.14 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 37676.91 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920097.300767 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339092.23 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 33909.22 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954006.523613 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305183.01 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 30518.30 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984524.824174 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274664.71 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 27466.47 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011991.294679 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247198.23 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 24719.82 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036711.118133 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222478.41 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 22247.84 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058958.959242 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200230.57 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 20023.06 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078982.016240 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180207.51 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 18020.75 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097002.767538 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162186.76 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 16218.68 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113221.443707 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145968.09 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 14596.81 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127818.252258 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131371.28 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 13137.13 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140955.379955 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118234.15 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 11823.41 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152778.794882 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106410.73 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 10641.07 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163419.868316 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95769.66 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 9576.97 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172996.834406 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86192.69 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 8619.27 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181616.103888 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77573.43 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 7757.34 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189373.446421 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69816.08 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 6981.61 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196355.054702 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62834.47 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 6283.45 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202638.502154 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56551.03 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 5655.10 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208293.604861 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50895.92 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 5089.59 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213383.197297 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45806.33 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 4580.63 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217963.830489 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41225.70 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 4122.57 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222086.400363 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37103.13 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3710.31 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225796.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33392.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229396.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29792.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232996.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26192.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236596.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22592.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240196.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18992.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243796.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15392.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247396.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11792.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250996.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8192.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254596.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4592.82 sec) 05:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258196.713249 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 992.82 sec) 05:27:00 [GPUGRID] [rr_sim] 2259189.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993543856.65G/439.78G) 05:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851850.44 05:27:00 [---] pick_jobs_to_run() start 05:27:00 [---] pick_jobs_to_run() end 05:27:00 [---] [rr_sim] end 05:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:27:00 [---] [cpu_sched_debug] preliminary job list: 05:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:27:00 [---] [cpu_sched_debug] final job list: 05:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:27:00 [---] [cpu_sched_debug] enforce_run_list: end 05:27:00 [---] choose_project(): 19620.000000 05:27:00 [iThena.Measurements] doing RPC to get work 05:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415979.09 cpu 2160632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:27:00 [---] Got 0 tasks 05:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:28:00 [---] [rr_sim] doing sim: CPU sched 05:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993516266.29G/439.78G) 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259126.79 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 225912.68 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225912.679234 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033214.11 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 203321.41 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429234.090545 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829892.70 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 182989.27 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612223.360724 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646903.43 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 164690.34 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776913.703886 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482213.09 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 148221.31 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925135.012731 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333991.78 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 133399.18 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058534.190692 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200592.60 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 120059.26 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178593.450857 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080533.34 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 108053.33 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286646.785005 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972480.01 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 97248.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383894.785739 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875232.01 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 87523.20 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471417.986399 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787708.81 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 78770.88 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550188.866993 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708937.93 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 70893.79 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621082.659528 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638044.13 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 63804.41 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684887.072809 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574239.72 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 57423.97 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742311.044762 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516815.75 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 51681.57 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793992.619520 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465134.17 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 46513.42 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840506.036802 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418620.76 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 41862.08 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882368.112356 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376758.68 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 37675.87 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920043.980354 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339082.81 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 33908.28 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953952.261553 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305174.53 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 30517.45 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984469.714632 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274657.08 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 27465.71 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011935.422403 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247191.37 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 24719.14 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036654.559396 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222472.23 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 22247.22 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058901.782691 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200225.01 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 20022.50 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078924.283656 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180202.51 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 18020.25 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096944.534524 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162182.26 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 16218.23 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113162.760306 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145964.03 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 14596.40 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127759.163509 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131367.63 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 13136.76 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140895.926392 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118230.87 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 11823.09 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152719.012987 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106407.78 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 10640.78 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163359.790923 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95767.00 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 9576.70 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172936.491064 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86190.30 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 8619.03 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181555.521192 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77571.27 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 7757.13 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189312.648307 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69814.14 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 6981.41 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196294.062710 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62832.73 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 6283.27 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202577.335673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56549.46 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 5654.95 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208232.281340 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50894.51 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 5089.45 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213321.732440 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45805.06 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 4580.51 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217902.238430 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41224.55 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 4122.46 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222024.693821 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37102.10 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3710.21 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225734.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33391.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229334.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29791.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232934.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26191.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236534.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22591.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240134.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18991.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243734.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15391.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247334.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11791.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250934.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8191.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254534.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4591.89 sec) 05:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258134.903673 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 991.89 sec) 05:28:00 [GPUGRID] [rr_sim] 2259126.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993516266.29G/439.78G) 05:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851847.71 05:28:00 [---] pick_jobs_to_run() start 05:28:00 [---] pick_jobs_to_run() end 05:28:00 [---] [rr_sim] end 05:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:28:00 [---] [cpu_sched_debug] preliminary job list: 05:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:28:00 [---] [cpu_sched_debug] final job list: 05:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:28:00 [---] [cpu_sched_debug] enforce_run_list: end 05:28:00 [---] choose_project(): 19680.000000 05:28:00 [iThena.Measurements] doing RPC to get work 05:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415919.09 cpu 2160572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:28:00 [---] Got 0 tasks 05:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:29:00 [---] [rr_sim] doing sim: CPU sched 05:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993488675.93G/439.78G) 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259064.06 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 225906.41 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225906.405546 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033157.65 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 203315.76 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429222.170537 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829841.88 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 182984.19 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612206.359029 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646857.70 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 164685.77 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776892.128672 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482171.93 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 148217.19 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925109.321351 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333954.73 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 133395.47 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058504.794761 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200559.26 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 120055.93 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178560.720831 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080503.33 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 108050.33 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286611.054294 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972453.00 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 97245.30 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383856.354410 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875207.70 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 87520.77 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471377.124515 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787686.93 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 78768.69 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550145.817609 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708918.24 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 70891.82 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1621037.641394 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638026.41 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 63802.64 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684840.282801 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574223.77 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 57422.38 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742262.660066 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516801.40 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 51680.14 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793942.799606 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465121.26 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 46512.13 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840454.925191 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418609.13 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 41860.91 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882315.838218 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376748.22 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 37674.82 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919990.659942 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339073.40 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 33907.34 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953897.999493 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305166.06 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 30516.61 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984414.605090 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274649.45 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 27464.95 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011879.550127 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247184.51 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 24718.45 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036598.000660 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222466.05 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 22246.61 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058844.606140 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200219.45 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 20021.94 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078866.551072 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180197.50 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 18019.75 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096886.301510 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162177.75 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 16217.78 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113104.076905 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145959.98 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 14596.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127700.074760 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131363.98 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 13136.40 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140836.472830 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118227.58 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 11822.76 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152659.231093 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106404.82 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 10640.48 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163299.713529 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95764.34 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 9576.43 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172876.147722 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86187.91 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 8618.79 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181494.938496 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77569.12 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 7756.91 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189251.850192 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69812.21 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 6981.22 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196233.070719 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62830.98 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 6283.10 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202516.169193 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56547.89 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 5654.79 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208170.957819 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50893.10 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 5089.31 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213260.267583 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45803.79 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 4580.38 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217840.646371 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41223.41 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 4122.34 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221962.987279 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37101.07 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3710.11 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225673.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33390.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229273.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29790.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232873.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26190.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236473.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22590.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240073.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18990.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243673.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15390.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247273.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11790.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250873.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8190.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254473.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4590.96 sec) 05:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258073.094097 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 990.96 sec) 05:29:00 [GPUGRID] [rr_sim] 2259064.06: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993488675.93G/439.78G) 05:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851844.97 05:29:00 [---] pick_jobs_to_run() start 05:29:00 [---] pick_jobs_to_run() end 05:29:00 [---] [rr_sim] end 05:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:29:00 [---] [cpu_sched_debug] preliminary job list: 05:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:29:00 [---] [cpu_sched_debug] final job list: 05:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:29:00 [---] [cpu_sched_debug] enforce_run_list: end 05:29:00 [---] choose_project(): 19740.000000 05:29:00 [iThena.Measurements] doing RPC to get work 05:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415859.09 cpu 2160512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:29:00 [---] Got 0 tasks 05:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:30:00 [---] [rr_sim] doing sim: CPU sched 05:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993461085.58G/439.78G) 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2259001.32 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 225900.13 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225900.131858 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033101.19 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 203310.12 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429210.250529 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829791.07 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 182979.11 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612189.357334 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646811.96 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 164681.20 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776870.553458 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482130.77 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 148213.08 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925083.629970 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333917.69 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 133391.77 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058475.398831 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200525.92 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 120052.59 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178527.990805 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080473.33 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 108047.33 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286575.323582 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972425.99 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 97242.60 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383817.923082 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875183.40 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 87518.34 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471336.262631 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787665.06 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 78766.51 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550102.768226 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708898.55 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 70889.86 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620992.623261 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638008.70 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 63800.87 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684793.492792 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574207.83 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 57420.78 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742214.275371 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516787.04 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 51678.70 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793892.979691 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465108.34 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 46510.83 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840403.813580 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418597.50 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 41859.75 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882263.564079 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376737.75 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 37673.78 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919937.339529 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339063.98 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 33906.40 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953843.737434 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305157.58 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 30515.76 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984359.495548 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274641.82 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 27464.18 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011823.677851 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247177.64 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 24717.76 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036541.441923 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222459.88 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 22245.99 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058787.429589 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200213.89 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 20021.39 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078808.818487 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180192.50 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 18019.25 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096828.068496 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162173.25 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 16217.33 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113045.393504 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145955.93 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 14595.59 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127640.986011 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131360.33 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 13136.03 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140777.019268 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118224.30 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 11822.43 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152599.449199 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106401.87 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 10640.19 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163239.636136 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95761.68 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 9576.17 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172815.804380 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86185.51 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 8618.55 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181434.355800 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77566.96 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 7756.70 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189191.052078 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69810.27 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 6981.03 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196172.078727 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62829.24 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 6282.92 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202455.002712 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56546.32 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 5654.63 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208109.634299 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50891.68 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 5089.17 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213198.802726 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45802.52 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 4580.25 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217779.054311 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41222.26 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 4122.23 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221901.280738 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37100.04 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3710.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225611.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33390.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229211.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29790.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232811.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26190.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236411.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22590.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240011.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18990.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243611.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15390.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247211.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11790.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250811.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8190.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254411.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4590.03 sec) 05:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2258011.284522 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 990.03 sec) 05:30:00 [GPUGRID] [rr_sim] 2259001.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993461085.58G/439.78G) 05:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851842.23 05:30:00 [---] pick_jobs_to_run() start 05:30:00 [---] pick_jobs_to_run() end 05:30:00 [---] [rr_sim] end 05:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:30:00 [---] [cpu_sched_debug] preliminary job list: 05:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:30:00 [---] [cpu_sched_debug] final job list: 05:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:30:00 [---] [cpu_sched_debug] enforce_run_list: end 05:30:00 [---] choose_project(): 19800.000000 05:30:00 [iThena.Measurements] doing RPC to get work 05:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415799.09 cpu 2160452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:30:00 [---] Got 0 tasks 05:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:31:00 [---] [rr_sim] doing sim: CPU sched 05:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993433495.22G/439.78G) 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258938.58 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 225893.86 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225893.858169 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033044.72 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 203304.47 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429198.330522 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829740.25 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 182974.03 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612172.355639 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646766.23 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 164676.62 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776848.978245 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482089.60 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 148208.96 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925057.938589 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333880.64 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 133388.06 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058446.002900 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200492.58 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 120049.26 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178495.260779 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080443.32 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 108044.33 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286539.592871 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972398.99 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 97239.90 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383779.491753 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875159.09 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 87515.91 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471295.400747 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787643.18 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 78764.32 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550059.718842 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708878.86 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 70887.89 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620947.605127 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637990.98 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 63799.10 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684746.702784 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574191.88 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 57419.19 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742165.890675 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516772.69 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 51677.27 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793843.159777 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465095.42 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 46509.54 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840352.701968 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418585.88 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 41858.59 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882211.289941 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376727.29 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 37672.73 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919884.019116 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339054.56 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 33905.46 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953789.475374 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305149.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 30514.91 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984304.386006 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274634.20 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 27463.42 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011767.805575 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247170.78 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 24717.08 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036484.883187 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222453.70 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 22245.37 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058730.253037 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200208.33 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 20020.83 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078751.085903 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180187.50 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 18018.75 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096769.835482 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162168.75 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 16216.87 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112986.710103 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145951.87 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 14595.19 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127581.897262 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131356.68 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 13135.67 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140717.565706 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118221.02 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 11822.10 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152539.667304 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106398.91 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 10639.89 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163179.558743 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95759.02 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 9575.90 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172755.461038 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86183.12 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 8618.31 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181373.773104 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77564.81 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 7756.48 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189130.253963 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69808.33 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 6980.83 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196111.086736 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62827.49 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 6282.75 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202393.836232 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56544.75 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 5654.47 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208048.310778 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50890.27 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 5089.03 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213137.337870 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45801.24 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 4580.12 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217717.462252 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41221.12 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 4122.11 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221839.574196 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37099.01 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3709.90 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225549.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33389.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229149.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29789.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232749.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26189.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236349.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22589.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239949.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18989.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243549.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15389.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247149.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11789.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250749.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8189.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254349.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4589.11 sec) 05:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257949.474946 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 989.11 sec) 05:31:00 [GPUGRID] [rr_sim] 2258938.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993433495.22G/439.78G) 05:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851839.49 05:31:00 [---] pick_jobs_to_run() start 05:31:00 [---] pick_jobs_to_run() end 05:31:00 [---] [rr_sim] end 05:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:31:00 [---] [cpu_sched_debug] preliminary job list: 05:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:31:00 [---] [cpu_sched_debug] final job list: 05:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:31:00 [---] [cpu_sched_debug] enforce_run_list: end 05:31:00 [---] choose_project(): 19860.000000 05:31:00 [iThena.Measurements] doing RPC to get work 05:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415739.09 cpu 2160392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:31:00 [---] Got 0 tasks 05:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:32:00 [---] [rr_sim] doing sim: CPU sched 05:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993433495.22G/439.78G) 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258938.58 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 225893.86 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225893.858169 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2033044.72 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 203304.47 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429198.330522 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829740.25 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 182974.03 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612172.355639 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646766.23 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 164676.62 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776848.978245 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482089.60 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 148208.96 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925057.938589 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333880.64 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 133388.06 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058446.002900 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200492.58 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 120049.26 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178495.260779 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080443.32 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 108044.33 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286539.592871 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972398.99 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 97239.90 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383779.491753 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875159.09 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 87515.91 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471295.400747 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787643.18 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 78764.32 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550059.718842 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708878.86 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 70887.89 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620947.605127 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637990.98 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 63799.10 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684746.702784 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574191.88 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 57419.19 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742165.890675 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516772.69 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 51677.27 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793843.159777 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465095.42 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 46509.54 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840352.701968 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418585.88 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 41858.59 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882211.289941 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376727.29 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 37672.73 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919884.019116 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339054.56 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 33905.46 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953789.475374 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305149.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 30514.91 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984304.386006 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274634.20 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 27463.42 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011767.805575 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247170.78 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 24717.08 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036484.883187 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222453.70 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 22245.37 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058730.253037 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200208.33 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 20020.83 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078751.085903 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180187.50 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 18018.75 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096769.835482 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162168.75 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 16216.87 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112986.710103 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145951.87 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 14595.19 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127581.897262 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131356.68 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 13135.67 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140717.565706 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118221.02 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 11822.10 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152539.667304 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106398.91 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 10639.89 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163179.558743 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95759.02 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 9575.90 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172755.461038 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86183.12 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 8618.31 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181373.773104 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77564.81 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 7756.48 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189130.253963 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69808.33 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 6980.83 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196111.086736 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62827.49 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 6282.75 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202393.836232 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56544.75 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 5654.47 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208048.310778 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50890.27 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 5089.03 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213137.337870 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45801.24 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 4580.12 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217717.462252 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41221.12 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 4122.11 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221839.574196 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37099.01 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3709.90 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225549.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33389.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229149.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29789.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232749.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26189.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236349.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22589.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239949.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18989.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243549.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15389.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247149.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11789.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250749.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8189.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254349.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4589.11 sec) 05:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257949.474946 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 989.11 sec) 05:32:00 [GPUGRID] [rr_sim] 2258938.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993433495.22G/439.78G) 05:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851899.49 05:32:00 [---] pick_jobs_to_run() start 05:32:00 [---] pick_jobs_to_run() end 05:32:00 [---] [rr_sim] end 05:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:32:00 [---] [cpu_sched_debug] preliminary job list: 05:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:32:00 [---] [cpu_sched_debug] final job list: 05:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:32:00 [---] [cpu_sched_debug] enforce_run_list: end 05:32:00 [---] choose_project(): 19920.000000 05:32:00 [iThena.Measurements] doing RPC to get work 05:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415679.09 cpu 2160392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:32:00 [---] Got 0 tasks 05:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:33:00 [---] [rr_sim] doing sim: CPU sched 05:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993405904.87G/439.78G) 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258875.84 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 225887.58 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225887.584481 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032988.26 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 203298.83 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429186.410514 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829689.43 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 182968.94 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612155.353944 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646720.49 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 164672.05 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776827.403031 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482048.44 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 148204.84 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925032.247209 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333843.60 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 133384.36 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058416.606969 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200459.24 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 120045.92 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178462.530753 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080413.31 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 108041.33 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286503.862159 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972371.98 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 97237.20 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383741.060425 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875134.78 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 87513.48 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471254.538863 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787621.31 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 78762.13 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550016.669458 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708859.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 70885.92 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620902.586993 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637973.26 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 63797.33 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684699.912775 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574175.93 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 57417.59 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742117.505979 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516758.34 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 51675.83 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793793.339862 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465082.50 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 46508.25 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840301.590357 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418574.25 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 41857.43 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882159.015803 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376716.83 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 37671.68 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919830.698704 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339045.15 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 33904.51 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953735.213314 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305140.63 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 30514.06 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984249.276464 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274626.57 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 27462.66 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011711.933299 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247163.91 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 24716.39 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036428.324450 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222447.52 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 22244.75 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058673.076486 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200202.77 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 20020.28 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078693.353319 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180182.49 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 18018.25 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096711.602468 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162164.24 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 16216.42 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112928.026703 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145947.82 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 14594.78 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127522.808513 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131353.04 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 13135.30 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140658.112143 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118217.73 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 11821.77 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152479.885410 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106395.96 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 10639.60 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163119.481350 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95756.36 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 9575.64 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172695.117696 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86180.73 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 8618.07 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181313.190408 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77562.65 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 7756.27 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189069.455848 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69806.39 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 6980.64 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196050.094745 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62825.75 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 6282.58 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202332.669751 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56543.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 5654.32 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207986.987257 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50888.86 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 5088.89 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213075.873013 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45799.97 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 4580.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217655.870193 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41219.97 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 4122.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221777.867655 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37097.98 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3709.80 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225487.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33388.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229087.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29788.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232687.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26188.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236287.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22588.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239887.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18988.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243487.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15388.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247087.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11788.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250687.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8188.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254287.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4588.18 sec) 05:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257887.665370 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 988.18 sec) 05:33:00 [GPUGRID] [rr_sim] 2258875.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993405904.87G/439.78G) 05:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851896.76 05:33:00 [---] pick_jobs_to_run() start 05:33:00 [---] pick_jobs_to_run() end 05:33:00 [---] [rr_sim] end 05:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:33:00 [---] [cpu_sched_debug] preliminary job list: 05:33:00 [---] [cpu_sched_debug] final job list: 05:33:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 05:33:00 [---] [work_fetch] Request work fetch: CPUs idle 05:33:00 [---] [cpu_sched_debug] enforce_run_list: end 05:33:00 [---] choose_project(): 19980.000000 05:33:00 [iThena.Measurements] doing RPC to get work 05:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415619.09 cpu 2160332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:33:00 [---] Got 0 tasks 05:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:34:00 [---] [rr_sim] doing sim: CPU sched 05:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993405904.87G/439.78G) 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258875.84 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 225887.58 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225887.584481 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032988.26 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 203298.83 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429186.410514 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829689.43 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 182968.94 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612155.353944 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646720.49 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 164672.05 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776827.403031 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482048.44 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 148204.84 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925032.247209 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333843.60 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 133384.36 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058416.606969 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200459.24 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 120045.92 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178462.530753 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080413.31 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 108041.33 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286503.862159 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972371.98 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 97237.20 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383741.060425 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875134.78 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 87513.48 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471254.538863 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787621.31 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 78762.13 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1550016.669458 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708859.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 70885.92 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620902.586993 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637973.26 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 63797.33 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684699.912775 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574175.93 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 57417.59 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742117.505979 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516758.34 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 51675.83 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793793.339862 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465082.50 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 46508.25 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840301.590357 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418574.25 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 41857.43 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882159.015803 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376716.83 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 37671.68 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919830.698704 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339045.15 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 33904.51 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953735.213314 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305140.63 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 30514.06 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984249.276464 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274626.57 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 27462.66 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011711.933299 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247163.91 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 24716.39 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036428.324450 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222447.52 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 22244.75 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058673.076486 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200202.77 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 20020.28 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078693.353319 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180182.49 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 18018.25 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096711.602468 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162164.24 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 16216.42 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112928.026703 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145947.82 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 14594.78 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127522.808513 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131353.04 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 13135.30 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140658.112143 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118217.73 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 11821.77 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152479.885410 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106395.96 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 10639.60 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163119.481350 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95756.36 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 9575.64 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172695.117696 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86180.73 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 8618.07 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181313.190408 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77562.65 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 7756.27 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189069.455848 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69806.39 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 6980.64 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196050.094745 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62825.75 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 6282.58 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202332.669751 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56543.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 5654.32 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207986.987257 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50888.86 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 5088.89 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213075.873013 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45799.97 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 4580.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217655.870193 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41219.97 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 4122.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221777.867655 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37097.98 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3709.80 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225487.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33388.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229087.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29788.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232687.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26188.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236287.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22588.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239887.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18988.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243487.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15388.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247087.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11788.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250687.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8188.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254287.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4588.18 sec) 05:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257887.665370 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 988.18 sec) 05:34:00 [GPUGRID] [rr_sim] 2258875.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993405904.87G/439.78G) 05:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851956.76 05:34:00 [---] pick_jobs_to_run() start 05:34:00 [---] pick_jobs_to_run() end 05:34:00 [---] [rr_sim] end 05:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:34:00 [---] [cpu_sched_debug] preliminary job list: 05:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:34:00 [---] [cpu_sched_debug] final job list: 05:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 05:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:34:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 993405904.87G 05:34:00 [---] [cpu_sched_debug] enforce_run_list: end 05:34:00 [---] choose_project(): 20040.000000 05:34:00 [iThena.Measurements] doing RPC to get work 05:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415559.09 cpu 2160332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:34:00 [---] Got 0 tasks 05:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:35:00 [---] [rr_sim] doing sim: CPU sched 05:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993378314.51G/439.78G) 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258813.11 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 225881.31 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225881.310793 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032931.80 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 203293.18 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429174.490507 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829638.62 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 182963.86 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612138.352249 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646674.76 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 164667.48 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776805.827817 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1482007.28 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 148200.73 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 925006.555828 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333806.55 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 133380.66 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058387.211038 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200425.90 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 120042.59 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178429.800728 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080383.31 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 108038.33 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286468.131448 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972344.98 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 97234.50 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383702.629096 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875110.48 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 87511.05 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471213.676979 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787599.43 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 78759.94 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549973.620074 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708839.49 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 70883.95 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620857.568860 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637955.54 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 63795.55 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684653.122767 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574159.99 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 57416.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742069.121283 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516743.99 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 51674.40 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793743.519948 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465069.59 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 46506.96 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840250.478746 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418562.63 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 41856.26 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882106.741664 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376706.37 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 37670.64 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919777.378291 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339035.73 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 33903.57 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953680.951255 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305132.16 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 30513.22 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984194.166922 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274618.94 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 27461.89 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011656.061023 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247157.05 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 24715.70 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036371.765714 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222441.34 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 22244.13 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058615.899935 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200197.21 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 20019.72 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078635.620735 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180177.49 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 18017.75 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096653.369454 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162159.74 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 16215.97 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112869.343302 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145943.76 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 14594.38 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127463.719764 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131349.39 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 13134.94 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140598.658581 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118214.45 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 11821.44 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152420.103516 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106393.00 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 10639.30 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163059.403957 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95753.70 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 9575.37 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172634.774354 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86178.33 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 8617.83 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181252.607712 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77560.50 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 7756.05 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189008.657734 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69804.45 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 6980.45 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195989.102753 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62824.01 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 6282.40 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202271.503271 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56541.60 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 5654.16 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207925.663737 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50887.44 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 5088.74 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213014.408156 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45798.70 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 4579.87 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217594.278133 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41218.83 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 4121.88 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221716.161113 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37096.95 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3709.69 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225425.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33387.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229025.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29787.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232625.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26187.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236225.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22587.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239825.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18987.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243425.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15387.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247025.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11787.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250625.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8187.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254225.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4587.25 sec) 05:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257825.855795 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 987.25 sec) 05:35:00 [GPUGRID] [rr_sim] 2258813.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993378314.51G/439.78G) 05:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851954.02 05:35:00 [---] pick_jobs_to_run() start 05:35:00 [---] pick_jobs_to_run() end 05:35:00 [---] [rr_sim] end 05:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:35:00 [---] [cpu_sched_debug] preliminary job list: 05:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 05:35:00 [---] [cpu_sched_debug] final job list: 05:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:35:00 [---] [cpu_sched_debug] enforce_run_list: end 05:35:00 [---] choose_project(): 20100.000000 05:35:00 [iThena.Measurements] doing RPC to get work 05:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415499.09 cpu 2160272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:35:00 [---] Got 0 tasks 05:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:36:00 [---] [rr_sim] doing sim: CPU sched 05:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993350724.15G/439.78G) 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258750.37 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 225875.04 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225875.037105 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032875.33 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 203287.53 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429162.570499 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829587.80 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 182958.78 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612121.350554 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646629.02 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 164662.90 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776784.252603 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481966.12 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 148196.61 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924980.864448 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333769.51 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 133376.95 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058357.815108 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200392.56 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 120039.26 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178397.070702 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080353.30 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 108035.33 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286432.400736 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972317.97 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 97231.80 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383664.197767 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875086.17 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 87508.62 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471172.815095 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787577.56 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 78757.76 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549930.570691 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708819.80 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 70881.98 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620812.550726 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637937.82 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 63793.78 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684606.332758 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574144.04 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 57414.40 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742020.736587 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516729.63 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 51672.96 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793693.700033 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465056.67 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 46505.67 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840199.367135 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418551.00 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 41855.10 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882054.467526 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376695.90 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 37669.59 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919724.057878 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339026.31 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 33902.63 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953626.689195 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305123.68 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 30512.37 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984139.057380 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274611.31 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 27461.13 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011600.188747 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247150.18 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 24715.02 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036315.206977 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222435.16 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 22243.52 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058558.723384 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200191.65 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 20019.16 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078577.888150 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180172.48 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 18017.25 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096595.136440 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162155.23 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 16215.52 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112810.659901 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145939.71 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 14593.97 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127404.631016 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131345.74 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 13134.57 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140539.205019 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118211.17 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 11821.12 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152360.321622 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106390.05 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 10639.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162999.326564 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95751.04 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 9575.10 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172574.431013 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86175.94 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 8617.59 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181192.025016 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77558.35 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 7755.83 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188947.859619 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69802.51 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 6980.25 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195928.110762 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62822.26 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 6282.23 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202210.336791 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56540.03 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 5654.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207864.340216 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50886.03 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 5088.60 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212952.943299 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45797.43 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 4579.74 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217532.686074 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41217.68 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 4121.77 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221654.454571 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37095.92 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3709.59 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225364.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33386.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228964.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29786.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232564.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26186.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236164.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22586.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239764.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18986.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243364.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15386.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246964.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11786.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250564.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8186.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254164.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4586.32 sec) 05:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257764.046219 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 986.32 sec) 05:36:00 [GPUGRID] [rr_sim] 2258750.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993350724.15G/439.78G) 05:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851951.28 05:36:00 [---] pick_jobs_to_run() start 05:36:00 [---] pick_jobs_to_run() end 05:36:00 [---] [rr_sim] end 05:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:36:00 [---] [cpu_sched_debug] preliminary job list: 05:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:36:00 [---] [cpu_sched_debug] final job list: 05:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:36:00 [---] [cpu_sched_debug] enforce_run_list: end 05:36:00 [---] choose_project(): 20160.000000 05:36:00 [iThena.Measurements] doing RPC to get work 05:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415439.09 cpu 2160212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:36:00 [---] Got 0 tasks 05:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:37:00 [---] [rr_sim] doing sim: CPU sched 05:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993323133.80G/439.78G) 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258687.63 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 225868.76 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225868.763417 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032818.87 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 203281.89 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429150.650491 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829536.98 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 182953.70 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612104.348859 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646583.29 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 164658.33 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776762.677389 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481924.96 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 148192.50 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924955.173067 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333732.46 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 133373.25 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058328.419177 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200359.21 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 120035.92 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178364.340676 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080323.29 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 108032.33 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286396.670025 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972290.96 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 97229.10 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383625.766439 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875061.87 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 87506.19 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471131.953211 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787555.68 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 78755.57 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549887.521307 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708800.11 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 70880.01 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620767.532593 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637920.10 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 63792.01 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684559.542750 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574128.09 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 57412.81 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741972.351891 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516715.28 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 51671.53 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793643.880119 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465043.75 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 46504.38 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840148.255523 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418539.38 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 41853.94 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882002.193388 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376685.44 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 37668.54 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919670.737465 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339016.90 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 33901.69 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953572.427135 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305115.21 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 30511.52 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984083.947838 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274603.69 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 27460.37 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011544.316471 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247143.32 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 24714.33 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036258.648240 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222428.99 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 22242.90 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058501.546833 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200186.09 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 20018.61 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078520.155566 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180167.48 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 18016.75 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096536.903426 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162150.73 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 16215.07 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112751.976500 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145935.66 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 14593.57 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127345.542267 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131342.09 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 13134.21 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140479.751456 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118207.88 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 11820.79 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152300.539727 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106387.09 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 10638.71 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162939.249171 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95748.38 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 9574.84 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172514.087671 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86173.55 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 8617.35 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181131.442320 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77556.19 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 7755.62 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188887.061505 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69800.57 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 6980.06 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195867.118771 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62820.52 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 6282.05 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202149.170310 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56538.46 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 5653.85 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207803.016696 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50884.62 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 5088.46 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212891.478443 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45796.16 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 4579.62 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217471.094015 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41216.54 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 4121.65 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221592.748030 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37094.89 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3709.49 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225302.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33385.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228902.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29785.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232502.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26185.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236102.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22585.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239702.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18985.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243302.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15385.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246902.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11785.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250502.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8185.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254102.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4585.40 sec) 05:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257702.236643 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 985.40 sec) 05:37:00 [GPUGRID] [rr_sim] 2258687.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993323133.80G/439.78G) 05:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851948.55 05:37:00 [---] pick_jobs_to_run() start 05:37:00 [---] pick_jobs_to_run() end 05:37:00 [---] [rr_sim] end 05:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:37:00 [---] [cpu_sched_debug] preliminary job list: 05:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:37:00 [---] [cpu_sched_debug] final job list: 05:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:37:00 [---] [cpu_sched_debug] enforce_run_list: end 05:37:00 [---] choose_project(): 20220.000000 05:37:00 [iThena.Measurements] doing RPC to get work 05:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415379.09 cpu 2160152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:37:00 [---] Got 0 tasks 05:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:38:00 [---] [rr_sim] doing sim: CPU sched 05:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993295543.44G/439.78G) 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258624.90 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 225862.49 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225862.489728 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032762.41 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 203276.24 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429138.730484 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829486.17 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 182948.62 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612087.347164 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646537.55 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 164653.76 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776741.102176 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481883.80 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 148188.38 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924929.481686 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333695.42 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 133369.54 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058299.023246 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200325.87 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 120032.59 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178331.610650 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080293.29 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 108029.33 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286360.939313 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972263.96 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 97226.40 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383587.335110 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875037.56 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 87503.76 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471091.091327 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787533.81 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 78753.38 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549844.471923 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708780.43 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 70878.04 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620722.514459 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637902.38 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 63790.24 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684512.752741 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574112.14 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 57411.21 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741923.967196 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516700.93 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 51670.09 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793594.060204 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465030.84 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 46503.08 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840097.143912 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418527.75 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 41852.78 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881949.919249 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376674.98 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 37667.50 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919617.417053 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339007.48 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 33900.75 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953518.165076 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305106.73 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 30510.67 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984028.838296 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274596.06 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 27459.61 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011488.444195 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247136.45 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 24713.65 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036202.089504 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222422.81 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 22242.28 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058444.370282 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200180.53 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 20018.05 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078462.422982 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180162.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 18016.25 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096478.670412 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162146.23 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 16214.62 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112693.293099 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145931.60 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 14593.16 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127286.453518 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131338.44 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 13133.84 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140420.297894 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118204.60 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 11820.46 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152240.757833 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106384.14 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 10638.41 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162879.171778 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95745.73 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 9574.57 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172453.744329 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86171.15 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 8617.12 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181070.859624 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77554.04 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 7755.40 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188826.263390 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69798.63 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 6979.86 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195806.126779 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62818.77 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 6281.88 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202088.003830 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56536.89 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 5653.69 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207741.693175 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50883.20 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 5088.32 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212830.013586 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45794.88 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 4579.49 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217409.501956 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41215.40 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 4121.54 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221531.041488 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37093.86 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3709.39 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225240.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33384.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228840.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29784.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232440.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26184.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236040.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22584.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239640.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18984.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243240.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15384.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246840.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11784.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250440.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8184.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254040.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4584.47 sec) 05:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257640.427068 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 984.47 sec) 05:38:00 [GPUGRID] [rr_sim] 2258624.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993295543.44G/439.78G) 05:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851945.81 05:38:00 [---] pick_jobs_to_run() start 05:38:00 [---] pick_jobs_to_run() end 05:38:00 [---] [rr_sim] end 05:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:38:00 [---] [cpu_sched_debug] preliminary job list: 05:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:38:00 [---] [cpu_sched_debug] final job list: 05:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:38:00 [---] [cpu_sched_debug] enforce_run_list: end 05:38:00 [---] choose_project(): 20280.000000 05:38:00 [iThena.Measurements] doing RPC to get work 05:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415319.09 cpu 2160092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:38:00 [---] Got 0 tasks 05:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:39:00 [---] [rr_sim] doing sim: CPU sched 05:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993267953.08G/439.78G) 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258562.16 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 225856.22 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225856.216040 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032705.94 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 203270.59 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429126.810476 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829435.35 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 182943.53 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612070.345469 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646491.81 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 164649.18 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776719.526962 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481842.63 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 148184.26 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924903.790306 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333658.37 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 133365.84 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058269.627315 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200292.53 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 120029.25 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178298.880624 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080263.28 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 108026.33 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286325.208602 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972236.95 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 97223.70 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383548.903782 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875013.26 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 87501.33 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471050.229443 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787511.93 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 78751.19 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549801.422539 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708760.74 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 70876.07 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620677.496325 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637884.66 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 63788.47 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684465.962733 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574096.20 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 57409.62 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741875.582500 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516686.58 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 51668.66 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793544.240290 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465017.92 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 46501.79 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840046.032301 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418516.13 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 41851.61 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881897.645111 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376664.52 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 37666.45 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919564.096640 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338998.06 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 33899.81 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953463.903016 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305098.26 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 30509.83 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983973.728755 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274588.43 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 27458.84 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011432.571919 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247129.59 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 24712.96 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036145.530767 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222416.63 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 22241.66 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058387.193731 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200174.97 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 20017.50 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078404.690398 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180157.47 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 18015.75 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096420.437398 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162141.72 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 16214.17 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112634.609698 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145927.55 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 14592.76 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127227.364769 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131334.80 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 13133.48 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140360.844332 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118201.32 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 11820.13 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152180.975939 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106381.18 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 10638.12 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162819.094385 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95743.07 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 9574.31 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172393.400987 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86168.76 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 8616.88 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181010.276928 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77551.88 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 7755.19 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188765.465275 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69796.70 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 6979.67 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195745.134788 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62817.03 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 6281.70 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202026.837349 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56535.32 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 5653.53 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207680.369654 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50881.79 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 5088.18 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212768.548729 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45793.61 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 4579.36 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217347.909896 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41214.25 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 4121.43 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221469.334947 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37092.83 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3709.28 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225178.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33383.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228778.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29783.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232378.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26183.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235978.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22583.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239578.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18983.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243178.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15383.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246778.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11783.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250378.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8183.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253978.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4583.54 sec) 05:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257578.617492 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 983.54 sec) 05:39:00 [GPUGRID] [rr_sim] 2258562.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993267953.08G/439.78G) 05:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851943.07 05:39:00 [---] pick_jobs_to_run() start 05:39:00 [---] pick_jobs_to_run() end 05:39:00 [---] [rr_sim] end 05:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:39:00 [---] [cpu_sched_debug] preliminary job list: 05:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:39:00 [---] [cpu_sched_debug] final job list: 05:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:39:00 [---] [cpu_sched_debug] enforce_run_list: end 05:39:00 [---] choose_project(): 20340.000000 05:39:00 [iThena.Measurements] doing RPC to get work 05:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 588059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415259.09 cpu 2160032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2160032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2160032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:39:00 [---] Got 0 tasks 05:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:40:00 [---] [rr_sim] doing sim: CPU sched 05:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993240362.73G/439.78G) 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258499.42 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 225849.94 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225849.942352 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032649.48 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 203264.95 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429114.890469 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829384.53 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 182938.45 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612053.343774 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646446.08 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 164644.61 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776697.951748 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481801.47 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 148180.15 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924878.098925 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333621.32 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 133362.13 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058240.231385 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200259.19 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 120025.92 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178266.150598 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080233.27 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 108023.33 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286289.477890 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972209.95 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 97220.99 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383510.472453 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874988.95 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 87498.90 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1471009.367560 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787490.06 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 78749.01 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549758.373155 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708741.05 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 70874.11 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620632.478192 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637866.95 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 63786.69 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684419.172724 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574080.25 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 57408.03 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741827.197804 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516672.23 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 51667.22 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793494.420375 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465005.00 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 46500.50 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839994.920690 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418504.50 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 41850.45 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881845.370973 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376654.05 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 37665.41 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919510.776227 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338988.65 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 33898.86 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953409.640956 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305089.78 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 30508.98 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983918.619213 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274580.80 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 27458.08 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011376.699643 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247122.72 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 24712.27 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036088.972031 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222410.45 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 22241.05 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058330.017180 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200169.41 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 20016.94 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078346.957814 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180152.47 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 18015.25 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096362.204384 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162137.22 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 16213.72 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112575.926298 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145923.50 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 14592.35 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127168.276020 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131331.15 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 13133.11 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140301.390770 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118198.03 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 11819.80 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152121.194044 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106378.23 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 10637.82 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162759.016992 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95740.41 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 9574.04 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172333.057645 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86166.37 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 8616.64 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180949.694232 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77549.73 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 7754.97 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188704.667161 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69794.76 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 6979.48 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195684.142797 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62815.28 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 6281.53 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201965.670869 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56533.75 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 5653.38 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207619.046134 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50880.38 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 5088.04 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212707.083872 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45792.34 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 4579.23 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217286.317837 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41213.11 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 4121.31 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221407.628405 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37091.80 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3709.18 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225116.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33382.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228716.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29782.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232316.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26182.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235916.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22582.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239516.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18982.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243116.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15382.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246716.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11782.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250316.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8182.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253916.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4582.62 sec) 05:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257516.807916 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 982.62 sec) 05:40:00 [GPUGRID] [rr_sim] 2258499.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993240362.73G/439.78G) 05:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851940.34 05:40:00 [---] pick_jobs_to_run() start 05:40:00 [---] pick_jobs_to_run() end 05:40:00 [---] [rr_sim] end 05:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:40:00 [---] [cpu_sched_debug] preliminary job list: 05:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:40:00 [---] [cpu_sched_debug] final job list: 05:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:40:00 [---] [cpu_sched_debug] enforce_run_list: end 05:40:00 [---] choose_project(): 20400.000000 05:40:00 [iThena.Measurements] doing RPC to get work 05:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415199.09 cpu 2159972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:40:00 [---] Got 0 tasks 05:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:41:00 [---] [rr_sim] doing sim: CPU sched 05:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993212772.37G/439.78G) 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258436.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 225843.67 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225843.668664 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032593.02 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 203259.30 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429102.970461 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829333.72 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 182933.37 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612036.342079 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646400.34 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 164640.03 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776676.376534 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481760.31 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 148176.03 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924852.407545 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333584.28 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 133358.43 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058210.835454 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200225.85 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 120022.59 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178233.420572 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080203.27 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 108020.33 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286253.747179 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972182.94 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 97218.29 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383472.041124 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874964.65 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 87496.46 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470968.505676 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787468.18 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 78746.82 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549715.323772 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708721.36 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 70872.14 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620587.460058 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637849.23 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 63784.92 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684372.382716 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574064.30 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 57406.43 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741778.813108 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516657.87 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 51665.79 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793444.600461 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464992.09 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 46499.21 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839943.809078 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418492.88 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 41849.29 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881793.096834 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376643.59 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 37664.36 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919457.455815 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338979.23 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 33897.92 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953355.378897 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305081.31 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 30508.13 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983863.509671 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274573.18 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 27457.32 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011320.827367 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247115.86 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 24711.59 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036032.413294 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222404.27 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 22240.43 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058272.840628 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200163.85 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 20016.38 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078289.225229 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180147.46 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 18014.75 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096303.971370 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162132.72 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 16213.27 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112517.242897 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145919.44 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 14591.94 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127109.187271 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131327.50 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 13132.75 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140241.937207 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118194.75 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 11819.47 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152061.412150 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106375.27 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 10637.53 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162698.939599 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95737.75 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 9573.77 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172272.714303 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86163.97 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 8616.40 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180889.111536 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77547.58 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 7754.76 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188643.869046 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69792.82 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 6979.28 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195623.150805 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62813.54 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 6281.35 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201904.504388 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56532.18 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 5653.22 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207557.722613 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50878.96 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 5087.90 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212645.619015 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45791.07 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 4579.11 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217224.725778 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41211.96 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 4121.20 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221345.921864 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37090.76 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3709.08 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225054.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33381.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228654.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29781.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232254.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26181.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235854.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22581.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239454.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18981.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243054.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15381.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246654.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11781.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250254.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8181.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253854.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4581.69 sec) 05:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257454.998341 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 981.69 sec) 05:41:00 [GPUGRID] [rr_sim] 2258436.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993212772.37G/439.78G) 05:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851937.60 05:41:00 [---] pick_jobs_to_run() start 05:41:00 [---] pick_jobs_to_run() end 05:41:00 [---] [rr_sim] end 05:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:41:00 [---] [cpu_sched_debug] preliminary job list: 05:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:41:00 [---] [cpu_sched_debug] final job list: 05:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:41:00 [---] [cpu_sched_debug] enforce_run_list: end 05:41:00 [---] choose_project(): 20460.000000 05:41:00 [iThena.Measurements] doing RPC to get work 05:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415139.09 cpu 2159912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:41:00 [---] Got 0 tasks 05:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:42:00 [---] [rr_sim] doing sim: CPU sched 05:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993185182.02G/439.78G) 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258373.95 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 225837.39 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225837.394975 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032536.55 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 203253.66 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429091.050453 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829282.90 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 182928.29 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612019.340383 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646354.61 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 164635.46 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776654.801321 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481719.15 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 148171.91 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924826.716164 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333547.23 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 133354.72 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058181.439523 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200192.51 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 120019.25 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178200.690546 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080173.26 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 108017.33 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286218.016467 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972155.93 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 97215.59 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383433.609796 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874940.34 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 87494.03 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470927.643792 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787446.31 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 78744.63 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549672.274388 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708701.68 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 70870.17 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620542.441925 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637831.51 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 63783.15 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684325.592708 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574048.36 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 57404.84 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741730.428412 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516643.52 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 51664.35 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793394.780546 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464979.17 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 46497.92 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839892.697467 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418481.25 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 41848.13 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881740.822696 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376633.13 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 37663.31 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919404.135402 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338969.81 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 33896.98 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953301.116837 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305072.83 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 30507.28 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983808.400129 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274565.55 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 27456.55 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011264.955091 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247108.99 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 24710.90 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035975.854558 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222398.10 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 22239.81 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058215.664077 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200158.29 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 20015.83 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078231.492645 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180142.46 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 18014.25 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096245.738356 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162128.21 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 16212.82 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112458.559496 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145915.39 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 14591.54 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127050.098522 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131323.85 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 13132.39 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140182.483645 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118191.47 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 11819.15 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152001.630256 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106372.32 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 10637.23 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162638.862206 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95735.09 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 9573.51 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172212.370961 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86161.58 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 8616.16 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180828.528840 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77545.42 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 7754.54 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188583.070931 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69790.88 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 6979.09 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195562.158814 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62811.79 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 6281.18 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201843.337908 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56530.61 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 5653.06 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207496.399093 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50877.55 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 5087.76 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212584.154159 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45789.80 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 4578.98 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217163.133718 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41210.82 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 4121.08 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221284.215322 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37089.73 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3708.97 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224993.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33380.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228593.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29780.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232193.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26180.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235793.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22580.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239393.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18980.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242993.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15380.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246593.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11780.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250193.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8180.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253793.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4580.76 sec) 05:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257393.188765 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 980.76 sec) 05:42:00 [GPUGRID] [rr_sim] 2258373.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993185182.02G/439.78G) 05:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851934.86 05:42:00 [---] pick_jobs_to_run() start 05:42:00 [---] pick_jobs_to_run() end 05:42:00 [---] [rr_sim] end 05:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:42:00 [---] [cpu_sched_debug] preliminary job list: 05:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:42:00 [---] [cpu_sched_debug] final job list: 05:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:42:00 [---] [cpu_sched_debug] enforce_run_list: end 05:42:00 [---] choose_project(): 20520.000000 05:42:00 [iThena.Measurements] doing RPC to get work 05:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415079.09 cpu 2159852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:42:00 [---] Got 0 tasks 05:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:43:00 [---] [rr_sim] doing sim: CPU sched 05:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993157591.66G/439.78G) 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258311.21 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 225831.12 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225831.121287 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032480.09 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 203248.01 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429079.130446 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829232.08 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 182923.21 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 612002.338688 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646308.87 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 164630.89 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776633.226107 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481677.99 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 148167.80 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924801.024783 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333510.19 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 133351.02 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058152.043592 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200159.17 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 120015.92 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178167.960520 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080143.25 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 108014.33 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286182.285755 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972128.93 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 97212.89 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383395.178467 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874916.03 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 87491.60 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470886.781908 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787424.43 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 78742.44 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549629.225004 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708681.99 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 70868.20 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620497.423791 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637813.79 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 63781.38 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684278.802699 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574032.41 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 57403.24 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741682.043716 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516629.17 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 51662.92 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793344.960632 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464966.25 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 46496.63 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839841.585856 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418469.63 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 41846.96 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881688.548558 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376622.66 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 37662.27 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919350.814989 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338960.40 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 33896.04 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953246.854777 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305064.36 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 30506.44 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983753.290587 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274557.92 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 27455.79 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011209.082815 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247102.13 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 24710.21 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035919.295821 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222391.92 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 22239.19 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058158.487526 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200152.73 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 20015.27 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078173.760061 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180137.45 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 18013.75 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096187.505342 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162123.71 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 16212.37 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112399.876095 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145911.34 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 14591.13 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126991.009773 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131320.20 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 13132.02 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140123.030083 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118188.18 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 11818.82 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151941.848362 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106369.36 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 10636.94 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162578.784813 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95732.43 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 9573.24 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172152.027619 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86159.19 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 8615.92 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180767.946144 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77543.27 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 7754.33 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188522.272817 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69788.94 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 6978.89 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195501.166822 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62810.05 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 6281.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201782.171427 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56529.04 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 5652.90 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207435.075572 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50876.14 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 5087.61 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212522.689302 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45788.52 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 4578.85 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217101.541659 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41209.67 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 4120.97 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221222.508780 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37088.70 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3708.87 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224931.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33379.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228531.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29779.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232131.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26179.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235731.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22579.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239331.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18979.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242931.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15379.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246531.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11779.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250131.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8179.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253731.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4579.83 sec) 05:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257331.379190 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 979.83 sec) 05:43:00 [GPUGRID] [rr_sim] 2258311.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993157591.66G/439.78G) 05:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851932.13 05:43:00 [---] pick_jobs_to_run() start 05:43:00 [---] pick_jobs_to_run() end 05:43:00 [---] [rr_sim] end 05:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:43:00 [---] [cpu_sched_debug] preliminary job list: 05:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:43:00 [---] [cpu_sched_debug] final job list: 05:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:43:00 [---] [cpu_sched_debug] enforce_run_list: end 05:43:00 [---] choose_project(): 20580.000000 05:43:00 [iThena.Measurements] doing RPC to get work 05:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 415019.09 cpu 2159792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:43:00 [---] Got 0 tasks 05:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:44:00 [---] [rr_sim] doing sim: CPU sched 05:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993130001.30G/439.78G) 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258248.48 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 225824.85 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225824.847599 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032423.63 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 203242.36 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429067.210438 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829181.27 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 182918.13 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611985.336993 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646263.14 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 164626.31 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776611.650893 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481636.83 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 148163.68 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924775.333403 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333473.14 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 133347.31 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058122.647661 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200125.83 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 120012.58 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178135.230494 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080113.25 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 108011.32 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286146.555044 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972101.92 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 97210.19 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383356.747139 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874891.73 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 87489.17 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470845.920024 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787402.56 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 78740.26 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549586.175620 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708662.30 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 70866.23 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620452.405657 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637796.07 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 63779.61 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684232.012691 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574016.46 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 57401.65 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741633.659021 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516614.82 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 51661.48 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793295.140718 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464953.34 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 46495.33 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839790.474245 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418458.00 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 41845.80 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881636.274419 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376612.20 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 37661.22 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919297.494576 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338950.98 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 33895.10 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953192.592718 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305055.88 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 30505.59 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983698.181045 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274550.29 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 27455.03 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011153.210540 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247095.27 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 24709.53 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035862.737085 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222385.74 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 22238.57 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058101.310975 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200147.17 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 20014.72 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078116.027477 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180132.45 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 18013.24 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096129.272328 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162119.20 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 16211.92 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112341.192694 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145907.28 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 14590.73 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126931.921024 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131316.55 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 13131.66 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140063.576520 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118184.90 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 11818.49 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151882.066467 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106366.41 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 10636.64 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162518.707420 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95729.77 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 9572.98 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172091.684277 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86156.79 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 8615.68 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180707.363448 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77541.11 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 7754.11 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188461.474702 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69787.00 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 6978.70 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195440.174831 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62808.30 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 6280.83 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201721.004947 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56527.47 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 5652.75 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207373.752051 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50874.72 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 5087.47 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212461.224445 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45787.25 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 4578.73 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217039.949600 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41208.53 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 4120.85 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221160.802239 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37087.67 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3708.77 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224869.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33378.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228469.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29778.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232069.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26178.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235669.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22578.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239269.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18978.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242869.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15378.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246469.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11778.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250069.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8178.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253669.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4578.91 sec) 05:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257269.569614 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 978.91 sec) 05:44:00 [GPUGRID] [rr_sim] 2258248.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993130001.30G/439.78G) 05:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851929.39 05:44:00 [---] pick_jobs_to_run() start 05:44:00 [---] pick_jobs_to_run() end 05:44:00 [---] [rr_sim] end 05:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:44:00 [---] [cpu_sched_debug] preliminary job list: 05:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:44:00 [---] [cpu_sched_debug] final job list: 05:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:44:00 [---] [cpu_sched_debug] enforce_run_list: end 05:44:00 [---] choose_project(): 20640.000000 05:44:00 [iThena.Measurements] doing RPC to get work 05:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414959.09 cpu 2159732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:44:00 [---] Got 0 tasks 05:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:45:00 [---] [rr_sim] doing sim: CPU sched 05:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993102410.95G/439.78G) 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258185.74 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 225818.57 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225818.573911 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032367.17 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 203236.72 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429055.290431 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829130.45 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 182913.04 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611968.335298 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646217.40 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 164621.74 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776590.075679 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481595.66 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 148159.57 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924749.642022 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333436.10 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 133343.61 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058093.251731 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200092.49 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 120009.25 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178102.500468 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080083.24 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 108008.32 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286110.824332 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972074.91 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 97207.49 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383318.315810 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874867.42 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 87486.74 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470805.058140 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787380.68 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 78738.07 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549543.126237 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708642.61 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 70864.26 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620407.387524 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637778.35 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 63777.84 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684185.222682 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574000.52 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 57400.05 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741585.274325 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516600.46 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 51660.05 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793245.320803 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464940.42 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 46494.04 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839739.362634 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418446.38 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 41844.64 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881584.000281 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376601.74 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 37660.17 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919244.174164 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338941.56 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 33894.16 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953138.330658 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305047.41 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 30504.74 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983643.071503 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274542.67 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 27454.27 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011097.338264 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247088.40 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 24708.84 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035806.178348 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222379.56 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 22237.96 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058044.134424 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200141.60 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 20014.16 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078058.294892 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180127.44 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 18012.74 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096071.039314 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162114.70 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 16211.47 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112282.509293 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145903.23 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 14590.32 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126872.832275 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131312.91 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 13131.29 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140004.122958 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118181.62 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 11818.16 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151822.284573 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106363.45 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 10636.35 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162458.630027 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95727.11 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 9572.71 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172031.340935 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86154.40 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 8615.44 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180646.780752 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77538.96 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 7753.90 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188400.676588 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69785.06 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 6978.51 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195379.182840 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62806.56 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 6280.66 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201659.838467 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56525.90 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 5652.59 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207312.428531 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50873.31 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 5087.33 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212399.759588 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45785.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 4578.60 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216978.357540 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41207.38 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 4120.74 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221099.095697 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37086.64 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3708.66 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224807.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33377.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228407.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29777.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232007.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26177.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235607.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22577.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239207.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18977.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242807.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15377.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246407.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11777.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250007.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8177.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253607.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4577.98 sec) 05:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257207.760038 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977.98 sec) 05:45:00 [GPUGRID] [rr_sim] 2258185.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993102410.95G/439.78G) 05:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851926.65 05:45:00 [---] pick_jobs_to_run() start 05:45:00 [---] pick_jobs_to_run() end 05:45:00 [---] [rr_sim] end 05:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:45:00 [---] [cpu_sched_debug] preliminary job list: 05:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:45:00 [---] [cpu_sched_debug] final job list: 05:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:45:00 [---] [cpu_sched_debug] enforce_run_list: end 05:45:00 [---] choose_project(): 20700.000000 05:45:00 [iThena.Measurements] doing RPC to get work 05:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414899.09 cpu 2159672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:45:00 [---] Got 0 tasks 05:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:46:00 [---] [rr_sim] doing sim: CPU sched 05:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993074820.59G/439.78G) 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258123.00 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 225812.30 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225812.300223 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032310.70 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 203231.07 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429043.370423 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829079.63 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 182907.96 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611951.333603 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646171.67 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 164617.17 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776568.500465 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481554.50 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 148155.45 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924723.950641 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333399.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 133339.91 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058063.855800 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200059.15 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 120005.91 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178069.770443 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080053.23 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 108005.32 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286075.093621 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972047.91 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 97204.79 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383279.884481 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874843.12 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 87484.31 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470764.196256 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787358.81 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 78735.88 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549500.076853 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708622.93 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 70862.29 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620362.369390 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637760.63 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 63776.06 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684138.432674 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573984.57 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 57398.46 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741536.889629 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516586.11 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 51658.61 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793195.500889 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464927.50 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 46492.75 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839688.251022 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418434.75 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 41843.48 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881531.726143 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376591.28 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 37659.13 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919190.853751 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338932.15 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 33893.21 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953084.068598 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305038.93 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 30503.89 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983587.961961 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274535.04 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 27453.50 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011041.465988 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247081.54 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 24708.15 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035749.619611 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222373.38 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 22237.34 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057986.957873 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200136.04 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 20013.60 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078000.562308 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180122.44 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 18012.24 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096012.806300 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162110.20 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 16211.02 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112223.825893 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145899.18 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 14589.92 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126813.743526 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131309.26 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 13130.93 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139944.669396 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118178.33 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 11817.83 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151762.502679 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106360.50 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 10636.05 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162398.552634 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95724.45 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 9572.44 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171970.997593 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86152.00 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 8615.20 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180586.198056 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77536.80 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 7753.68 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188339.878473 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69783.12 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 6978.31 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195318.190848 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62804.81 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 6280.48 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201598.671986 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56524.33 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 5652.43 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207251.105010 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50871.90 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 5087.19 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212338.294732 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45784.71 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 4578.47 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216916.765481 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41206.24 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 4120.62 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221037.389156 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37085.61 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3708.56 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224745.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33377.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228345.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29777.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231945.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26177.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235545.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22577.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239145.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18977.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242745.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15377.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246345.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11777.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249945.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8177.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253545.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4577.05 sec) 05:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257145.950463 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 977.05 sec) 05:46:00 [GPUGRID] [rr_sim] 2258123.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993074820.59G/439.78G) 05:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851923.91 05:46:00 [---] pick_jobs_to_run() start 05:46:00 [---] pick_jobs_to_run() end 05:46:00 [---] [rr_sim] end 05:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:46:00 [---] [cpu_sched_debug] preliminary job list: 05:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:46:00 [---] [cpu_sched_debug] final job list: 05:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:46:00 [---] [cpu_sched_debug] enforce_run_list: end 05:46:00 [---] choose_project(): 20760.000000 05:46:00 [iThena.Measurements] doing RPC to get work 05:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414839.09 cpu 2159612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:46:00 [---] Got 0 tasks 05:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:47:00 [---] [rr_sim] doing sim: CPU sched 05:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993047230.23G/439.78G) 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2258060.27 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 225806.03 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225806.026534 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032254.24 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 203225.42 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429031.450415 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1829028.81 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 182902.88 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611934.331908 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646125.93 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 164612.59 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776546.925252 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481513.34 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 148151.33 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924698.259261 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333362.01 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 133336.20 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058034.459869 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1200025.81 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 120002.58 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178037.040417 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1080023.22 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 108002.32 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286039.362909 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972020.90 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 97202.09 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383241.453153 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874818.81 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 87481.88 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470723.334372 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787336.93 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 78733.69 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549457.027469 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708603.24 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 70860.32 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620317.351256 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637742.91 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 63774.29 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684091.642665 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573968.62 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 57396.86 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741488.504933 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516571.76 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 51657.18 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793145.680974 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464914.58 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 46491.46 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839637.139411 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418423.13 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 41842.31 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881479.452004 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376580.81 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 37658.08 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919137.533338 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338922.73 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 33892.27 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953029.806539 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305030.46 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 30503.05 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983532.852419 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274527.41 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 27452.74 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010985.593712 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247074.67 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 24707.47 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035693.060875 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222367.20 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 22236.72 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057929.781322 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200130.48 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 20013.05 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077942.829724 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180117.44 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 18011.74 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095954.573286 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162105.69 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 16210.57 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112165.142492 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145895.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 14589.51 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126754.654777 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131305.61 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 13130.56 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139885.215834 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118175.05 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 11817.50 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151702.720785 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106357.54 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 10635.75 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162338.475240 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95721.79 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 9572.18 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171910.654251 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86149.61 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 8614.96 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180525.615360 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77534.65 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 7753.46 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188279.080358 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69781.18 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 6978.12 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195257.198857 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62803.07 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 6280.31 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201537.505506 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56522.76 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 5652.28 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207189.781489 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50870.48 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 5087.05 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212276.829875 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45783.44 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 4578.34 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216855.173422 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41205.09 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 4120.51 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220975.682614 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37084.58 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3708.46 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224684.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33376.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228284.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29776.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231884.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26176.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235484.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22576.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239084.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18976.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242684.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15376.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246284.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11776.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249884.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8176.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253484.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4576.12 sec) 05:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257084.140887 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 976.12 sec) 05:47:00 [GPUGRID] [rr_sim] 2258060.27: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993047230.23G/439.78G) 05:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851921.18 05:47:00 [---] pick_jobs_to_run() start 05:47:00 [---] pick_jobs_to_run() end 05:47:00 [---] [rr_sim] end 05:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:47:00 [---] [cpu_sched_debug] preliminary job list: 05:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:47:00 [---] [cpu_sched_debug] final job list: 05:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:47:00 [---] [cpu_sched_debug] enforce_run_list: end 05:47:00 [---] choose_project(): 20820.000000 05:47:00 [iThena.Measurements] doing RPC to get work 05:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414779.09 cpu 2159552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:47:00 [---] Got 0 tasks 05:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:48:00 [---] [rr_sim] doing sim: CPU sched 05:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (993019639.88G/439.78G) 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257997.53 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 225799.75 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225799.752846 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032197.78 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 203219.78 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429019.530408 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828978.00 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 182897.80 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611917.330213 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646080.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 164608.02 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776525.350038 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481472.18 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 148147.22 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924672.567880 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333324.96 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 133332.50 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1058005.063938 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199992.46 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 119999.25 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1178004.310391 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079993.22 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 107999.32 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1286003.632198 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971993.90 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 97199.39 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383203.021824 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874794.51 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 87479.45 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470682.472488 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787315.06 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 78731.51 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549413.978085 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708583.55 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 70858.36 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620272.333123 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637725.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 63772.52 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1684044.852657 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573952.68 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 57395.27 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741440.120237 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516557.41 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 51655.74 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793095.861060 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464901.67 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 46490.17 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839586.027800 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418411.50 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 41841.15 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881427.177866 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376570.35 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 37657.04 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919084.212926 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338913.32 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 33891.33 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952975.544479 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305021.98 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 30502.20 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983477.742877 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274519.79 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 27451.98 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010929.721436 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247067.81 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 24706.78 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035636.502138 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222361.03 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 22236.10 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057872.604771 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200124.92 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 20012.49 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077885.097140 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180112.43 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 18011.24 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095896.340272 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162101.19 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 16210.12 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112106.459091 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145891.07 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 14589.11 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126695.566028 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131301.96 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 13130.20 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139825.762271 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118171.77 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 11817.18 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151642.938890 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106354.59 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 10635.46 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162278.397847 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95719.13 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 9571.91 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171850.310909 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86147.22 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 8614.72 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180465.032664 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77532.50 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 7753.25 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188218.282244 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69779.25 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 6977.92 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195196.206866 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62801.32 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 6280.13 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201476.339025 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56521.19 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 5652.12 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207128.457969 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50869.07 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 5086.91 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212215.365018 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45782.16 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 4578.22 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216793.581362 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41203.95 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 4120.39 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220913.976072 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37083.55 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3708.36 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224622.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33375.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228222.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29775.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231822.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26175.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235422.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22575.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239022.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18975.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242622.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15375.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246222.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11775.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249822.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8175.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253422.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4575.20 sec) 05:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2257022.331311 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 975.20 sec) 05:48:00 [GPUGRID] [rr_sim] 2257997.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (993019639.88G/439.78G) 05:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851918.44 05:48:00 [---] pick_jobs_to_run() start 05:48:00 [---] pick_jobs_to_run() end 05:48:00 [---] [rr_sim] end 05:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:48:00 [---] [cpu_sched_debug] preliminary job list: 05:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:48:00 [---] [cpu_sched_debug] final job list: 05:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:48:00 [---] [cpu_sched_debug] enforce_run_list: end 05:48:00 [---] choose_project(): 20880.000000 05:48:00 [iThena.Measurements] doing RPC to get work 05:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414719.09 cpu 2159492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:48:00 [---] Got 0 tasks 05:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:49:00 [---] [rr_sim] doing sim: CPU sched 05:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992992049.52G/439.78G) 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257934.79 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 225793.48 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225793.479158 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032141.31 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 203214.13 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 429007.610400 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828927.18 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 182892.72 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611900.328518 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1646034.46 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 164603.45 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776503.774824 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481431.02 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 148143.10 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924646.876500 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333287.92 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 133328.79 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057975.668008 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199959.12 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 119995.91 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177971.580365 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079963.21 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 107996.32 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285967.901486 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971966.89 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 97196.69 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383164.590496 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874770.20 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 87477.02 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470641.610604 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787293.18 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 78729.32 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549370.928701 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708563.86 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 70856.39 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620227.314989 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637707.48 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 63770.75 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683998.062648 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573936.73 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 57393.67 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741391.735541 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516543.06 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 51654.31 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793046.041145 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464888.75 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 46488.88 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839534.916189 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418399.88 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 41839.99 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881374.903728 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376559.89 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 37655.99 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919030.892513 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338903.90 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 33890.39 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952921.282419 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305013.51 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 30501.35 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983422.633335 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274512.16 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 27451.22 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010873.849160 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247060.94 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 24706.09 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035579.943402 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222354.85 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 22235.48 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057815.428220 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200119.36 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 20011.94 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077827.364556 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180107.43 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 18010.74 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095838.107258 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162096.68 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 16209.67 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112047.775690 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145887.02 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 14588.70 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126636.477279 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131298.31 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 13129.83 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139766.308709 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118168.48 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 11816.85 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151583.156996 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106351.63 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 10635.16 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162218.320454 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95716.47 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 9571.65 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171789.967567 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86144.82 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 8614.48 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180404.449968 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77530.34 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 7753.03 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188157.484129 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69777.31 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 6977.73 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195135.214874 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62799.58 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 6279.96 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201415.172545 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56519.62 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 5651.96 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207067.134448 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50867.66 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 5086.77 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212153.900161 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45780.89 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 4578.09 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216731.989303 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41202.80 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 4120.28 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220852.269531 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37082.52 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3708.25 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224560.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33374.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228160.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29774.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231760.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26174.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235360.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22574.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238960.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18974.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242560.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15374.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246160.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11774.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249760.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8174.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253360.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4574.27 sec) 05:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256960.521736 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 974.27 sec) 05:49:00 [GPUGRID] [rr_sim] 2257934.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992992049.52G/439.78G) 05:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851915.70 05:49:00 [---] pick_jobs_to_run() start 05:49:00 [---] pick_jobs_to_run() end 05:49:00 [---] [rr_sim] end 05:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:49:00 [---] [cpu_sched_debug] preliminary job list: 05:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:49:00 [---] [cpu_sched_debug] final job list: 05:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:49:00 [---] [cpu_sched_debug] enforce_run_list: end 05:49:00 [---] choose_project(): 20940.000000 05:49:00 [iThena.Measurements] doing RPC to get work 05:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414659.09 cpu 2159432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:49:00 [---] Got 0 tasks 05:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:50:00 [---] [rr_sim] doing sim: CPU sched 05:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992964459.17G/439.78G) 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257872.05 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 225787.21 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225787.205470 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032084.85 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 203208.48 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428995.690392 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828876.36 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 182887.64 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611883.326823 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645988.73 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 164598.87 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776482.199610 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481389.86 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 148138.99 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924621.185119 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333250.87 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 133325.09 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057946.272077 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199925.78 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 119992.58 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177938.850339 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079933.20 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 107993.32 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285932.170775 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971939.88 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 97193.99 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383126.159167 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874745.90 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 87474.59 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470600.748720 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787271.31 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 78727.13 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549327.879318 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708544.18 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 70854.42 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620182.296856 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637689.76 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 63768.98 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683951.272640 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573920.78 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 57392.08 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741343.350846 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516528.70 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 51652.87 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792996.221231 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464875.83 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 46487.58 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839483.804577 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418388.25 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 41838.83 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881322.629589 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376549.43 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 37654.94 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918977.572100 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338894.48 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 33889.45 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952867.020360 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305005.03 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 30500.50 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983367.523794 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274504.53 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 27450.45 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010817.976884 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247054.08 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 24705.41 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035523.384665 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222348.67 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 22234.87 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057758.251668 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200113.80 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 20011.38 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077769.631971 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180102.42 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 18010.24 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095779.874244 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162092.18 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 16209.22 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111989.092289 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145882.96 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 14588.30 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126577.388530 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131294.67 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 13129.47 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139706.855147 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118165.20 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 11816.52 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151523.375102 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106348.68 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 10634.87 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162158.243061 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95713.81 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 9571.38 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171729.624225 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86142.43 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 8614.24 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180343.867272 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77528.19 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 7752.82 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188096.686015 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69775.37 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 6977.54 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195074.222883 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62797.83 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 6279.78 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201354.006064 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56518.05 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 5651.80 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207005.810928 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50866.24 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 5086.62 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212092.435305 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45779.62 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 4577.96 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216670.397244 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41201.66 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 4120.17 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220790.562989 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37081.49 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3708.15 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224498.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33373.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228098.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29773.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231698.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26173.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235298.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22573.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238898.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18973.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242498.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15373.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246098.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11773.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249698.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8173.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253298.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4573.34 sec) 05:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256898.712160 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 973.34 sec) 05:50:00 [GPUGRID] [rr_sim] 2257872.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992964459.17G/439.78G) 05:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851912.97 05:50:00 [---] pick_jobs_to_run() start 05:50:00 [---] pick_jobs_to_run() end 05:50:00 [---] [rr_sim] end 05:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:50:00 [---] [cpu_sched_debug] preliminary job list: 05:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:50:00 [---] [cpu_sched_debug] final job list: 05:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:50:00 [---] [cpu_sched_debug] enforce_run_list: end 05:50:00 [---] choose_project(): 21000.000000 05:50:00 [iThena.Measurements] doing RPC to get work 05:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414599.09 cpu 2159372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:50:00 [---] Got 0 tasks 05:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:51:00 [---] [rr_sim] doing sim: CPU sched 05:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992936868.81G/439.78G) 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257809.32 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 225780.93 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225780.931781 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2032028.39 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 203202.84 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428983.770385 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828825.55 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 182882.55 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611866.325128 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645942.99 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 164594.30 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776460.624397 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481348.69 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 148134.87 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924595.493738 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333213.82 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 133321.38 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057916.876146 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199892.44 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 119989.24 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177906.120313 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079903.20 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 107990.32 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285896.440063 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971912.88 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 97191.29 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383087.727838 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874721.59 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 87472.16 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470559.886836 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787249.43 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 78724.94 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549284.829934 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708524.49 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 70852.45 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620137.278722 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637672.04 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 63767.20 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683904.482631 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573904.84 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 57390.48 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741294.966150 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516514.35 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 51651.44 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792946.401316 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464862.92 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 46486.29 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839432.692966 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418376.62 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 41837.66 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881270.355451 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376538.96 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 37653.90 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918924.251687 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338885.07 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 33888.51 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952812.758300 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304996.56 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 30499.66 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983312.414252 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274496.90 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 27449.69 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010762.104608 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247047.21 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 24704.72 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035466.825929 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222342.49 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 22234.25 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057701.075117 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200108.24 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 20010.82 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077711.899387 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180097.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 18009.74 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095721.641230 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162087.68 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 16208.77 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111930.408888 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145878.91 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 14587.89 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126518.299781 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131291.02 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 13129.10 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139647.401584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118161.92 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 11816.19 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151463.593207 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106345.72 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 10634.57 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162098.165668 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95711.15 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 9571.12 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171669.280883 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86140.04 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 8614.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180283.284576 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77526.03 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 7752.60 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188035.887900 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69773.43 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 6977.34 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195013.230892 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62796.09 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 6279.61 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201292.839584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56516.48 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 5651.65 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206944.487407 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50864.83 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 5086.48 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212030.970448 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45778.35 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 4577.83 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216608.805184 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41200.51 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 4120.05 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220728.856448 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37080.46 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3708.05 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224436.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33372.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228036.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29772.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231636.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26172.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235236.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22572.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238836.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18972.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242436.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15372.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246036.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11772.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249636.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8172.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253236.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4572.42 sec) 05:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256836.902584 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 972.42 sec) 05:51:00 [GPUGRID] [rr_sim] 2257809.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992936868.81G/439.78G) 05:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851910.23 05:51:00 [---] pick_jobs_to_run() start 05:51:00 [---] pick_jobs_to_run() end 05:51:00 [---] [rr_sim] end 05:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:51:00 [---] [cpu_sched_debug] preliminary job list: 05:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:51:00 [---] [cpu_sched_debug] final job list: 05:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:51:00 [---] [cpu_sched_debug] enforce_run_list: end 05:51:00 [---] choose_project(): 21060.000000 05:51:00 [iThena.Measurements] doing RPC to get work 05:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414539.09 cpu 2159312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:51:00 [---] Got 0 tasks 05:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:52:00 [---] [rr_sim] doing sim: CPU sched 05:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992909278.45G/439.78G) 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257746.58 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 225774.66 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225774.658093 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031971.92 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 203197.19 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428971.850377 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828774.73 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 182877.47 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611849.323433 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645897.26 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 164589.73 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776439.049183 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481307.53 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 148130.75 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924569.802358 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333176.78 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 133317.68 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057887.480215 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199859.10 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 119985.91 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177873.390287 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079873.19 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 107987.32 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285860.709352 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971885.87 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 97188.59 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383049.296510 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874697.28 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 87469.73 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470519.024952 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787227.56 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 78722.76 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549241.780550 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708504.80 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 70850.48 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620092.260588 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637654.32 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 63765.43 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683857.692623 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573888.89 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 57388.89 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741246.581454 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516500.00 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 51650.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792896.581402 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464850.00 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 46485.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839381.581355 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418365.00 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 41836.50 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881218.081313 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376528.50 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 37652.85 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918870.931275 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338875.65 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 33887.56 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952758.496240 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304988.08 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 30498.81 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983257.304710 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274489.28 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 27448.93 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010706.232332 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247040.35 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 24704.03 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035410.267192 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222336.31 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 22233.63 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057643.898566 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200102.68 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 20010.27 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077654.166803 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180092.41 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 18009.24 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095663.408216 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162083.17 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 16208.32 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111871.725488 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145874.86 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 14587.49 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126459.211032 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131287.37 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 13128.74 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139587.948022 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118158.63 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 11815.86 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151403.811313 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106342.77 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 10634.28 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162038.088275 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95708.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 9570.85 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171608.937541 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86137.64 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 8613.76 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180222.701880 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77523.88 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 7752.39 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187975.089785 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69771.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 6977.15 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194952.238900 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62794.34 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 6279.43 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201231.673103 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56514.91 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 5651.49 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206883.163886 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50863.42 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 5086.34 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211969.505591 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45777.08 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 4577.71 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216547.213125 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41199.37 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 4119.94 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220667.149906 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37079.43 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3707.94 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224375.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33371.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227975.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29771.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231575.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26171.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235175.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22571.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238775.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18971.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242375.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15371.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245975.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11771.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249575.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8171.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253175.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4571.49 sec) 05:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256775.093009 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971.49 sec) 05:52:00 [GPUGRID] [rr_sim] 2257746.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992909278.45G/439.78G) 05:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851907.49 05:52:00 [---] pick_jobs_to_run() start 05:52:00 [---] pick_jobs_to_run() end 05:52:00 [---] [rr_sim] end 05:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:52:00 [---] [cpu_sched_debug] preliminary job list: 05:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:52:00 [---] [cpu_sched_debug] final job list: 05:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:52:00 [---] [cpu_sched_debug] enforce_run_list: end 05:52:00 [---] choose_project(): 21120.000000 05:52:00 [iThena.Measurements] doing RPC to get work 05:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414479.09 cpu 2159252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:52:00 [---] Got 0 tasks 05:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:53:00 [---] [rr_sim] doing sim: CPU sched 05:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992881688.10G/439.78G) 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257683.84 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 225768.38 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225768.384405 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031915.46 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 203191.55 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428959.930370 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828723.91 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 182872.39 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611832.321738 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645851.52 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 164585.15 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776417.473969 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481266.37 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 148126.64 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924544.110977 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333139.73 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 133313.97 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057858.084285 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199825.76 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 119982.58 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177840.660261 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079843.18 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 107984.32 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285824.978640 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971858.87 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 97185.89 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1383010.865181 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874672.98 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 87467.30 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470478.163068 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787205.68 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 78720.57 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549198.731166 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708485.11 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 70848.51 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620047.242455 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637636.60 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 63763.66 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683810.902614 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573872.94 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 57387.29 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741198.196758 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516485.65 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 51648.56 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792846.761487 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464837.08 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 46483.71 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839330.469744 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418353.37 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 41835.34 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881165.807174 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376518.04 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 37651.80 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918817.610862 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338866.23 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 33886.62 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952704.234181 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304979.61 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 30497.96 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983202.195168 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274481.65 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 27448.16 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010650.360056 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247033.48 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 24703.35 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035353.708456 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222330.14 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 22233.01 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057586.722015 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200097.12 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 20009.71 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077596.434219 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180087.41 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 18008.74 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095605.175202 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162078.67 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 16207.87 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111813.042087 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145870.80 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 14587.08 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126400.122283 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131283.72 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 13128.37 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139528.494460 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118155.35 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 11815.53 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151344.029419 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106339.81 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 10633.98 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161978.010882 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95705.83 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 9570.58 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171548.594199 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86135.25 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 8613.52 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180162.119184 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77521.72 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 7752.17 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187914.291671 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69769.55 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 6976.96 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194891.246909 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62792.60 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 6279.26 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201170.506623 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56513.34 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 5651.33 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206821.840366 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50862.00 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 5086.20 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211908.040734 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45775.80 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 4577.58 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216485.621066 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41198.22 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 4119.82 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220605.443364 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37078.40 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3707.84 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224313.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33370.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227913.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29770.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231513.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26170.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235113.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22570.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238713.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18970.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242313.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15370.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245913.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11770.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249513.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8170.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253113.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4570.56 sec) 05:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256713.283433 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970.56 sec) 05:53:00 [GPUGRID] [rr_sim] 2257683.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992881688.10G/439.78G) 05:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851904.76 05:53:00 [---] pick_jobs_to_run() start 05:53:00 [---] pick_jobs_to_run() end 05:53:00 [---] [rr_sim] end 05:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:53:00 [---] [cpu_sched_debug] preliminary job list: 05:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:53:00 [---] [cpu_sched_debug] final job list: 05:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:53:00 [---] [cpu_sched_debug] enforce_run_list: end 05:53:00 [---] choose_project(): 21180.000000 05:53:00 [iThena.Measurements] doing RPC to get work 05:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414419.09 cpu 2159192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:53:00 [---] Got 0 tasks 05:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:54:00 [---] [rr_sim] doing sim: CPU sched 05:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992854097.74G/439.78G) 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257621.11 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 225762.11 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225762.110717 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031859.00 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 203185.90 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428948.010362 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828673.10 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 182867.31 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611815.320043 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645805.79 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 164580.58 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776395.898755 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481225.21 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 148122.52 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924518.419597 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333102.69 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 133310.27 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057828.688354 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199792.42 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 119979.24 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177807.930235 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079813.18 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 107981.32 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285789.247929 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971831.86 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 97183.19 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382972.433853 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874648.67 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 87464.87 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470437.301184 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787183.81 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 78718.38 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549155.681783 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708465.43 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 70846.54 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1620002.224321 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637618.88 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 63761.89 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683764.112606 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573856.99 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 57385.70 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741149.812062 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516471.30 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 51647.13 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792796.941573 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464824.17 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 46482.42 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839279.358132 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418341.75 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 41834.17 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881113.533036 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376507.57 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 37650.76 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918764.290449 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338856.82 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 33885.68 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952649.972121 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304971.14 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 30497.11 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983147.085626 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274474.02 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 27447.40 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010594.487780 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247026.62 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 24702.66 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035297.149719 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222323.96 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 22232.40 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057529.545464 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200091.56 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 20009.16 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077538.701634 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180082.41 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 18008.24 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095546.942188 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162074.16 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 16207.42 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111754.358686 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145866.75 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 14586.67 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126341.033534 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131280.07 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 13128.01 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139469.040898 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118152.07 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 11815.21 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151284.247525 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106336.86 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 10633.69 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161917.933489 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95703.17 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 9570.32 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171488.250857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86132.86 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 8613.29 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180101.536488 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77519.57 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 7751.96 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187853.493556 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69767.61 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 6976.76 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194830.254917 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62790.85 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 6279.09 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201109.340143 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56511.77 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 5651.18 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206760.516845 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50860.59 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 5086.06 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211846.575877 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45774.53 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 4577.45 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216424.029007 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41197.08 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 4119.71 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220543.736823 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37077.37 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3707.74 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224251.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33369.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227851.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29769.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231451.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26169.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235051.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22569.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238651.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18969.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242251.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15369.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245851.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11769.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249451.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8169.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253051.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4569.63 sec) 05:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256651.473857 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969.63 sec) 05:54:00 [GPUGRID] [rr_sim] 2257621.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992854097.74G/439.78G) 05:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851902.02 05:54:00 [---] pick_jobs_to_run() start 05:54:00 [---] pick_jobs_to_run() end 05:54:00 [---] [rr_sim] end 05:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:54:00 [---] [cpu_sched_debug] preliminary job list: 05:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:54:00 [---] [cpu_sched_debug] final job list: 05:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:54:00 [---] [cpu_sched_debug] enforce_run_list: end 05:54:00 [---] choose_project(): 21240.000000 05:54:00 [iThena.Measurements] doing RPC to get work 05:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414359.09 cpu 2159132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:54:00 [---] Got 0 tasks 05:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:55:00 [---] [rr_sim] doing sim: CPU sched 05:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992826507.39G/439.78G) 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257558.37 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 225755.84 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225755.837029 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031802.53 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 203180.25 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428936.090354 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828622.28 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 182862.23 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611798.318348 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645760.05 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 164576.01 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776374.323541 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481184.05 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 148118.40 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924492.728216 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333065.64 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 133306.56 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057799.292423 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199759.08 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 119975.91 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177775.200209 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079783.17 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 107978.32 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285753.517217 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971804.85 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 97180.49 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382934.002524 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874624.37 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 87462.44 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470396.439300 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787161.93 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 78716.19 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549112.632399 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708445.74 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 70844.57 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619957.206188 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637601.16 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 63760.12 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683717.322597 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573841.05 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 57384.10 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741101.427366 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516456.94 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 51645.69 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792747.121658 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464811.25 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 46481.12 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839228.246521 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418330.12 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 41833.01 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881061.258898 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376497.11 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 37649.71 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918710.970037 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338847.40 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 33884.74 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952595.710061 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304962.66 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 30496.27 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983091.976084 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274466.39 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 27446.64 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010538.615504 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247019.75 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 24701.98 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035240.590982 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222317.78 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 22231.78 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057472.368913 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200086.00 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 20008.60 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077480.969050 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180077.40 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 18007.74 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095488.709174 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162069.66 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 16206.97 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111695.675285 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145862.70 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 14586.27 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126281.944785 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131276.43 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 13127.64 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139409.587335 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118148.78 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 11814.88 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151224.465630 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106333.90 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 10633.39 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161857.856096 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95700.51 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 9570.05 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171427.907515 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86130.46 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 8613.05 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180040.953792 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77517.42 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 7751.74 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187792.695442 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69765.67 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 6976.57 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194769.262926 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62789.11 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 6278.91 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201048.173662 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56510.20 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 5651.02 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206699.193324 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50859.18 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 5085.92 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211785.111021 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45773.26 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 4577.33 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216362.436947 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41195.93 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 4119.59 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220482.030281 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37076.34 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3707.63 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224189.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33368.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227789.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29768.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231389.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26168.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234989.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22568.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238589.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18968.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242189.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15368.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245789.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11768.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249389.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8168.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252989.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4568.71 sec) 05:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256589.664282 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968.71 sec) 05:55:00 [GPUGRID] [rr_sim] 2257558.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992826507.39G/439.78G) 05:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851899.28 05:55:00 [---] pick_jobs_to_run() start 05:55:00 [---] pick_jobs_to_run() end 05:55:00 [---] [rr_sim] end 05:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:55:00 [---] [cpu_sched_debug] preliminary job list: 05:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:55:00 [---] [cpu_sched_debug] final job list: 05:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:55:00 [---] [cpu_sched_debug] enforce_run_list: end 05:55:00 [---] choose_project(): 21300.000000 05:55:00 [iThena.Measurements] doing RPC to get work 05:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414299.09 cpu 2159072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:55:00 [---] Got 0 tasks 05:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:56:00 [---] [rr_sim] doing sim: CPU sched 05:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992798917.03G/439.78G) 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257495.63 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 225749.56 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225749.563340 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031746.07 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 203174.61 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428924.170347 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828571.46 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 182857.15 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611781.316653 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645714.32 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 164571.43 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776352.748328 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481142.89 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 148114.29 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924467.036835 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1333028.60 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 133302.86 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057769.896492 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199725.74 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 119972.57 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177742.470183 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079753.16 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 107975.32 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285717.786505 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971777.85 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 97177.78 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382895.571195 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874600.06 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 87460.01 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470355.577416 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787140.06 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 78714.01 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549069.583015 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708426.05 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 70842.61 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619912.188054 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637583.45 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 63758.34 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683670.532589 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573825.10 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 57382.51 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741053.042670 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516442.59 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 51644.26 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792697.301744 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464798.33 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 46479.83 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839177.134910 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418318.50 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 41831.85 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881008.984759 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376486.65 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 37648.66 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918657.649624 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338837.98 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 33883.80 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952541.448002 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304954.19 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 30495.42 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983036.866542 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274458.77 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 27445.88 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010482.743228 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247012.89 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 24701.29 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035184.032246 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222311.60 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 22231.16 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057415.192362 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200080.44 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 20008.04 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077423.236466 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180072.40 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 18007.24 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095430.476160 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162065.16 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 16206.52 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111636.991884 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145858.64 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 14585.86 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126222.856036 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131272.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 13127.28 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139350.133773 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118145.50 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 11814.55 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151164.683736 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106330.95 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 10633.09 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161797.778703 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95697.85 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 9569.79 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171367.564173 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86128.07 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 8612.81 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179980.371096 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77515.26 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 7751.53 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187731.897327 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69763.74 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 6976.37 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194708.270935 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62787.36 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 6278.74 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200987.007182 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56508.63 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 5650.86 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206637.869804 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50857.76 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 5085.78 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211723.646164 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45771.99 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 4577.20 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216300.844888 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41194.79 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 4119.48 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220420.323740 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37075.31 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3707.53 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224127.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33367.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227727.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29767.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231327.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26167.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234927.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22567.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238527.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18967.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242127.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15367.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245727.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11767.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249327.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8167.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252927.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4567.78 sec) 05:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256527.854706 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967.78 sec) 05:56:00 [GPUGRID] [rr_sim] 2257495.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992798917.03G/439.78G) 05:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851896.55 05:56:00 [---] pick_jobs_to_run() start 05:56:00 [---] pick_jobs_to_run() end 05:56:00 [---] [rr_sim] end 05:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:56:00 [---] [cpu_sched_debug] preliminary job list: 05:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:56:00 [---] [cpu_sched_debug] final job list: 05:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:56:00 [---] [cpu_sched_debug] enforce_run_list: end 05:56:00 [---] choose_project(): 21360.000000 05:56:00 [iThena.Measurements] doing RPC to get work 05:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 587039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414239.09 cpu 2159012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2159012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2159012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:56:00 [---] Got 0 tasks 05:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:57:00 [---] [rr_sim] doing sim: CPU sched 05:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992771326.67G/439.78G) 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257432.90 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 225743.29 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225743.289652 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031689.61 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 203168.96 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428912.250339 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828520.65 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 182852.06 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611764.314957 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645668.58 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 164566.86 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776331.173114 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481101.72 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 148110.17 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924441.345455 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332991.55 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 133299.16 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057740.500561 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199692.40 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 119969.24 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177709.740158 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079723.16 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 107972.32 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285682.055794 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971750.84 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 97175.08 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382857.139867 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874575.76 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 87457.58 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470314.715532 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787118.18 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 78711.82 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1549026.533631 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708406.36 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 70840.64 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619867.169920 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637565.73 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 63756.57 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683623.742580 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573809.15 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 57380.92 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741004.657975 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516428.24 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 51642.82 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792647.481829 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464785.41 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 46478.54 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839126.023299 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418306.87 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 41830.69 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880956.710621 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376476.19 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 37647.62 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918604.329211 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338828.57 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 33882.86 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952487.185942 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304945.71 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 30494.57 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982981.757000 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274451.14 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 27445.11 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010426.870952 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247006.03 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 24700.60 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035127.473509 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222305.42 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 22230.54 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057358.015811 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200074.88 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 20007.49 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077365.503882 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180067.39 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 18006.74 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095372.243146 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162060.65 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 16206.07 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111578.308483 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145854.59 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 14585.46 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126163.767287 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131269.13 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 13126.91 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139290.680211 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118142.22 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 11814.22 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151104.901842 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106327.99 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 10632.80 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161737.701310 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95695.20 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 9569.52 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171307.220831 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86125.68 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 8612.57 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179919.788400 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77513.11 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 7751.31 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187671.099212 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69761.80 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 6976.18 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194647.278943 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62785.62 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 6278.56 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200925.840701 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56507.06 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 5650.71 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206576.546283 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50856.35 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 5085.64 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211662.181307 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45770.72 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 4577.07 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216239.252829 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41193.64 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 4119.36 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220358.617198 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37074.28 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3707.43 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224066.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33366.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227666.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29766.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231266.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26166.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234866.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22566.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238466.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18966.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242066.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15366.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245666.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11766.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249266.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8166.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252866.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4566.85 sec) 05:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256466.045130 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966.85 sec) 05:57:00 [GPUGRID] [rr_sim] 2257432.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992771326.67G/439.78G) 05:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851893.81 05:57:00 [---] pick_jobs_to_run() start 05:57:00 [---] pick_jobs_to_run() end 05:57:00 [---] [rr_sim] end 05:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:57:00 [---] [cpu_sched_debug] preliminary job list: 05:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:57:00 [---] [cpu_sched_debug] final job list: 05:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:57:00 [---] [cpu_sched_debug] enforce_run_list: end 05:57:00 [---] choose_project(): 21420.000000 05:57:00 [iThena.Measurements] doing RPC to get work 05:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414179.09 cpu 2158952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:57:00 [---] Got 0 tasks 05:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:58:00 [---] [rr_sim] doing sim: CPU sched 05:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992743736.32G/439.78G) 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257370.16 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 225737.02 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225737.015964 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031633.14 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 203163.31 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428900.330332 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828469.83 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 182846.98 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611747.313262 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645622.85 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 164562.28 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776309.597900 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481060.56 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 148106.06 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924415.654074 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332954.51 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 133295.45 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057711.104631 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199659.06 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 119965.91 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177677.010132 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079693.15 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 107969.31 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285646.325082 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971723.83 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 97172.38 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382818.708538 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874551.45 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 87455.15 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470273.853648 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787096.31 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 78709.63 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548983.484247 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708386.68 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 70838.67 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619822.151787 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637548.01 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 63754.80 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683576.952572 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573793.21 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 57379.32 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740956.273279 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516413.89 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 51641.39 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792597.661915 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464772.50 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 46477.25 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839074.911687 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418295.25 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 41829.52 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880904.436483 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376465.72 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 37646.57 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918551.008798 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338819.15 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 33881.92 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952432.923883 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304937.24 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 30493.72 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982926.647458 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274443.51 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 27444.35 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010370.998676 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246999.16 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 24699.92 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035070.914773 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222299.24 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 22229.92 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057300.839259 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200069.32 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 20006.93 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077307.771297 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180062.39 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 18006.24 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095314.010132 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162056.15 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 16205.61 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111519.625083 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145850.53 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 14585.05 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126104.678538 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131265.48 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 13126.55 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139231.226648 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118138.93 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 11813.89 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151045.119948 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106325.04 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 10632.50 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161677.623917 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95692.54 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 9569.25 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171246.877489 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86123.28 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 8612.33 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179859.205704 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77510.95 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 7751.10 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187610.301098 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69759.86 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 6975.99 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194586.286952 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62783.87 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 6278.39 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200864.674221 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56505.49 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 5650.55 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206515.222763 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50854.94 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 5085.49 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211600.716450 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45769.44 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 4576.94 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216177.660769 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41192.50 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 4119.25 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220296.910656 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37073.25 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3707.32 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224004.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33365.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227604.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29765.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231204.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26165.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234804.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22565.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238404.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18965.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242004.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15365.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245604.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11765.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249204.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8165.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252804.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4565.92 sec) 05:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256404.235555 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965.92 sec) 05:58:00 [GPUGRID] [rr_sim] 2257370.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992743736.32G/439.78G) 05:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851891.07 05:58:00 [---] pick_jobs_to_run() start 05:58:00 [---] pick_jobs_to_run() end 05:58:00 [---] [rr_sim] end 05:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:58:00 [---] [cpu_sched_debug] preliminary job list: 05:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:58:00 [---] [cpu_sched_debug] final job list: 05:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:58:00 [---] [cpu_sched_debug] enforce_run_list: end 05:58:00 [---] choose_project(): 21480.000000 05:58:00 [iThena.Measurements] doing RPC to get work 05:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414119.09 cpu 2158892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:58:00 [---] Got 0 tasks 05:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 05:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 05:59:00 [---] [rr_sim] doing sim: CPU sched 05:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992716145.96G/439.78G) 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257307.42 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 225730.74 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225730.742276 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031576.68 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 203157.67 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428888.410324 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828419.01 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 182841.90 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611730.311567 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645577.11 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 164557.71 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776288.022686 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1481019.40 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 148101.94 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924389.962693 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332917.46 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 133291.75 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057681.708700 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199625.71 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 119962.57 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177644.280106 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079663.14 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 107966.31 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285610.594371 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971696.83 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 97169.68 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382780.277210 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874527.15 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 87452.71 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470232.991764 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787074.43 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 78707.44 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548940.434864 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708366.99 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 70836.70 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619777.133653 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637530.29 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 63753.03 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683530.162564 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573777.26 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 57377.73 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740907.888583 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516399.53 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 51639.95 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792547.842000 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464759.58 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 46475.96 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839023.800076 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418283.62 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 41828.36 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880852.162344 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376455.26 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 37645.53 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918497.688386 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338809.73 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 33880.97 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952378.661823 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304928.76 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 30492.88 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982871.537916 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274435.88 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 27443.59 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010315.126400 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246992.30 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 24699.23 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035014.356036 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222293.07 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 22229.31 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057243.662708 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200063.76 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 20006.38 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077250.038713 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180057.38 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 18005.74 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095255.777118 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162051.65 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 16205.16 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111460.941682 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145846.48 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 14584.65 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126045.589789 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131261.83 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 13126.18 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139171.773086 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118135.65 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 11813.56 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150985.338053 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106322.08 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 10632.21 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161617.546524 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95689.88 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 9568.99 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171186.534147 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86120.89 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 8612.09 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179798.623008 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77508.80 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 7750.88 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187549.502983 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69757.92 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 6975.79 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194525.294961 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62782.13 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 6278.21 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200803.507740 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56503.92 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 5650.39 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206453.899242 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50853.52 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 5085.35 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211539.251594 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45768.17 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 4576.82 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216116.068710 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41191.35 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 4119.14 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220235.204115 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37072.22 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3707.22 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223942.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33365.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227542.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29765.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231142.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26165.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234742.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22565.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238342.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18965.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241942.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15365.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245542.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11765.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249142.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8165.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252742.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4565.00 sec) 05:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256342.425979 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965.00 sec) 05:59:00 [GPUGRID] [rr_sim] 2257307.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992716145.96G/439.78G) 05:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851888.34 05:59:00 [---] pick_jobs_to_run() start 05:59:00 [---] pick_jobs_to_run() end 05:59:00 [---] [rr_sim] end 05:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 05:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 05:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 05:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 05:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 05:59:00 [---] [cpu_sched_debug] preliminary job list: 05:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:59:00 [---] [cpu_sched_debug] final job list: 05:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 05:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 05:59:00 [---] [cpu_sched_debug] enforce_run_list: end 05:59:00 [---] choose_project(): 21540.000000 05:59:00 [iThena.Measurements] doing RPC to get work 05:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 414059.09 cpu 2158832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 05:59:00 [---] Got 0 tasks 06:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:00:00 [---] [rr_sim] doing sim: CPU sched 06:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992688555.60G/439.78G) 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257244.69 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 225724.47 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225724.468588 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031520.22 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 203152.02 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428876.490316 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828368.20 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 182836.82 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611713.309872 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645531.38 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 164553.14 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776266.447473 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480978.24 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 148097.82 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924364.271313 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332880.41 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 133288.04 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057652.312769 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199592.37 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 119959.24 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177611.550080 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079633.14 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 107963.31 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285574.863659 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971669.82 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 97166.98 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382741.845881 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874502.84 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 87450.28 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470192.129880 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787052.56 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 78705.26 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548897.385480 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708347.30 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 70834.73 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619732.115519 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637512.57 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 63751.26 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683483.372555 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573761.31 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 57376.13 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740859.503887 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516385.18 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 51638.52 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792498.022086 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464746.66 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 46474.67 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838972.688465 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418272.00 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 41827.20 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880799.888206 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376444.80 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 37644.48 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918444.367973 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338800.32 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 33880.03 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952324.399763 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304920.29 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 30492.03 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982816.428374 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274428.26 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 27442.83 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010259.254125 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246985.43 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 24698.54 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034957.797300 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222286.89 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 22228.69 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057186.486157 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200058.20 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 20005.82 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077192.306129 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180052.38 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 18005.24 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095197.544104 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162047.14 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 16204.71 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111402.258281 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145842.43 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 14584.24 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125986.501040 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131258.18 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 13125.82 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139112.319524 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118132.37 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 11813.24 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150925.556159 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106319.13 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 10631.91 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161557.469131 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95687.22 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 9568.72 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171126.190805 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86118.50 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 8611.85 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179738.040312 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77506.65 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 7750.66 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187488.704869 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69755.98 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 6975.60 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194464.302969 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62780.38 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 6278.04 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200742.341260 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56502.34 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 5650.23 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206392.575721 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50852.11 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 5085.21 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211477.786737 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45766.90 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 4576.69 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216054.476651 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41190.21 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 4119.02 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220173.497573 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37071.19 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3707.12 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223880.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33364.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227480.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29764.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231080.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26164.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234680.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22564.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238280.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18964.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241880.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15364.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245480.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11764.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249080.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8164.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252680.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4564.07 sec) 06:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256280.616403 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964.07 sec) 06:00:00 [GPUGRID] [rr_sim] 2257244.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992688555.60G/439.78G) 06:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851885.60 06:00:00 [---] pick_jobs_to_run() start 06:00:00 [---] pick_jobs_to_run() end 06:00:00 [---] [rr_sim] end 06:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:00:00 [---] [cpu_sched_debug] preliminary job list: 06:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:00:00 [---] [cpu_sched_debug] final job list: 06:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:00:00 [---] [cpu_sched_debug] enforce_run_list: end 06:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:01:00 [---] [rr_sim] doing sim: CPU sched 06:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992660965.25G/439.78G) 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257181.95 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 225718.19 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225718.194899 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031463.75 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 203146.38 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428864.570309 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828317.38 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 182831.74 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611696.308177 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645485.64 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 164548.56 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776244.872259 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480937.08 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 148093.71 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924338.579932 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332843.37 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 133284.34 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057622.916838 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199559.03 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 119955.90 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177578.820054 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079603.13 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 107960.31 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285539.132948 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971642.82 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 97164.28 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382703.414552 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874478.53 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 87447.85 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470151.267996 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787030.68 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 78703.07 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548854.336096 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708327.61 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 70832.76 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619687.097386 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637494.85 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 63749.49 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683436.582547 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573745.37 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 57374.54 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740811.119191 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516370.83 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 51637.08 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792448.202171 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464733.75 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 46473.37 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838921.576854 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418260.37 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 41826.04 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880747.614068 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376434.33 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 37643.43 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918391.047560 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338790.90 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 33879.09 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952270.137704 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304911.81 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 30491.18 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982761.318833 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274420.63 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 27442.06 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010203.381849 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246978.57 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 24697.86 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034901.238563 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222280.71 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 22228.07 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057129.309606 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200052.64 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 20005.26 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077134.573545 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180047.38 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 18004.74 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095139.311090 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162042.64 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 16204.26 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111343.574880 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145838.37 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 14583.84 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125927.412291 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131254.54 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 13125.45 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139052.865962 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118129.08 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 11812.91 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150865.774265 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106316.17 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 10631.62 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161497.391738 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95684.56 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 9568.46 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171065.847463 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86116.10 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 8611.61 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179677.457616 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77504.49 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 7750.45 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187427.906754 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69754.04 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 6975.40 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194403.310978 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62778.64 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 6277.86 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200681.174779 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56500.77 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 5650.08 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206331.252201 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50850.70 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 5085.07 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211416.321880 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45765.63 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 4576.56 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215992.884591 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41189.06 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 4118.91 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220111.791032 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37070.16 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3707.02 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223818.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33363.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227418.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29763.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231018.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26163.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234618.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22563.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238218.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18963.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241818.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15363.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245418.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11763.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249018.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8163.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252618.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4563.14 sec) 06:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256218.806828 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963.14 sec) 06:01:00 [GPUGRID] [rr_sim] 2257181.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992660965.25G/439.78G) 06:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851882.86 06:01:00 [---] pick_jobs_to_run() start 06:01:00 [---] pick_jobs_to_run() end 06:01:00 [---] [rr_sim] end 06:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:01:00 [---] [cpu_sched_debug] preliminary job list: 06:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:01:00 [---] [cpu_sched_debug] final job list: 06:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:01:00 [---] [cpu_sched_debug] enforce_run_list: end 06:01:00 [---] choose_project(): 21660.000000 06:01:00 [iThena.Measurements] doing RPC to get work 06:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413939.09 cpu 2158712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:01:00 [---] Got 0 tasks 06:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:02:00 [---] [rr_sim] doing sim: CPU sched 06:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992633374.89G/439.78G) 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257119.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 225711.92 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225711.921211 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031407.29 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 203140.73 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428852.650301 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828266.56 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 182826.66 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611679.306482 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645439.91 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 164543.99 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776223.297045 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480895.92 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 148089.59 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924312.888552 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332806.32 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 133280.63 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057593.520908 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199525.69 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 119952.57 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177546.090028 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079573.12 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 107957.31 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285503.402236 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971615.81 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 97161.58 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382664.983224 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874454.23 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 87445.42 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470110.406113 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787008.81 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 78700.88 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548811.286712 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708307.93 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 70830.79 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619642.079252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637477.13 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 63747.71 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683389.792538 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573729.42 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 57372.94 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740762.734495 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516356.48 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 51635.65 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792398.382257 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464720.83 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 46472.08 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838870.465242 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418248.75 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 41824.87 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880695.339929 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376423.87 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 37642.39 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918337.727147 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338781.48 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 33878.15 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952215.875644 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304903.34 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 30490.33 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982706.209291 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274413.00 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 27441.30 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010147.509573 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246971.70 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 24697.17 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034844.679827 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222274.53 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 22227.45 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057072.133055 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200047.08 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 20004.71 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077076.840961 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180042.37 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 18004.24 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095081.078076 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162038.13 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 16203.81 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111284.891479 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145834.32 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 14583.43 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125868.323542 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131250.89 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 13125.09 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138993.412399 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118125.80 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 11812.58 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150805.992370 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106313.22 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 10631.32 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161437.314345 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95681.90 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 9568.19 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171005.504121 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86113.71 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 8611.37 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179616.874920 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77502.34 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 7750.23 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187367.108639 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69752.10 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 6975.21 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194342.318986 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62776.89 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 6277.69 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200620.008299 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56499.20 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 5649.92 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206269.928680 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50849.28 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 5084.93 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211354.857023 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45764.36 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 4576.44 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215931.292532 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41187.92 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 4118.79 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220050.084490 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37069.13 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3706.91 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223756.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33362.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227356.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29762.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230956.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26162.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234556.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22562.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238156.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18962.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241756.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15362.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245356.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11762.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248956.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8162.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252556.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4562.21 sec) 06:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256156.997252 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962.21 sec) 06:02:00 [GPUGRID] [rr_sim] 2257119.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992633374.89G/439.78G) 06:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851880.12 06:02:00 [---] pick_jobs_to_run() start 06:02:00 [---] pick_jobs_to_run() end 06:02:00 [---] [rr_sim] end 06:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:02:00 [---] [cpu_sched_debug] preliminary job list: 06:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:02:00 [---] [cpu_sched_debug] final job list: 06:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:02:00 [---] [cpu_sched_debug] enforce_run_list: end 06:02:00 [---] choose_project(): 21720.000000 06:02:00 [iThena.Measurements] doing RPC to get work 06:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413879.09 cpu 2158652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:02:00 [---] Got 0 tasks 06:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:03:00 [---] [rr_sim] doing sim: CPU sched 06:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992605784.54G/439.78G) 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2257056.48 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 225705.65 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225705.647523 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031350.83 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 203135.08 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428840.730294 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828215.74 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 182821.57 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611662.304787 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645394.17 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 164539.42 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776201.721831 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480854.75 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 148085.48 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924287.197171 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332769.28 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 133276.93 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057564.124977 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199492.35 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 119949.24 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177513.360002 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079543.12 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 107954.31 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285467.671525 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971588.80 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 97158.88 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382626.551895 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874429.92 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 87442.99 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470069.544229 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786986.93 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 78698.69 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548768.237329 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708288.24 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 70828.82 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619597.061119 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637459.41 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 63745.94 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683343.002530 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573713.47 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 57371.35 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740714.349800 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516342.13 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 51634.21 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792348.562343 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464707.91 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 46470.79 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838819.353631 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418237.12 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 41823.71 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880643.065791 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376413.41 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 37641.34 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918284.406735 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338772.07 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 33877.21 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952161.613584 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304894.86 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 30489.49 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982651.099749 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274405.38 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 27440.54 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010091.637297 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246964.84 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 24696.48 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034788.121090 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222268.35 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 22226.84 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057014.956504 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200041.52 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 20004.15 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077019.108376 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180037.37 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 18003.74 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095022.845062 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162033.63 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 16203.36 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111226.208078 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145830.27 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 14583.03 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125809.234793 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131247.24 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 13124.72 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138933.958837 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118122.52 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 11812.25 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150746.210476 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106310.26 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 10631.03 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161377.236951 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95679.24 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 9567.92 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170945.160779 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86111.31 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 8611.13 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179556.292224 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77500.18 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 7750.02 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187306.310525 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69750.16 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 6975.02 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194281.326995 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62775.15 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 6277.51 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200558.841819 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56497.63 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 5649.76 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206208.605160 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50847.87 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 5084.79 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211293.392167 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45763.08 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 4576.31 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215869.700473 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41186.77 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 4118.68 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219988.377948 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37068.10 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3706.81 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223695.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33361.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227295.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29761.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230895.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26161.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234495.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22561.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238095.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18961.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241695.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15361.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245295.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11761.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248895.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8161.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252495.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4561.29 sec) 06:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256095.187676 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961.29 sec) 06:03:00 [GPUGRID] [rr_sim] 2257056.48: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992605784.54G/439.78G) 06:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851877.39 06:03:00 [---] pick_jobs_to_run() start 06:03:00 [---] pick_jobs_to_run() end 06:03:00 [---] [rr_sim] end 06:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:03:00 [---] [cpu_sched_debug] preliminary job list: 06:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:03:00 [---] [cpu_sched_debug] final job list: 06:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:03:00 [---] [cpu_sched_debug] enforce_run_list: end 06:03:00 [---] choose_project(): 21780.000000 06:03:00 [iThena.Measurements] doing RPC to get work 06:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413819.09 cpu 2158592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:03:00 [---] Got 0 tasks 06:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:04:00 [---] [rr_sim] doing sim: CPU sched 06:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992578194.18G/439.78G) 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256993.74 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 225699.37 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225699.373835 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031294.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 203129.44 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428828.810286 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828164.93 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 182816.49 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611645.303092 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645348.44 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 164534.84 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776180.146617 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480813.59 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 148081.36 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924261.505790 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332732.23 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 133273.22 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057534.729046 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199459.01 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 119945.90 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177480.629976 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079513.11 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 107951.31 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285431.940813 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971561.80 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 97156.18 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382588.120567 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874405.62 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 87440.56 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1470028.682345 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786965.06 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 78696.51 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548725.187945 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708268.55 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 70826.86 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619552.042985 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637441.70 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 63744.17 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683296.212521 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573697.53 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 57369.75 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740665.965104 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516327.77 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 51632.78 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792298.742428 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464695.00 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 46469.50 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838768.242020 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418225.50 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 41822.55 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880590.791653 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376402.95 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 37640.29 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918231.086322 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338762.65 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 33876.27 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952107.351525 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304886.39 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 30488.64 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982595.990207 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274397.75 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 27439.77 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010035.765021 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246957.97 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 24695.80 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034731.562353 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222262.18 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 22226.22 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056957.779953 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200035.96 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 20003.60 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076961.375792 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180032.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 18003.24 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094964.612048 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162029.13 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 16202.91 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111167.524678 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145826.21 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 14582.62 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125750.146044 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131243.59 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 13124.36 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138874.505275 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118119.23 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 11811.92 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150686.428582 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106307.31 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 10630.73 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161317.159558 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95676.58 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 9567.66 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170884.817437 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86108.92 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 8610.89 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179495.709528 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77498.03 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 7749.80 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187245.512410 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69748.23 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 6974.82 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194220.335004 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62773.40 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 6277.34 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200497.675338 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56496.06 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 5649.61 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206147.281639 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50846.46 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 5084.65 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211231.927310 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45761.81 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 4576.18 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215808.108413 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41185.63 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 4118.56 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219926.671407 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37067.07 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3706.71 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223633.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33360.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227233.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29760.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230833.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26160.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234433.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22560.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238033.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18960.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241633.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15360.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245233.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11760.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248833.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8160.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252433.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4560.36 sec) 06:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2256033.378101 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960.36 sec) 06:04:00 [GPUGRID] [rr_sim] 2256993.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992578194.18G/439.78G) 06:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851874.65 06:04:00 [---] pick_jobs_to_run() start 06:04:00 [---] pick_jobs_to_run() end 06:04:00 [---] [rr_sim] end 06:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:04:00 [---] [cpu_sched_debug] preliminary job list: 06:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:04:00 [---] [cpu_sched_debug] final job list: 06:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:04:00 [---] [cpu_sched_debug] enforce_run_list: end 06:04:00 [---] choose_project(): 21840.000000 06:04:00 [iThena.Measurements] doing RPC to get work 06:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413759.09 cpu 2158532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:04:00 [---] Got 0 tasks 06:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:05:00 [---] [rr_sim] doing sim: CPU sched 06:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992550603.82G/439.78G) 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256931.00 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 225693.10 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225693.100146 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031237.90 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 203123.79 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428816.890278 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828114.11 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 182811.41 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611628.301397 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645302.70 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 164530.27 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776158.571404 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480772.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 148077.24 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924235.814410 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332695.19 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 133269.52 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057505.333115 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199425.67 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 119942.57 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177447.899950 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079483.10 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 107948.31 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285396.210102 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971534.79 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 97153.48 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382549.689238 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874381.31 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 87438.13 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469987.820461 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786943.18 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 78694.32 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548682.138561 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708248.86 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 70824.89 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619507.024851 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637423.98 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 63742.40 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683249.422513 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573681.58 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 57368.16 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740617.580408 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516313.42 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 51631.34 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792248.922514 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464682.08 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 46468.21 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838717.130409 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418213.87 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 41821.39 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880538.517514 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376392.48 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 37639.25 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918177.765909 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338753.24 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 33875.32 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952053.089465 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304877.91 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 30487.79 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982540.880665 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274390.12 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 27439.01 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009979.892745 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246951.11 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 24695.11 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034675.003617 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222256.00 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 22225.60 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056900.603402 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200030.40 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 20003.04 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076903.643208 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180027.36 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 18002.74 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094906.379034 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162024.62 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 16202.46 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111108.841277 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145822.16 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 14582.22 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125691.057296 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131239.94 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 13123.99 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138815.051712 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118115.95 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 11811.59 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150626.646688 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106304.35 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 10630.44 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161257.082165 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95673.92 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 9567.39 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170824.474095 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86106.53 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 8610.65 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179435.126832 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77495.87 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 7749.59 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187184.714295 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69746.29 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 6974.63 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194159.343012 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62771.66 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 6277.17 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200436.508858 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56494.49 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 5649.45 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206085.958118 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50845.04 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 5084.50 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211170.462453 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45760.54 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 4576.05 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215746.516354 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41184.49 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 4118.45 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219864.964865 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37066.04 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3706.60 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223571.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33359.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227171.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29759.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230771.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26159.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234371.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22559.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237971.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18959.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241571.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15359.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245171.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11759.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248771.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8159.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252371.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4559.43 sec) 06:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255971.568525 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959.43 sec) 06:05:00 [GPUGRID] [rr_sim] 2256931.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992550603.82G/439.78G) 06:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851871.91 06:05:00 [---] pick_jobs_to_run() start 06:05:00 [---] pick_jobs_to_run() end 06:05:00 [---] [rr_sim] end 06:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:05:00 [---] [cpu_sched_debug] preliminary job list: 06:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:05:00 [---] [cpu_sched_debug] final job list: 06:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:05:00 [---] [cpu_sched_debug] enforce_run_list: end 06:05:00 [---] choose_project(): 21900.000000 06:05:00 [iThena.Measurements] doing RPC to get work 06:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413699.09 cpu 2158472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:05:00 [---] Got 0 tasks 06:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:06:00 [---] [rr_sim] doing sim: CPU sched 06:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992523013.47G/439.78G) 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256868.26 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 225686.83 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225686.826458 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031181.44 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 203118.14 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428804.970271 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828063.29 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 182806.33 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611611.299702 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645256.96 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 164525.70 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776136.996190 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480731.27 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 148073.13 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924210.123029 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332658.14 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 133265.81 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057475.937185 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199392.33 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 119939.23 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177415.169924 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079453.09 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 107945.31 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285360.479390 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971507.79 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 97150.78 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382511.257909 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874357.01 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 87435.70 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469946.958577 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786921.31 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 78692.13 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548639.089177 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708229.18 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 70822.92 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619462.006718 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637406.26 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 63740.63 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683202.632504 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573665.63 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 57366.56 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740569.195712 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516299.07 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 51629.91 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792199.102599 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464669.16 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 46466.92 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838666.018797 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418202.25 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 41820.22 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880486.243376 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376382.02 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 37638.20 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918124.445497 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338743.82 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 33874.38 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951998.827405 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304869.44 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 30486.94 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982485.771123 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274382.49 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 27438.25 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009924.020469 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246944.24 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 24694.42 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034618.444880 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222249.82 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 22224.98 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056843.426850 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200024.84 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 20002.48 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076845.910624 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180022.35 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 18002.24 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094848.146020 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162020.12 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 16202.01 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111050.157876 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145818.11 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 14581.81 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125631.968547 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131236.30 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 13123.63 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138755.598150 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118112.67 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 11811.27 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150566.864793 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106301.40 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 10630.14 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161197.004772 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95671.26 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 9567.13 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170764.130753 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86104.13 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 8610.41 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179374.544136 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77493.72 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 7749.37 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187123.916181 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69744.35 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 6974.43 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194098.351021 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62769.91 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 6276.99 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200375.342377 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56492.92 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 5649.29 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206024.634598 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50843.63 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 5084.36 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211108.997596 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45759.27 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 4575.93 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215684.924295 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41183.34 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 4118.33 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219803.258324 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37065.01 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3706.50 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223509.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33358.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227109.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29758.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230709.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26158.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234309.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22558.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237909.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18958.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241509.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15358.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245109.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11758.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248709.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8158.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252309.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4558.51 sec) 06:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255909.758949 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958.51 sec) 06:06:00 [GPUGRID] [rr_sim] 2256868.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992523013.47G/439.78G) 06:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851869.18 06:06:00 [---] pick_jobs_to_run() start 06:06:00 [---] pick_jobs_to_run() end 06:06:00 [---] [rr_sim] end 06:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:06:00 [---] [cpu_sched_debug] preliminary job list: 06:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:06:00 [---] [cpu_sched_debug] final job list: 06:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:06:00 [---] [cpu_sched_debug] enforce_run_list: end 06:06:00 [---] choose_project(): 21960.000000 06:06:00 [iThena.Measurements] doing RPC to get work 06:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413639.09 cpu 2158412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:06:00 [---] Got 0 tasks 06:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:07:00 [---] [rr_sim] doing sim: CPU sched 06:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992495423.11G/439.78G) 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256805.53 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 225680.55 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225680.552770 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031124.97 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 203112.50 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428793.050263 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1828012.48 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 182801.25 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611594.298007 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645211.23 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 164521.12 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776115.420976 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480690.11 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 148069.01 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924184.431649 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332621.10 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 133262.11 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057446.541254 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199358.99 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 119935.90 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177382.439898 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079423.09 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 107942.31 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285324.748679 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971480.78 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 97148.08 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382472.826581 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874332.70 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 87433.27 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469906.096693 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786899.43 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 78689.94 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548596.039793 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708209.49 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 70820.95 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619416.988584 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637388.54 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 63738.85 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683155.842496 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573649.69 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 57364.97 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740520.811016 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516284.72 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 51628.47 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792149.282685 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464656.25 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 46465.62 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838614.907186 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418190.62 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 41819.06 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880433.969238 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376371.56 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 37637.16 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918071.125084 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338734.40 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 33873.44 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951944.565346 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304860.96 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 30486.10 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982430.661581 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274374.87 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 27437.49 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009868.148193 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246937.38 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 24693.74 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034561.886144 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222243.64 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 22224.36 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056786.250299 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200019.28 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 20001.93 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076788.178039 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180017.35 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 18001.73 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094789.913006 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162015.61 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 16201.56 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110991.474475 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145814.05 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 14581.41 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125572.879798 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131232.65 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 13123.26 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138696.144588 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118109.38 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 11810.94 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150507.082899 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106298.44 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 10629.84 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161136.927379 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95668.60 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 9566.86 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170703.787411 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86101.74 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 8610.17 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179313.961440 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77491.57 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 7749.16 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187063.118066 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69742.41 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 6974.24 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194037.359030 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62768.17 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 6276.82 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200314.175897 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56491.35 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 5649.14 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205963.311077 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50842.22 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 5084.22 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211047.532739 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45757.99 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 4575.80 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215623.332236 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41182.20 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 4118.22 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219741.551782 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37063.98 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3706.40 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223447.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33357.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227047.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29757.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230647.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26157.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234247.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22557.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237847.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18957.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241447.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15357.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245047.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11757.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248647.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8157.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252247.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4557.58 sec) 06:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255847.949374 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957.58 sec) 06:07:00 [GPUGRID] [rr_sim] 2256805.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992495423.11G/439.78G) 06:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851866.44 06:07:00 [---] pick_jobs_to_run() start 06:07:00 [---] pick_jobs_to_run() end 06:07:00 [---] [rr_sim] end 06:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:07:00 [---] [cpu_sched_debug] preliminary job list: 06:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:07:00 [---] [cpu_sched_debug] final job list: 06:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:07:00 [---] [cpu_sched_debug] enforce_run_list: end 06:07:00 [---] choose_project(): 22020.000000 06:07:00 [iThena.Measurements] doing RPC to get work 06:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413579.09 cpu 2158352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:07:00 [---] Got 0 tasks 06:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:08:00 [---] [rr_sim] doing sim: CPU sched 06:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992467832.75G/439.78G) 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256742.79 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 225674.28 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225674.279082 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031068.51 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 203106.85 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428781.130255 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827961.66 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 182796.17 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611577.296312 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645165.49 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 164516.55 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776093.845762 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480648.95 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 148064.89 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924158.740268 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332584.05 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 133258.41 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057417.145323 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199325.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 119932.56 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177349.709872 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079393.08 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 107939.31 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285289.017967 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971453.77 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 97145.38 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382434.395252 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874308.40 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 87430.84 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469865.234809 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786877.56 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 78687.76 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548552.990410 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708189.80 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 70818.98 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619371.970451 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637370.82 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 63737.08 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683109.052487 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573633.74 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 57363.37 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740472.426320 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516270.36 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 51627.04 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792099.462770 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464643.33 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 46464.33 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838563.795575 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418179.00 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 41817.90 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880381.695099 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376361.10 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 37636.11 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918017.804671 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338724.99 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 33872.50 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951890.303286 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304852.49 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 30485.25 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982375.552039 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274367.24 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 27436.72 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009812.275917 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246930.51 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 24693.05 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034505.327407 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222237.46 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 22223.75 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056729.073748 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200013.72 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 20001.37 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076730.445455 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180012.35 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 18001.23 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094731.679992 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162011.11 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 16201.11 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110932.791074 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145810.00 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 14581.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125513.791049 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131229.00 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 13122.90 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138636.691026 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118106.10 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 11810.61 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150447.301005 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106295.49 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 10629.55 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161076.849986 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95665.94 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 9566.59 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170643.444069 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86099.35 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 8609.93 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179253.378744 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77489.41 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 7748.94 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187002.319952 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69740.47 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 6974.05 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193976.367038 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62766.42 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 6276.64 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200253.009416 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56489.78 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 5648.98 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205901.987556 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50840.80 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 5084.08 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210986.067883 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45756.72 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 4575.67 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215561.740176 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41181.05 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 4118.11 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219679.845240 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37062.95 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3706.29 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223386.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33356.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226986.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29756.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230586.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26156.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234186.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22556.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237786.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18956.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241386.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15356.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244986.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11756.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248586.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8156.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252186.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4556.65 sec) 06:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255786.139798 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956.65 sec) 06:08:00 [GPUGRID] [rr_sim] 2256742.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992467832.75G/439.78G) 06:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851863.70 06:08:00 [---] pick_jobs_to_run() start 06:08:00 [---] pick_jobs_to_run() end 06:08:00 [---] [rr_sim] end 06:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:08:00 [---] [cpu_sched_debug] preliminary job list: 06:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:08:00 [---] [cpu_sched_debug] final job list: 06:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:08:00 [---] [cpu_sched_debug] enforce_run_list: end 06:08:00 [---] choose_project(): 22080.000000 06:08:00 [iThena.Measurements] doing RPC to get work 06:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413519.09 cpu 2158292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:08:00 [---] Got 0 tasks 06:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:09:00 [---] [rr_sim] doing sim: CPU sched 06:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992440242.40G/439.78G) 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256680.05 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 225668.01 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225668.005394 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2031012.05 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 203101.20 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428769.210248 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827910.84 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 182791.08 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611560.294617 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645119.76 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 164511.98 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776072.270549 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480607.78 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 148060.78 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924133.048887 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332547.01 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 133254.70 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057387.749392 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199292.30 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 119929.23 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177316.979847 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079363.07 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 107936.31 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285253.287256 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971426.77 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 97142.68 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382395.963924 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874284.09 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 87428.41 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469824.372925 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786855.68 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 78685.57 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548509.941026 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708170.11 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 70817.01 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619326.952317 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637353.10 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 63735.31 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683062.262479 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573617.79 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 57361.78 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740424.041625 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516256.01 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 51625.60 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792049.642856 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464630.41 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 46463.04 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838512.683964 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418167.37 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 41816.74 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880329.420961 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376350.63 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 37635.06 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917964.484258 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338715.57 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 33871.56 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951836.041226 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304844.01 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 30484.40 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982320.442497 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274359.61 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 27435.96 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009756.403641 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246923.65 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 24692.37 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034448.768671 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222231.29 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 22223.13 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056671.897197 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200008.16 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 20000.82 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076672.712871 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180007.34 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 18000.73 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094673.446978 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162006.61 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 16200.66 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110874.107673 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145805.95 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 14580.59 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125454.702300 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131225.35 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 13122.54 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138577.237463 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118102.82 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 11810.28 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150387.519111 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106292.53 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 10629.25 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161016.772593 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95663.28 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 9566.33 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170583.100727 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86096.95 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 8609.70 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179192.796048 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77487.26 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 7748.73 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186941.521837 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69738.53 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 6973.85 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193915.375047 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62764.68 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 6276.47 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200191.842936 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56488.21 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 5648.82 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205840.664036 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50839.39 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 5083.94 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210924.603026 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45755.45 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 4575.55 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215500.148117 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41179.91 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 4117.99 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219618.138699 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37061.92 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3706.19 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223324.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33355.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226924.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29755.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230524.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26155.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234124.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22555.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237724.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18955.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241324.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15355.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244924.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11755.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248524.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8155.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252124.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4555.72 sec) 06:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255724.330223 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955.72 sec) 06:09:00 [GPUGRID] [rr_sim] 2256680.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992440242.40G/439.78G) 06:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851860.97 06:09:00 [---] pick_jobs_to_run() start 06:09:00 [---] pick_jobs_to_run() end 06:09:00 [---] [rr_sim] end 06:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:09:00 [---] [cpu_sched_debug] preliminary job list: 06:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:09:00 [---] [cpu_sched_debug] final job list: 06:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:09:00 [---] [cpu_sched_debug] enforce_run_list: end 06:09:00 [---] choose_project(): 22140.000000 06:09:00 [iThena.Measurements] doing RPC to get work 06:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413459.09 cpu 2158232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:09:00 [---] Got 0 tasks 06:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:10:00 [---] [rr_sim] doing sim: CPU sched 06:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992412652.04G/439.78G) 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256617.32 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 225661.73 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225661.731705 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030955.59 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 203095.56 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428757.290240 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827860.03 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 182786.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611543.292922 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645074.02 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 164507.40 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776050.695335 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480566.62 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 148056.66 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924107.357507 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332509.96 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 133251.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057358.353461 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199258.96 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 119925.90 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177284.249821 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079333.07 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 107933.31 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285217.556544 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971399.76 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 97139.98 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382357.532595 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874259.78 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 87425.98 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469783.511041 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786833.81 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 78683.38 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548466.891642 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708150.43 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 70815.04 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619281.934183 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637335.38 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 63733.54 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1683015.472470 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573601.84 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 57360.18 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740375.656929 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516241.66 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 51624.17 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791999.822941 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464617.49 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 46461.75 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838461.572353 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418155.74 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 41815.57 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880277.146823 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376340.17 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 37634.02 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917911.163846 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338706.15 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 33870.62 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951781.779167 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304835.54 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 30483.55 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982265.332955 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274351.98 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 27435.20 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009700.531365 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246916.79 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 24691.68 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034392.209934 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222225.11 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 22222.51 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056614.720646 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200002.60 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 20000.26 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076614.980287 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180002.34 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 18000.23 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094615.213964 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162002.10 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 16200.21 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110815.424273 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145801.89 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 14580.19 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125395.613551 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131221.70 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 13122.17 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138517.783901 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118099.53 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 11809.95 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150327.737216 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106289.58 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 10628.96 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160956.695200 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95660.62 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 9566.06 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170522.757385 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86094.56 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 8609.46 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179132.213352 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77485.10 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 7748.51 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186880.723722 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69736.59 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 6973.66 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193854.383056 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62762.93 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 6276.29 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200130.676455 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56486.64 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 5648.66 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205779.340515 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50837.98 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 5083.80 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210863.138169 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45754.18 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 4575.42 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215438.556058 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41178.76 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 4117.88 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219556.432157 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37060.88 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3706.09 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223262.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33354.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226862.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29754.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230462.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26154.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234062.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22554.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237662.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18954.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241262.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15354.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244862.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11754.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248462.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8154.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252062.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4554.80 sec) 06:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255662.520647 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954.80 sec) 06:10:00 [GPUGRID] [rr_sim] 2256617.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992412652.04G/439.78G) 06:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851858.23 06:10:00 [---] pick_jobs_to_run() start 06:10:00 [---] pick_jobs_to_run() end 06:10:00 [---] [rr_sim] end 06:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:10:00 [---] [cpu_sched_debug] preliminary job list: 06:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:10:00 [---] [cpu_sched_debug] final job list: 06:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:10:00 [---] [cpu_sched_debug] enforce_run_list: end 06:10:00 [---] choose_project(): 22200.000000 06:10:00 [iThena.Measurements] doing RPC to get work 06:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413399.09 cpu 2158172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:10:00 [---] Got 0 tasks 06:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:11:00 [---] [rr_sim] doing sim: CPU sched 06:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992385061.69G/439.78G) 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256554.58 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 225655.46 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225655.458017 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030899.12 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 203089.91 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428745.370233 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827809.21 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 182780.92 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611526.291227 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1645028.29 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 164502.83 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776029.120121 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480525.46 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 148052.55 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924081.666126 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332472.91 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 133247.29 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057328.957531 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199225.62 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 119922.56 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177251.519795 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079303.06 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 107930.31 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285181.825832 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971372.75 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 97137.28 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382319.101266 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874235.48 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 87423.55 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469742.649157 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786811.93 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 78681.19 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548423.842258 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708130.74 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 70813.07 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619236.916050 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637317.66 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 63731.77 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682968.682462 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573585.90 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 57358.59 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740327.272233 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516227.31 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 51622.73 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791950.003027 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464604.58 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 46460.46 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838410.460741 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418144.12 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 41814.41 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880224.872684 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376329.71 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 37632.97 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917857.843433 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338696.74 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 33869.67 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951727.517107 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304827.06 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 30482.71 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982210.223413 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274344.36 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 27434.44 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009644.659089 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246909.92 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 24690.99 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034335.651197 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222218.93 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 22221.89 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056557.544095 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199997.04 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 19999.70 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076557.247703 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179997.33 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 17999.73 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094556.980949 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161997.60 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 16199.76 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110756.740872 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145797.84 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 14579.78 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125336.524802 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131218.06 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 13121.81 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138458.330339 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118096.25 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 11809.62 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150267.955322 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106286.62 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 10628.66 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160896.617807 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95657.96 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 9565.80 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170462.414043 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86092.17 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 8609.22 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179071.630656 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77482.95 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 7748.29 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186819.925608 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69734.65 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 6973.47 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193793.391064 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62761.19 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 6276.12 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200069.509975 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56485.07 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 5648.51 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205718.016995 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50836.56 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 5083.66 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210801.673312 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45752.91 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 4575.29 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215376.963998 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41177.62 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 4117.76 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219494.725616 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37059.85 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3705.99 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223200.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33353.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226800.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29753.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230400.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26153.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234000.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22553.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237600.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18953.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241200.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15353.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244800.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11753.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248400.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8153.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252000.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4553.87 sec) 06:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255600.711071 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953.87 sec) 06:11:00 [GPUGRID] [rr_sim] 2256554.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992385061.69G/439.78G) 06:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851855.49 06:11:00 [---] pick_jobs_to_run() start 06:11:00 [---] pick_jobs_to_run() end 06:11:00 [---] [rr_sim] end 06:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:11:00 [---] [cpu_sched_debug] preliminary job list: 06:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:11:00 [---] [cpu_sched_debug] final job list: 06:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:11:00 [---] [cpu_sched_debug] enforce_run_list: end 06:11:00 [---] choose_project(): 22260.000000 06:11:00 [iThena.Measurements] doing RPC to get work 06:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413339.09 cpu 2158112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:11:00 [---] Got 0 tasks 06:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:12:00 [---] [rr_sim] doing sim: CPU sched 06:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992357471.33G/439.78G) 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256491.84 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 225649.18 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225649.184329 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030842.66 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 203084.27 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428733.450225 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827758.39 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 182775.84 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611509.289531 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644982.55 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 164498.26 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 776007.544907 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480484.30 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 148048.43 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924055.974745 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332435.87 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 133243.59 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057299.561600 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199192.28 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 119919.23 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177218.789769 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079273.05 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 107927.31 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285146.095121 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971345.75 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 97134.57 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382280.669938 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874211.17 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 87421.12 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469701.787273 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786790.06 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 78679.01 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548380.792875 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708111.05 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 70811.11 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619191.897916 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637299.95 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 63729.99 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682921.892453 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573569.95 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 57357.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740278.887537 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516212.96 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 51621.30 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791900.183112 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464591.66 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 46459.17 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838359.349130 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418132.49 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 41813.25 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880172.598546 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376319.24 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 37631.92 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917804.523020 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338687.32 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 33868.73 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951673.255047 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304818.59 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 30481.86 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982155.113871 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274336.73 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 27433.67 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009588.786813 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246903.06 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 24690.31 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034279.092461 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222212.75 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 22221.28 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056500.367544 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199991.48 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 19999.15 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076499.515118 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179992.33 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 17999.23 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094498.747935 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161993.10 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 16199.31 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110698.057471 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145793.79 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 14579.38 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125277.436053 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131214.41 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 13121.44 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138398.876776 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118092.97 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 11809.30 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150208.173428 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106283.67 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 10628.37 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160836.540414 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95655.30 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 9565.53 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170402.070701 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86089.77 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 8608.98 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179011.047960 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77480.80 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 7748.08 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186759.127493 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69732.72 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 6973.27 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193732.399073 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62759.44 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 6275.94 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200008.343495 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56483.50 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 5648.35 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205656.693474 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50835.15 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 5083.51 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210740.208456 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45751.63 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 4575.16 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215315.371939 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41176.47 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 4117.65 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219433.019074 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37058.82 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3705.88 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223138.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33352.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226738.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29752.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230338.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26152.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233938.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22552.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237538.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18952.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241138.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15352.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244738.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11752.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248338.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8152.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251938.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4552.94 sec) 06:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255538.901496 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952.94 sec) 06:12:00 [GPUGRID] [rr_sim] 2256491.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992357471.33G/439.78G) 06:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851852.76 06:12:00 [---] pick_jobs_to_run() start 06:12:00 [---] pick_jobs_to_run() end 06:12:00 [---] [rr_sim] end 06:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:12:00 [---] [cpu_sched_debug] preliminary job list: 06:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:12:00 [---] [cpu_sched_debug] final job list: 06:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:12:00 [---] [cpu_sched_debug] enforce_run_list: end 06:12:00 [---] choose_project(): 22320.000000 06:12:00 [iThena.Measurements] doing RPC to get work 06:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413279.09 cpu 2158052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2158052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2158052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:12:00 [---] Got 0 tasks 06:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:13:00 [---] [rr_sim] doing sim: CPU sched 06:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992329880.97G/439.78G) 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256429.11 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 225642.91 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225642.910641 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030786.20 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 203078.62 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428721.530217 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827707.58 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 182770.76 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611492.287836 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644936.82 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 164493.68 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775985.969693 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480443.14 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 148044.31 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924030.283365 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332398.82 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 133239.88 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057270.165669 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199158.94 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 119915.89 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177186.059743 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079243.05 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 107924.30 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285110.364409 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971318.74 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 97131.87 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382242.238609 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874186.87 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 87418.69 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469660.925389 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786768.18 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 78676.82 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548337.743491 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708091.36 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 70809.14 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619146.879782 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637282.23 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 63728.22 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682875.102445 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573554.00 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 57355.40 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740230.502841 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516198.60 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 51619.86 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791850.363198 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464578.74 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 46457.87 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838308.237519 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418120.87 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 41812.09 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880120.324408 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376308.78 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 37630.88 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917751.202608 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338677.90 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 33867.79 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951618.992988 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304810.11 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 30481.01 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982100.004330 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274329.10 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 27432.91 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009532.914537 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246896.19 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 24689.62 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034222.533724 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222206.57 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 22220.66 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056443.190993 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199985.92 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 19998.59 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076441.782534 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179987.32 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 17998.73 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094440.514921 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161988.59 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 16198.86 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110639.374070 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145789.73 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 14578.97 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125218.347304 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131210.76 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 13121.08 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138339.423214 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118089.68 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 11808.97 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150148.391533 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106280.71 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 10628.07 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160776.463021 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95652.64 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 9565.26 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170341.727359 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86087.38 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 8608.74 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178950.465264 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77478.64 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 7747.86 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186698.329379 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69730.78 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 6973.08 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193671.407081 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62757.70 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 6275.77 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199947.177014 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56481.93 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 5648.19 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205595.369953 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50833.74 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 5083.37 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210678.743599 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45750.36 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 4575.04 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215253.779880 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41175.33 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 4117.53 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219371.312532 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37057.79 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3705.78 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223077.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33352.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226677.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29752.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230277.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26152.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233877.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22552.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237477.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18952.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241077.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15352.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244677.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11752.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248277.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8152.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251877.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4552.01 sec) 06:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255477.091920 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952.01 sec) 06:13:00 [GPUGRID] [rr_sim] 2256429.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992329880.97G/439.78G) 06:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851850.02 06:13:00 [---] pick_jobs_to_run() start 06:13:00 [---] pick_jobs_to_run() end 06:13:00 [---] [rr_sim] end 06:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:13:00 [---] [cpu_sched_debug] preliminary job list: 06:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:13:00 [---] [cpu_sched_debug] final job list: 06:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:13:00 [---] [cpu_sched_debug] enforce_run_list: end 06:13:00 [---] choose_project(): 22380.000000 06:13:00 [iThena.Measurements] doing RPC to get work 06:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 586019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413219.09 cpu 2157992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:13:00 [---] Got 0 tasks 06:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:14:00 [---] [rr_sim] doing sim: CPU sched 06:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992302290.62G/439.78G) 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256366.37 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 225636.64 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225636.636953 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030729.73 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 203072.97 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428709.610210 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827656.76 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 182765.68 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611475.286141 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644891.08 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 164489.11 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775964.394480 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480401.98 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 148040.20 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 924004.591984 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332361.78 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 133236.18 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057240.769738 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199125.60 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 119912.56 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177153.329717 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079213.04 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 107921.30 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285074.633698 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971291.74 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 97129.17 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382203.807281 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874162.56 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 87416.26 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469620.063505 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786746.31 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 78674.63 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548294.694107 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708071.68 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 70807.17 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619101.861649 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637264.51 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 63726.45 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682828.312437 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573538.06 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 57353.81 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740182.118145 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516184.25 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 51618.43 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791800.543283 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464565.83 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 46456.58 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838257.125908 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418109.24 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 41810.92 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880068.050269 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376298.32 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 37629.83 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917697.882195 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338668.49 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 33866.85 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951564.730928 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304801.64 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 30480.16 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982044.894788 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274321.47 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 27432.15 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009477.042261 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246889.33 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 24688.93 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034165.974988 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222200.39 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 22220.04 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056386.014442 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199980.36 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 19998.04 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076384.049950 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179982.32 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 17998.23 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094382.281907 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161984.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 16198.41 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110580.690669 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145785.68 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 14578.57 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125159.258555 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131207.11 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 13120.71 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138279.969652 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118086.40 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 11808.64 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150088.609639 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106277.76 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 10627.78 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160716.385628 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95649.98 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 9565.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170281.384018 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86084.99 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 8608.50 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178889.882568 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77476.49 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 7747.65 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186637.531264 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69728.84 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 6972.88 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193610.415090 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62755.95 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 6275.60 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199886.010534 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56480.36 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 5648.04 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205534.046433 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50832.32 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 5083.23 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210617.278742 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45749.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 4574.91 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215192.187820 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41174.18 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 4117.42 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219309.605991 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37056.76 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3705.68 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223015.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33351.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226615.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29751.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230215.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26151.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233815.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22551.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237415.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18951.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241015.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15351.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244615.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11751.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248215.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8151.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251815.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4551.09 sec) 06:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255415.282344 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951.09 sec) 06:14:00 [GPUGRID] [rr_sim] 2256366.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992302290.62G/439.78G) 06:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851847.28 06:14:00 [---] pick_jobs_to_run() start 06:14:00 [---] pick_jobs_to_run() end 06:14:00 [---] [rr_sim] end 06:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:14:00 [---] [cpu_sched_debug] preliminary job list: 06:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:14:00 [---] [cpu_sched_debug] final job list: 06:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:14:00 [---] [cpu_sched_debug] enforce_run_list: end 06:14:00 [---] choose_project(): 22440.000000 06:14:00 [iThena.Measurements] doing RPC to get work 06:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413159.09 cpu 2157932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:14:00 [---] Got 0 tasks 06:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:15:00 [---] [rr_sim] doing sim: CPU sched 06:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992274700.26G/439.78G) 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256303.63 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 225630.36 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225630.363264 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030673.27 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 203067.33 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428697.690202 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827605.94 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 182760.59 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611458.284446 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644845.35 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 164484.53 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775942.819266 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480360.81 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 148036.08 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923978.900604 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332324.73 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 133232.47 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057211.373808 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199092.26 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 119909.23 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177120.599691 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079183.03 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 107918.30 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285038.902986 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971264.73 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 97126.47 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382165.375952 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874138.26 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 87413.83 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469579.201621 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786724.43 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 78672.44 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548251.644723 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708051.99 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 70805.20 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619056.843515 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637246.79 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 63724.68 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682781.522428 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573522.11 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 57352.21 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740133.733450 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516169.90 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 51616.99 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791750.723369 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464552.91 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 46455.29 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838206.014296 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418097.62 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 41809.76 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880015.776131 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376287.86 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 37628.79 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917644.561782 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338659.07 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 33865.91 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951510.468868 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304793.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 30479.32 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981989.785246 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274313.85 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 27431.38 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009421.169985 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246882.46 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 24688.25 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034109.416251 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222194.22 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 22219.42 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056328.837890 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199974.79 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 19997.48 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076326.317366 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179977.32 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 17997.73 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094324.048893 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161979.58 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 16197.96 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110522.007268 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145781.63 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 14578.16 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125100.169806 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131203.46 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 13120.35 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138220.516090 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118083.12 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 11808.31 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150028.827745 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106274.80 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 10627.48 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160656.308235 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95647.32 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 9564.73 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170221.040676 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86082.59 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 8608.26 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178829.299872 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77474.33 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 7747.43 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186576.733149 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69726.90 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 6972.69 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193549.423099 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62754.21 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 6275.42 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199824.844053 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56478.79 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 5647.88 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205472.722912 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50830.91 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 5083.09 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210555.813885 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45747.82 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 4574.78 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215130.595761 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41173.04 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 4117.30 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219247.899449 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37055.73 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3705.57 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222953.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33350.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226553.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29750.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230153.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26150.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233753.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22550.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237353.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18950.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240953.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15350.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244553.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11750.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248153.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8150.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251753.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4550.16 sec) 06:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255353.472769 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950.16 sec) 06:15:00 [GPUGRID] [rr_sim] 2256303.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992274700.26G/439.78G) 06:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851844.55 06:15:00 [---] pick_jobs_to_run() start 06:15:00 [---] pick_jobs_to_run() end 06:15:00 [---] [rr_sim] end 06:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:15:00 [---] [cpu_sched_debug] preliminary job list: 06:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:15:00 [---] [cpu_sched_debug] final job list: 06:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:15:00 [---] [cpu_sched_debug] enforce_run_list: end 06:15:00 [---] choose_project(): 22500.000000 06:15:00 [iThena.Measurements] doing RPC to get work 06:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413099.09 cpu 2157872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:15:00 [---] Got 0 tasks 06:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:16:00 [---] [rr_sim] doing sim: CPU sched 06:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992247109.90G/439.78G) 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256240.90 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 225624.09 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225624.089576 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030616.81 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 203061.68 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428685.770195 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827555.13 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 182755.51 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611441.282751 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644799.61 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 164479.96 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775921.244052 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480319.65 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 148031.97 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923953.209223 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332287.69 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 133228.77 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057181.977877 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199058.92 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 119905.89 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177087.869665 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079153.03 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 107915.30 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1285003.172275 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971237.72 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 97123.77 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382126.944623 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874113.95 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 87411.40 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469538.339737 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786702.56 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 78670.26 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548208.595340 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708032.30 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 70803.23 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1619011.825382 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637229.07 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 63722.91 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682734.732420 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573506.16 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 57350.62 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740085.348754 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516155.55 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 51615.55 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791700.903454 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464539.99 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 46454.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838154.902685 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418085.99 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 41808.60 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879963.501993 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376277.39 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 37627.74 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917591.241369 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338649.65 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 33864.97 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951456.206809 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304784.69 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 30478.47 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981934.675704 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274306.22 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 27430.62 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009365.297710 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246875.60 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 24687.56 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034052.857515 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222188.04 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 22218.80 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056271.661339 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199969.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 19996.92 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076268.584781 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179972.31 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 17997.23 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094265.815879 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161975.08 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 16197.51 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110463.323868 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145777.57 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 14577.76 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125041.081057 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131199.81 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 13119.98 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138161.062527 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118079.83 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 11807.98 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149969.045851 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106271.85 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 10627.18 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160596.230842 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95644.66 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 9564.47 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170160.697334 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86080.20 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 8608.02 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178768.717176 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77472.18 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 7747.22 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186515.935035 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69724.96 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 6972.50 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193488.431107 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62752.46 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 6275.25 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199763.677573 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56477.22 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 5647.72 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205411.399392 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50829.50 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 5082.95 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210494.349028 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45746.55 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 4574.65 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215069.003702 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41171.89 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 4117.19 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219186.192908 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37054.70 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3705.47 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222891.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33349.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226491.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29749.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230091.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26149.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233691.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22549.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237291.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18949.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240891.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15349.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244491.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11749.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248091.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8149.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251691.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4549.23 sec) 06:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255291.663193 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949.23 sec) 06:16:00 [GPUGRID] [rr_sim] 2256240.90: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992247109.90G/439.78G) 06:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851841.81 06:16:00 [---] pick_jobs_to_run() start 06:16:00 [---] pick_jobs_to_run() end 06:16:00 [---] [rr_sim] end 06:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:16:00 [---] [cpu_sched_debug] preliminary job list: 06:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:16:00 [---] [cpu_sched_debug] final job list: 06:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:16:00 [---] [cpu_sched_debug] enforce_run_list: end 06:16:00 [---] choose_project(): 22560.000000 06:16:00 [iThena.Measurements] doing RPC to get work 06:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 413039.09 cpu 2157812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:16:00 [---] Got 0 tasks 06:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:17:00 [---] [rr_sim] doing sim: CPU sched 06:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992219519.55G/439.78G) 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256178.16 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 225617.82 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225617.815888 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030560.34 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 203056.03 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428673.850187 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827504.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 182750.43 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611424.281056 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644753.88 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 164475.39 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775899.668838 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480278.49 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 148027.85 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923927.517842 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332250.64 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 133225.06 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057152.581946 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1199025.58 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 119902.56 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177055.139639 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079123.02 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 107912.30 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284967.441563 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971210.72 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 97121.07 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382088.513295 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874089.65 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 87408.96 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469497.477853 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786680.68 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 78668.07 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548165.545956 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708012.61 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 70801.26 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618966.807248 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637211.35 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 63721.14 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682687.942411 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573490.22 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 57349.02 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740036.964058 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516141.19 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 51614.12 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791651.083540 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464527.08 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 46452.71 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838103.791074 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418074.37 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 41807.44 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879911.227854 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376266.93 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 37626.69 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917537.920957 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338640.24 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 33864.02 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951401.944749 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304776.21 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 30477.62 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981879.566162 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274298.59 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 27429.86 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009309.425434 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246868.73 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 24686.87 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033996.298778 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222181.86 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 22218.19 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056214.484788 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199963.67 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 19996.37 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076210.852197 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179967.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 17996.73 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094207.582865 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161970.58 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 16197.06 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110404.640467 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145773.52 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 14577.35 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124981.992308 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131196.17 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 13119.62 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138101.608965 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118076.55 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 11807.65 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149909.263956 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106268.89 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 10626.89 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160536.153449 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95642.01 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 9564.20 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170100.353992 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86077.80 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 8607.78 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178708.134480 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77470.02 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 7747.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186455.136920 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69723.02 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 6972.30 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193427.439116 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62750.72 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 6275.07 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199702.511092 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56475.65 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 5647.56 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205350.075871 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50828.08 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 5082.81 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210432.884172 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45745.27 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 4574.53 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215007.411642 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41170.75 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 4117.07 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219124.486366 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37053.67 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3705.37 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222829.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33348.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226429.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29748.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230029.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26148.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233629.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22548.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237229.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18948.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240829.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15348.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244429.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11748.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248029.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8148.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251629.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4548.31 sec) 06:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255229.853617 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948.31 sec) 06:17:00 [GPUGRID] [rr_sim] 2256178.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992219519.55G/439.78G) 06:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851839.07 06:17:00 [---] pick_jobs_to_run() start 06:17:00 [---] pick_jobs_to_run() end 06:17:00 [---] [rr_sim] end 06:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:17:00 [---] [cpu_sched_debug] preliminary job list: 06:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:17:00 [---] [cpu_sched_debug] final job list: 06:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:17:00 [---] [cpu_sched_debug] enforce_run_list: end 06:17:00 [---] choose_project(): 22620.000000 06:17:00 [iThena.Measurements] doing RPC to get work 06:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412979.09 cpu 2157752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:17:00 [---] Got 0 tasks 06:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:18:00 [---] [rr_sim] doing sim: CPU sched 06:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992191929.19G/439.78G) 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256115.42 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 225611.54 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225611.542200 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030503.88 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 203050.39 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428661.930179 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827453.49 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 182745.35 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611407.279361 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644708.14 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 164470.81 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775878.093625 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480237.33 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 148023.73 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923901.826462 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332213.60 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 133221.36 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057123.186015 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198992.24 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 119899.22 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1177022.409613 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079093.01 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 107909.30 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284931.710852 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971183.71 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 97118.37 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382050.081966 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874065.34 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 87406.53 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469456.615969 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786658.81 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 78665.88 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548122.496572 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707992.93 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 70799.29 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618921.789114 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637193.63 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 63719.36 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682641.152403 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573474.27 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 57347.43 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739988.579362 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516126.84 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 51612.68 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791601.263625 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464514.16 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 46451.42 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838052.679463 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418062.74 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 41806.27 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879858.953716 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376256.47 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 37625.65 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917484.600544 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338630.82 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 33863.08 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951347.682689 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304767.74 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 30476.77 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981824.456620 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274290.97 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 27429.10 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009253.553158 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246861.87 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 24686.19 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033939.740042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222175.68 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 22217.57 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056157.308237 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199958.11 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 19995.81 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076153.119613 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179962.30 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 17996.23 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094149.349851 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161966.07 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 16196.61 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110345.957066 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145769.46 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 14576.95 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124922.903559 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131192.52 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 13119.25 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138042.155403 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118073.27 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 11807.33 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149849.482062 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106265.94 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 10626.59 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160476.076056 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95639.35 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 9563.93 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170040.010650 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86075.41 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 8607.54 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178647.551784 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77467.87 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 7746.79 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186394.338806 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69721.08 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 6972.11 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193366.447125 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62748.97 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 6274.90 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199641.344612 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56474.08 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 5647.41 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205288.752350 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50826.67 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 5082.67 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210371.419315 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45744.00 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 4574.40 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214945.819583 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41169.60 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 4116.96 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219062.779824 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37052.64 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3705.26 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222768.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33347.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226368.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29747.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229968.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26147.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233568.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22547.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237168.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18947.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240768.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15347.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244368.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11747.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247968.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8147.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251568.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4547.38 sec) 06:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255168.044042 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947.38 sec) 06:18:00 [GPUGRID] [rr_sim] 2256115.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992191929.19G/439.78G) 06:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851836.33 06:18:00 [---] pick_jobs_to_run() start 06:18:00 [---] pick_jobs_to_run() end 06:18:00 [---] [rr_sim] end 06:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:18:00 [---] [cpu_sched_debug] preliminary job list: 06:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:18:00 [---] [cpu_sched_debug] final job list: 06:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:18:00 [---] [cpu_sched_debug] enforce_run_list: end 06:18:00 [---] choose_project(): 22680.000000 06:18:00 [iThena.Measurements] doing RPC to get work 06:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412919.09 cpu 2157692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:18:00 [---] Got 0 tasks 06:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:19:00 [---] [rr_sim] doing sim: CPU sched 06:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992164338.84G/439.78G) 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2256052.69 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 225605.27 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225605.268511 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030447.42 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 203044.74 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428650.010172 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827402.67 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 182740.27 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611390.277666 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644662.41 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 164466.24 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775856.518411 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480196.17 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 148019.62 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923876.135081 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332176.55 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 133217.66 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057093.790085 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198958.90 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 119895.89 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176989.679587 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079063.01 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 107906.30 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284895.980140 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971156.70 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 97115.67 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1382011.650638 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874041.03 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 87404.10 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469415.754085 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786636.93 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 78663.69 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548079.447188 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707973.24 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 70797.32 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618876.770981 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637175.91 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 63717.59 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682594.362394 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573458.32 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 57345.83 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739940.194666 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516112.49 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 51611.25 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791551.443711 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464501.24 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 46450.12 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838001.567851 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418051.12 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 41805.11 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879806.679578 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376246.01 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 37624.60 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917431.280131 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338621.40 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 33862.14 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951293.420630 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304759.26 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 30475.93 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981769.347078 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274283.34 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 27428.33 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009197.680882 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246855.00 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 24685.50 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033883.181305 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222169.50 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 22216.95 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056100.131686 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199952.55 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 19995.26 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076095.387029 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179957.30 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 17995.73 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094091.116837 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161961.57 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 16196.16 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110287.273665 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145765.41 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 14576.54 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124863.814810 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131188.87 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 13118.89 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137982.701840 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118069.98 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 11807.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149789.700168 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106262.98 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 10626.30 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160415.998662 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95636.69 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 9563.67 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169979.667308 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86073.02 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 8607.30 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178586.969088 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77465.72 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 7746.57 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186333.540691 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69719.14 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 6971.91 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193305.455133 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62747.23 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 6274.72 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199580.178131 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56472.51 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 5647.25 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205227.428830 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50825.26 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 5082.53 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210309.954458 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45742.73 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 4574.27 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214884.227524 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41168.46 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 4116.85 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219001.073283 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37051.61 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3705.16 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222706.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33346.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226306.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29746.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229906.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26146.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233506.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22546.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237106.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18946.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240706.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15346.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244306.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11746.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247906.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8146.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251506.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4546.45 sec) 06:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255106.234466 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946.45 sec) 06:19:00 [GPUGRID] [rr_sim] 2256052.69: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992164338.84G/439.78G) 06:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851833.60 06:19:00 [---] pick_jobs_to_run() start 06:19:00 [---] pick_jobs_to_run() end 06:19:00 [---] [rr_sim] end 06:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:19:00 [---] [cpu_sched_debug] preliminary job list: 06:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:19:00 [---] [cpu_sched_debug] final job list: 06:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:19:00 [---] [cpu_sched_debug] enforce_run_list: end 06:19:00 [---] choose_project(): 22740.000000 06:19:00 [iThena.Measurements] doing RPC to get work 06:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412859.09 cpu 2157632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:19:00 [---] Got 0 tasks 06:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:20:00 [---] [rr_sim] doing sim: CPU sched 06:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992136748.48G/439.78G) 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255989.95 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 225598.99 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225598.994823 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030390.95 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 203039.10 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428638.090164 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827351.86 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 182735.19 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611373.275971 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644616.67 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 164461.67 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775834.943197 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480155.01 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 148015.50 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923850.443701 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332139.50 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 133213.95 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057064.394154 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198925.55 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 119892.56 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176956.949562 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079033.00 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 107903.30 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284860.249429 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971129.70 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 97112.97 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381973.219309 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874016.73 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 87401.67 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469374.892201 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786615.06 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 78661.51 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1548036.397804 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707953.55 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 70795.36 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618831.752847 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637158.20 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 63715.82 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682547.572386 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573442.38 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 57344.24 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739891.809970 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516098.14 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 51609.81 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791501.623797 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464488.32 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 46448.83 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837950.456240 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418039.49 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 41803.95 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879754.405439 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376235.54 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 37623.55 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917377.959719 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338611.99 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 33861.20 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951239.158570 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304750.79 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 30475.08 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981714.237536 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274275.71 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 27427.57 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009141.808606 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246848.14 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 24684.81 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033826.622568 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222163.33 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 22216.33 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056042.955135 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199946.99 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 19994.70 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076037.654445 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179952.29 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 17995.23 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094032.883823 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161957.06 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 16195.71 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110228.590264 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145761.36 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 14576.14 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124804.726061 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131185.22 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 13118.52 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137923.248278 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118066.70 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 11806.67 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149729.918274 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106260.03 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 10626.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160355.921269 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95634.03 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 9563.40 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169919.323966 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86070.62 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 8607.06 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178526.386392 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77463.56 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 7746.36 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186272.742576 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69717.21 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 6971.72 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193244.463142 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62745.49 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 6274.55 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199519.011651 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56470.94 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 5647.09 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205166.105309 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50823.84 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 5082.38 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210248.489601 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45741.46 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 4574.15 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214822.635464 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41167.31 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 4116.73 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218939.366741 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37050.58 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3705.06 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222644.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33345.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226244.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29745.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229844.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26145.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233444.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22545.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237044.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18945.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240644.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15345.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244244.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11745.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247844.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8145.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251444.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4545.52 sec) 06:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2255044.424890 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945.52 sec) 06:20:00 [GPUGRID] [rr_sim] 2255989.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992136748.48G/439.78G) 06:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851830.86 06:20:00 [---] pick_jobs_to_run() start 06:20:00 [---] pick_jobs_to_run() end 06:20:00 [---] [rr_sim] end 06:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:20:00 [---] [cpu_sched_debug] preliminary job list: 06:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:20:00 [---] [cpu_sched_debug] final job list: 06:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:20:00 [---] [cpu_sched_debug] enforce_run_list: end 06:20:00 [---] choose_project(): 22800.000000 06:20:00 [iThena.Measurements] doing RPC to get work 06:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412799.09 cpu 2157572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:20:00 [---] Got 0 tasks 06:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:21:00 [---] [rr_sim] doing sim: CPU sched 06:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992109158.12G/439.78G) 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255927.21 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 225592.72 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225592.721135 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030334.49 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 203033.45 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428626.170157 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827301.04 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 182730.10 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611356.274276 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644570.94 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 164457.09 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775813.367983 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480113.84 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 148011.38 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923824.752320 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332102.46 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 133210.25 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057034.998223 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198892.21 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 119889.22 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176924.219536 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1079002.99 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 107900.30 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284824.518717 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971102.69 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 97110.27 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381934.787980 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873992.42 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 87399.24 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469334.030317 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786593.18 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 78659.32 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547993.348421 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707933.86 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 70793.39 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618786.734714 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637140.48 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 63714.05 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682500.782377 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573426.43 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 57342.64 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739843.425275 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516083.79 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 51608.38 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791451.803882 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464475.41 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 46447.54 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837899.344629 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418027.87 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 41802.79 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879702.131301 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376225.08 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 37622.51 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917324.639306 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338602.57 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 33860.26 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951184.896510 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304742.31 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 30474.23 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981659.127994 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274268.08 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 27426.81 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009085.936330 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246841.28 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 24684.13 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033770.063832 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222157.15 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 22215.71 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055985.778584 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199941.43 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 19994.14 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075979.921860 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179947.29 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 17994.73 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093974.650809 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161952.56 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 16195.26 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110169.906863 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145757.30 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 14575.73 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124745.637312 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131181.57 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 13118.16 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137863.794716 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118063.42 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 11806.34 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149670.136379 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106257.07 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 10625.71 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160295.843876 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95631.37 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 9563.14 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169858.980624 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86068.23 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 8606.82 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178465.803696 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77461.41 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 7746.14 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186211.944462 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69715.27 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 6971.53 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193183.471151 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62743.74 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 6274.37 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199457.845170 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56469.37 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 5646.94 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205104.781788 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50822.43 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 5082.24 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210187.024745 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45740.19 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 4574.02 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214761.043405 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41166.17 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 4116.62 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218877.660200 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37049.55 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3704.96 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222582.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33344.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226182.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29744.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229782.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26144.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233382.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22544.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236982.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18944.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240582.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15344.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244182.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11744.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247782.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8144.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251382.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4544.60 sec) 06:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254982.615315 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944.60 sec) 06:21:00 [GPUGRID] [rr_sim] 2255927.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992109158.12G/439.78G) 06:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851828.12 06:21:00 [---] pick_jobs_to_run() start 06:21:00 [---] pick_jobs_to_run() end 06:21:00 [---] [rr_sim] end 06:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:21:00 [---] [cpu_sched_debug] preliminary job list: 06:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:21:00 [---] [cpu_sched_debug] final job list: 06:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:21:00 [---] [cpu_sched_debug] enforce_run_list: end 06:21:00 [---] choose_project(): 22860.000000 06:21:00 [iThena.Measurements] doing RPC to get work 06:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412739.09 cpu 2157512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:21:00 [---] Got 0 tasks 06:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:22:00 [---] [rr_sim] doing sim: CPU sched 06:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992081567.77G/439.78G) 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255864.47 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 225586.45 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225586.447447 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030278.03 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 203027.80 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428614.250149 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827250.22 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 182725.02 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611339.272581 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644525.20 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 164452.52 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775791.792770 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480072.68 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 148007.27 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923799.060939 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332065.41 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 133206.54 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1057005.602292 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198858.87 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 119885.89 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176891.489510 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078972.98 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 107897.30 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284788.788006 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971075.69 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 97107.57 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381896.356652 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873968.12 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 87396.81 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469293.168433 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786571.31 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 78657.13 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547950.299037 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707914.18 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 70791.42 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618741.716580 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637122.76 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 63712.28 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682453.992369 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573410.48 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 57341.05 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739795.040579 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516069.43 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 51606.94 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791401.983968 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464462.49 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 46446.25 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837848.233018 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418016.24 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 41801.62 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879649.857163 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376214.62 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 37621.46 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917271.318893 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338593.16 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 33859.32 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951130.634451 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304733.84 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 30473.38 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981604.018452 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274260.46 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 27426.05 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009030.064054 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246834.41 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 24683.44 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033713.505095 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222150.97 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 22215.10 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055928.602033 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199935.87 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 19993.59 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075922.189276 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179942.29 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 17994.23 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093916.417795 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161948.06 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 16194.81 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110111.223463 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145753.25 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 14575.33 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124686.548563 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131177.93 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 13117.79 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137804.341154 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118060.13 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 11806.01 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149610.354485 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106254.12 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 10625.41 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160235.766483 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95628.71 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 9562.87 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169798.637282 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86065.84 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 8606.58 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178405.221000 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77459.25 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 7745.93 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186151.146347 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69713.33 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 6971.33 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193122.479159 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62742.00 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 6274.20 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199396.678690 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56467.80 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 5646.78 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205043.458268 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50821.02 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 5082.10 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210125.559888 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45738.91 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 4573.89 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214699.451346 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41165.02 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 4116.50 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218815.953658 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37048.52 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3704.85 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222520.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33343.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226120.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29743.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229720.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26143.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233320.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22543.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236920.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18943.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240520.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15343.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244120.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11743.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247720.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8143.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251320.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4543.67 sec) 06:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254920.805739 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943.67 sec) 06:22:00 [GPUGRID] [rr_sim] 2255864.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992081567.77G/439.78G) 06:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851825.39 06:22:00 [---] pick_jobs_to_run() start 06:22:00 [---] pick_jobs_to_run() end 06:22:00 [---] [rr_sim] end 06:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:22:00 [---] [cpu_sched_debug] preliminary job list: 06:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:22:00 [---] [cpu_sched_debug] final job list: 06:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:22:00 [---] [cpu_sched_debug] enforce_run_list: end 06:22:00 [---] choose_project(): 22920.000000 06:22:00 [iThena.Measurements] doing RPC to get work 06:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412679.09 cpu 2157452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:22:00 [---] Got 0 tasks 06:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:23:00 [---] [rr_sim] doing sim: CPU sched 06:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992053977.41G/439.78G) 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255801.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 225580.17 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225580.173759 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030221.56 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 203022.16 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428602.330141 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827199.41 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 182719.94 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611322.270886 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644479.47 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 164447.95 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775770.217556 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1480031.52 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 148003.15 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923773.369559 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1332028.37 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 133202.84 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056976.206361 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198825.53 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 119882.55 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176858.759484 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078942.98 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 107894.30 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284753.057294 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971048.68 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 97104.87 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381857.925323 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873943.81 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 87394.38 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469252.306549 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786549.43 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 78654.94 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547907.249653 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707894.49 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 70789.45 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618696.698446 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637105.04 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 63710.50 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682407.202360 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573394.54 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 57339.45 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739746.655883 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516055.08 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 51605.51 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791352.164053 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464449.57 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 46444.96 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837797.121406 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418004.62 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 41800.46 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879597.583024 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376204.15 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 37620.42 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917217.998480 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338583.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 33858.37 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951076.372391 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304725.37 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 30472.54 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981548.908910 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274252.83 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 27425.28 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008974.191778 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246827.55 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 24682.75 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033656.946359 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222144.79 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 22214.48 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055871.425481 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199930.31 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 19993.03 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075864.456692 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179937.28 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 17993.73 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093858.184781 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161943.55 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 16194.36 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110052.540062 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145749.20 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 14574.92 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124627.459814 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131174.28 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 13117.43 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137744.887591 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118056.85 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 11805.68 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149550.572591 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106251.16 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 10625.12 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160175.689090 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95626.05 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 9562.60 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169738.293940 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86063.44 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 8606.34 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178344.638304 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77457.10 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 7745.71 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186090.348232 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69711.39 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 6971.14 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193061.487168 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62740.25 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 6274.03 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199335.512210 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56466.23 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 5646.62 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204982.134747 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50819.60 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 5081.96 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210064.095031 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45737.64 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 4573.76 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214637.859287 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41163.88 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 4116.39 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218754.247116 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37047.49 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3704.75 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222458.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33342.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226058.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29742.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229658.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26142.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233258.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22542.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236858.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18942.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240458.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15342.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244058.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11742.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247658.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8142.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251258.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4542.74 sec) 06:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254858.996163 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 942.74 sec) 06:23:00 [GPUGRID] [rr_sim] 2255801.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992053977.41G/439.78G) 06:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851822.65 06:23:00 [---] pick_jobs_to_run() start 06:23:00 [---] pick_jobs_to_run() end 06:23:00 [---] [rr_sim] end 06:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:23:00 [---] [cpu_sched_debug] preliminary job list: 06:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:23:00 [---] [cpu_sched_debug] final job list: 06:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:23:00 [---] [cpu_sched_debug] enforce_run_list: end 06:23:00 [---] choose_project(): 22980.000000 06:23:00 [iThena.Measurements] doing RPC to get work 06:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412619.09 cpu 2157392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:23:00 [---] Got 0 tasks 06:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:24:00 [---] [rr_sim] doing sim: CPU sched 06:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (992026387.05G/439.78G) 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255739.00 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 225573.90 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225573.900070 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030165.10 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 203016.51 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428590.410134 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827148.59 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 182714.86 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611305.269191 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644433.73 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 164443.37 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775748.642342 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479990.36 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 147999.04 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923747.678178 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331991.32 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 133199.13 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056946.810431 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198792.19 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 119879.22 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176826.029458 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078912.97 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 107891.30 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284717.326583 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 971021.67 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 97102.17 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381819.493995 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873919.51 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 87391.95 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469211.444666 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786527.56 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 78652.76 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547864.200269 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707874.80 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 70787.48 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618651.680313 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637087.32 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 63708.73 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682360.412352 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573378.59 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 57337.86 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739698.271187 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516040.73 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 51604.07 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791302.344139 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464436.66 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 46443.67 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837746.009795 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417992.99 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 41799.30 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879545.308886 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376193.69 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 37619.37 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917164.678068 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338574.32 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 33857.43 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951022.110331 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304716.89 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 30471.69 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981493.799369 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274245.20 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 27424.52 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008918.319502 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246820.68 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 24682.07 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033600.387622 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222138.61 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 22213.86 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055814.248930 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199924.75 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 19992.48 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075806.724108 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179932.28 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 17993.23 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093799.951767 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161939.05 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 16193.90 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109993.856661 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145745.14 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 14574.51 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124568.371065 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131170.63 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 13117.06 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137685.434029 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118053.57 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 11805.36 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149490.790696 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106248.21 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 10624.82 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160115.611697 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95623.39 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 9562.34 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169677.950598 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86061.05 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 8606.11 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178284.055608 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77454.95 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 7745.49 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186029.550118 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69709.45 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 6970.95 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193000.495176 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62738.51 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 6273.85 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199274.345729 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56464.65 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 5646.47 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204920.811227 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50818.19 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 5081.82 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210002.630174 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45736.37 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 4573.64 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214576.267227 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41162.73 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 4116.27 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218692.540575 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37046.46 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3704.65 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222397.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33341.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225997.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29741.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229597.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26141.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233197.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22541.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236797.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18941.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240397.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15341.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243997.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11741.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247597.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8141.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251197.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4541.81 sec) 06:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254797.186588 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 941.81 sec) 06:24:00 [GPUGRID] [rr_sim] 2255739.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (992026387.05G/439.78G) 06:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851819.91 06:24:00 [---] pick_jobs_to_run() start 06:24:00 [---] pick_jobs_to_run() end 06:24:00 [---] [rr_sim] end 06:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:24:00 [---] [cpu_sched_debug] preliminary job list: 06:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:24:00 [---] [cpu_sched_debug] final job list: 06:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:24:00 [---] [cpu_sched_debug] enforce_run_list: end 06:24:00 [---] choose_project(): 23040.000000 06:24:00 [iThena.Measurements] doing RPC to get work 06:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412559.09 cpu 2157332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:24:00 [---] Got 0 tasks 06:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:25:00 [---] [rr_sim] doing sim: CPU sched 06:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991998796.70G/439.78G) 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255676.26 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 225567.63 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225567.626382 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030108.64 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 203010.86 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428578.490126 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827097.77 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 182709.78 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611288.267496 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644388.00 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 164438.80 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775727.067128 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479949.20 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 147994.92 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923721.986797 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331954.28 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 133195.43 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056917.414500 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198758.85 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 119875.88 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176793.299432 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078882.96 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 107888.30 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284681.595871 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970994.67 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 97099.47 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381781.062666 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873895.20 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 87389.52 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469170.582782 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786505.68 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 78650.57 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547821.150886 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707855.11 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 70785.51 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618606.662179 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637069.60 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 63706.96 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682313.622343 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573362.64 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 57336.26 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739649.886491 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516026.38 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 51602.64 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791252.524224 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464423.74 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 46442.37 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837694.898184 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417981.37 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 41798.14 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879493.034748 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376183.23 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 37618.32 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917111.357655 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338564.91 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 33856.49 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950967.848272 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304708.42 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 30470.84 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981438.689827 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274237.57 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 27423.76 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008862.447226 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246813.82 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 24681.38 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033543.828886 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222132.43 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 22213.24 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055757.072379 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199919.19 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 19991.92 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075748.991523 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179927.27 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 17992.73 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093741.718753 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161934.55 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 16193.45 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109935.173260 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145741.09 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 14574.11 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124509.282316 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131166.98 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 13116.70 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137625.980467 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118050.28 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 11805.03 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149431.008802 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106245.26 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 10624.53 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160055.534304 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95620.73 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 9562.07 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169617.607256 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86058.66 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 8605.87 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178223.472912 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77452.79 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 7745.28 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185968.752003 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69707.51 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 6970.75 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192939.503185 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62736.76 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 6273.68 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199213.179249 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56463.08 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 5646.31 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204859.487706 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50816.78 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 5081.68 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209941.165318 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45735.10 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 4573.51 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214514.675168 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41161.59 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 4116.16 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218630.834033 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37045.43 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3704.54 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222335.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33340.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225935.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29740.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229535.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26140.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233135.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22540.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236735.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18940.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240335.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15340.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243935.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11740.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247535.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8140.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251135.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4540.89 sec) 06:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254735.377012 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 940.89 sec) 06:25:00 [GPUGRID] [rr_sim] 2255676.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991998796.70G/439.78G) 06:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851817.18 06:25:00 [---] pick_jobs_to_run() start 06:25:00 [---] pick_jobs_to_run() end 06:25:00 [---] [rr_sim] end 06:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:25:00 [---] [cpu_sched_debug] preliminary job list: 06:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:25:00 [---] [cpu_sched_debug] final job list: 06:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:25:00 [---] [cpu_sched_debug] enforce_run_list: end 06:25:00 [---] choose_project(): 23100.000000 06:25:00 [iThena.Measurements] doing RPC to get work 06:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412499.09 cpu 2157272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:25:00 [---] Got 0 tasks 06:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:26:00 [---] [rr_sim] doing sim: CPU sched 06:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991971206.34G/439.78G) 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255613.53 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 225561.35 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225561.352694 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2030052.17 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 203005.22 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428566.570118 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1827046.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 182704.70 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611271.265801 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644342.26 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 164434.23 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775705.491914 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479908.04 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 147990.80 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923696.295417 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331917.23 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 133191.72 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056888.018569 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198725.51 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 119872.55 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176760.569406 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078852.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 107885.30 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284645.865159 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970967.66 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 97096.77 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381742.631337 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873870.90 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 87387.09 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469129.720898 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786483.81 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 78648.38 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547778.101502 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707835.43 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 70783.54 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618561.644046 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637051.88 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 63705.19 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682266.832335 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573346.69 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 57334.67 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739601.501795 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516012.03 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 51601.20 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791202.704310 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464410.82 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 46441.08 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837643.786573 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417969.74 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 41796.97 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879440.760609 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376172.77 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 37617.28 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917058.037242 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338555.49 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 33855.55 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950913.586212 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304699.94 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 30469.99 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981383.580285 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274229.95 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 27422.99 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008806.574950 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246806.95 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 24680.70 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033487.270149 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222126.26 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 22212.63 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055699.895828 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199913.63 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 19991.36 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075691.258939 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179922.27 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 17992.23 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093683.485739 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161930.04 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 16193.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109876.489859 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145737.04 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 14573.70 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124450.193567 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131163.33 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 13116.33 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137566.526904 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118047.00 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 11804.70 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149371.226908 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106242.30 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 10624.23 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159995.456911 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95618.07 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 9561.81 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169557.263914 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86056.26 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 8605.63 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178162.890216 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77450.64 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 7745.06 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185907.953889 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69705.57 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 6970.56 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192878.511194 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62735.02 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 6273.50 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199152.012768 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56461.51 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 5646.15 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204798.164185 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50815.36 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 5081.54 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209879.700461 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45733.83 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 4573.38 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214453.083109 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41160.44 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 4116.04 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218569.127492 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37044.40 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3704.44 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222273.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33339.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225873.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29739.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229473.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26139.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233073.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22539.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236673.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18939.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240273.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15339.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243873.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11739.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247473.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8139.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251073.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4539.96 sec) 06:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254673.567436 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 939.96 sec) 06:26:00 [GPUGRID] [rr_sim] 2255613.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991971206.34G/439.78G) 06:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851814.44 06:26:00 [---] pick_jobs_to_run() start 06:26:00 [---] pick_jobs_to_run() end 06:26:00 [---] [rr_sim] end 06:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:26:00 [---] [cpu_sched_debug] preliminary job list: 06:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:26:00 [---] [cpu_sched_debug] final job list: 06:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:26:00 [---] [cpu_sched_debug] enforce_run_list: end 06:26:00 [---] choose_project(): 23160.000000 06:26:00 [iThena.Measurements] doing RPC to get work 06:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412439.09 cpu 2157212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:26:00 [---] Got 0 tasks 06:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:27:00 [---] [rr_sim] doing sim: CPU sched 06:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991943615.99G/439.78G) 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255550.79 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 225555.08 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225555.079006 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029995.71 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 202999.57 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428554.650111 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826996.14 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 182699.61 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611254.264105 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644296.53 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 164429.65 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775683.916701 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479866.87 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 147986.69 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923670.604036 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331880.19 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 133188.02 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056858.622638 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198692.17 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 119869.22 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176727.839380 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078822.95 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 107882.30 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284610.134448 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970940.66 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 97094.07 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381704.200009 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873846.59 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 87384.66 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469088.859014 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786461.93 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 78646.19 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547735.052118 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707815.74 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 70781.57 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618516.625912 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637034.16 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 63703.42 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682220.042326 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573330.75 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 57333.07 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739553.117099 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515997.67 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 51599.77 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791152.884395 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464397.91 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 46439.79 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837592.674961 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417958.12 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 41795.81 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879388.486471 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376162.30 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 37616.23 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917004.716830 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338546.07 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 33854.61 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950859.324152 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304691.47 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 30469.15 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981328.470743 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274222.32 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 27422.23 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008750.702674 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246800.09 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 24680.01 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033430.711413 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222120.08 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 22212.01 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055642.719277 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199908.07 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 19990.81 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075633.526355 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179917.26 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 17991.73 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093625.252725 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161925.54 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 16192.55 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109817.806458 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145732.98 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 14573.30 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124391.104818 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131159.69 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 13115.97 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137507.073342 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118043.72 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 11804.37 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149311.445014 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106239.35 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 10623.93 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159935.379518 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95615.41 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 9561.54 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169496.920572 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86053.87 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 8605.39 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178102.307520 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77448.48 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 7744.85 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185847.155774 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69703.63 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 6970.36 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192817.519202 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62733.27 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 6273.33 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199090.846288 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56459.94 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 5645.99 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204736.840665 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50813.95 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 5081.39 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209818.235604 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45732.55 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 4573.26 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214391.491049 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41159.30 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 4115.93 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218507.420950 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37043.37 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3704.34 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222211.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33339.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225811.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29739.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229411.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26139.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233011.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22539.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236611.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18939.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240211.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15339.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243811.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11739.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247411.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8139.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251011.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4539.03 sec) 06:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254611.757861 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 939.03 sec) 06:27:00 [GPUGRID] [rr_sim] 2255550.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991943615.99G/439.78G) 06:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851811.70 06:27:00 [---] pick_jobs_to_run() start 06:27:00 [---] pick_jobs_to_run() end 06:27:00 [---] [rr_sim] end 06:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:27:00 [---] [cpu_sched_debug] preliminary job list: 06:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:27:00 [---] [cpu_sched_debug] final job list: 06:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:27:00 [---] [cpu_sched_debug] enforce_run_list: end 06:27:00 [---] choose_project(): 23220.000000 06:27:00 [iThena.Measurements] doing RPC to get work 06:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412379.09 cpu 2157152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:27:00 [---] Got 0 tasks 06:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:28:00 [---] [rr_sim] doing sim: CPU sched 06:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991916025.63G/439.78G) 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255488.05 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 225548.81 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225548.805317 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029939.25 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 202993.92 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428542.730103 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826945.32 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 182694.53 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611237.262410 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644250.79 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 164425.08 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775662.341487 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479825.71 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 147982.57 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923644.912656 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331843.14 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 133184.31 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056829.226708 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198658.83 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 119865.88 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176695.109354 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078792.94 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 107879.29 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284574.403736 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970913.65 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 97091.36 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381665.768680 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873822.28 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 87382.23 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469047.997130 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786440.06 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 78644.01 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547692.002734 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707796.05 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 70779.61 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618471.607778 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637016.45 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 63701.64 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682173.252318 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573314.80 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 57331.48 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739504.732404 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515983.32 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 51598.33 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791103.064481 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464384.99 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 46438.50 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837541.563350 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417946.49 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 41794.65 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879336.212333 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376151.84 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 37615.18 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916951.396417 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338536.66 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 33853.67 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950805.062093 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304682.99 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 30468.30 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981273.361201 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274214.69 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 27421.47 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008694.830398 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246793.22 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 24679.32 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033374.152676 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222113.90 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 22211.39 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055585.542726 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199902.51 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 19990.25 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075575.793771 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179912.26 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 17991.23 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093567.019711 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161921.03 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 16192.10 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109759.123058 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145728.93 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 14572.89 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124332.016069 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131156.04 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 13115.60 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137447.619780 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118040.43 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 11804.04 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149251.663119 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106236.39 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 10623.64 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159875.302125 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95612.75 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 9561.28 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169436.577230 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86051.48 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 8605.15 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178041.724824 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77446.33 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 7744.63 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185786.357659 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69701.70 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 6970.17 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192756.527211 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62731.53 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 6273.15 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199029.679807 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56458.37 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 5645.84 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204675.517144 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50812.54 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 5081.25 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209756.770747 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45731.28 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 4573.13 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214329.898990 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41158.15 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 4115.82 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218445.714408 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37042.34 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3704.23 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222149.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33338.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225749.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29738.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229349.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26138.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232949.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22538.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236549.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18938.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240149.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15338.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243749.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11738.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247349.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8138.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250949.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4538.10 sec) 06:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254549.948285 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 938.10 sec) 06:28:00 [GPUGRID] [rr_sim] 2255488.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991916025.63G/439.78G) 06:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851808.97 06:28:00 [---] pick_jobs_to_run() start 06:28:00 [---] pick_jobs_to_run() end 06:28:00 [---] [rr_sim] end 06:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:28:00 [---] [cpu_sched_debug] preliminary job list: 06:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:28:00 [---] [cpu_sched_debug] final job list: 06:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:28:00 [---] [cpu_sched_debug] enforce_run_list: end 06:28:00 [---] choose_project(): 23280.000000 06:28:00 [iThena.Measurements] doing RPC to get work 06:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412319.09 cpu 2157092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:28:00 [---] Got 0 tasks 06:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:29:00 [---] [rr_sim] doing sim: CPU sched 06:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991888435.27G/439.78G) 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255425.32 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 225542.53 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225542.531629 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029882.78 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 202988.28 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428530.810096 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826894.51 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 182689.45 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611220.260715 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644205.06 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 164420.51 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775640.766273 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479784.55 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 147978.46 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923619.221275 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331806.10 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 133180.61 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056799.830777 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198625.49 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 119862.55 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176662.379328 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078762.94 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 107876.29 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284538.673025 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970886.64 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 97088.66 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381627.337352 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873797.98 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 87379.80 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1469007.135246 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786418.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 78641.82 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547648.953350 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707776.36 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 70777.64 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618426.589645 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636998.73 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 63699.87 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682126.462309 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573298.85 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 57329.89 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739456.347708 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515968.97 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 51596.90 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791053.244566 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464372.07 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 46437.21 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837490.451739 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417934.86 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 41793.49 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879283.938194 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376141.38 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 37614.14 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916898.076004 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338527.24 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 33852.72 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950750.800033 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304674.52 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 30467.45 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981218.251659 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274207.06 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 27420.71 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008638.958122 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246786.36 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 24678.64 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033317.593939 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222107.72 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 22210.77 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055528.366175 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199896.95 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 19989.70 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075518.061187 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179907.26 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 17990.73 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093508.786697 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161916.53 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 16191.65 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109700.439657 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145724.88 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 14572.49 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124272.927320 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131152.39 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 13115.24 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137388.166218 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118037.15 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 11803.72 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149191.881225 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106233.44 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 10623.34 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159815.224732 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95610.09 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 9561.01 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169376.233888 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86049.08 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 8604.91 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177981.142128 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77444.17 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 7744.42 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185725.559545 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69699.76 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 6969.98 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192695.535220 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62729.78 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 6272.98 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198968.513327 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56456.80 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 5645.68 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204614.193624 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50811.12 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 5081.11 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209695.305890 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45730.01 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 4573.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214268.306931 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41157.01 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 4115.70 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218384.007867 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37041.31 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3704.13 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222088.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33337.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225688.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29737.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229288.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26137.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232888.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22537.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236488.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18937.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240088.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15337.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243688.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11737.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247288.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8137.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250888.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4537.18 sec) 06:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254488.138709 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 937.18 sec) 06:29:00 [GPUGRID] [rr_sim] 2255425.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991888435.27G/439.78G) 06:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851806.23 06:29:00 [---] pick_jobs_to_run() start 06:29:00 [---] pick_jobs_to_run() end 06:29:00 [---] [rr_sim] end 06:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:29:00 [---] [cpu_sched_debug] preliminary job list: 06:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:29:00 [---] [cpu_sched_debug] final job list: 06:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:29:00 [---] [cpu_sched_debug] enforce_run_list: end 06:29:00 [---] choose_project(): 23340.000000 06:29:00 [iThena.Measurements] doing RPC to get work 06:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 585059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412259.09 cpu 2157032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2157032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2157032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:29:00 [---] Got 0 tasks 06:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:30:00 [---] [rr_sim] doing sim: CPU sched 06:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991860844.92G/439.78G) 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255362.58 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 225536.26 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225536.257941 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029826.32 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 202982.63 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428518.890088 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826843.69 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 182684.37 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611203.259020 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644159.32 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 164415.93 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775619.191059 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479743.39 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 147974.34 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923593.529894 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331769.05 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 133176.90 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056770.434846 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198592.14 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 119859.21 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176629.649302 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078732.93 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 107873.29 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284502.942313 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970859.64 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 97085.96 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381588.906023 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873773.67 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 87377.37 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468966.273362 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786396.31 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 78639.63 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547605.903967 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707756.68 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 70775.67 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618381.571511 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636981.01 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 63698.10 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682079.672301 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573282.91 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 57328.29 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739407.963012 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515954.62 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 51595.46 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791003.424652 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464359.15 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 46435.92 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837439.340128 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417923.24 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 41792.32 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879231.664056 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376130.92 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 37613.09 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916844.755591 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338517.82 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 33851.78 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950696.537973 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304666.04 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 30466.60 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981163.142117 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274199.44 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 27419.94 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008583.085846 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246779.49 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 24677.95 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033261.035203 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222101.54 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 22210.15 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055471.189624 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199891.39 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 19989.14 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075460.328602 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179902.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 17990.23 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093450.553683 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161912.03 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 16191.20 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109641.756256 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145720.82 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 14572.08 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124213.838571 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131148.74 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 13114.87 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137328.712655 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118033.87 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 11803.39 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149132.099331 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106230.48 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 10623.05 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159755.147339 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95607.43 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 9560.74 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169315.890546 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86046.69 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 8604.67 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177920.559432 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77442.02 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 7744.20 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185664.761430 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69697.82 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 6969.78 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192634.543228 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62728.04 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 6272.80 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198907.346846 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56455.23 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 5645.52 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204552.870103 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50809.71 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 5080.97 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209633.841034 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45728.74 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 4572.87 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214206.714871 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41155.86 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 4115.59 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218322.301325 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37040.28 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3704.03 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222026.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33336.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225626.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29736.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229226.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26136.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232826.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22536.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236426.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18936.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240026.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15336.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243626.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11736.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247226.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8136.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250826.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4536.25 sec) 06:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254426.329134 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 936.25 sec) 06:30:00 [GPUGRID] [rr_sim] 2255362.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991860844.92G/439.78G) 06:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851803.49 06:30:00 [---] pick_jobs_to_run() start 06:30:00 [---] pick_jobs_to_run() end 06:30:00 [---] [rr_sim] end 06:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:30:00 [---] [cpu_sched_debug] preliminary job list: 06:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:30:00 [---] [cpu_sched_debug] final job list: 06:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:30:00 [---] [cpu_sched_debug] enforce_run_list: end 06:30:00 [---] choose_project(): 23400.000000 06:30:00 [iThena.Measurements] doing RPC to get work 06:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412199.09 cpu 2156972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:30:00 [---] Got 0 tasks 06:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:31:00 [---] [rr_sim] doing sim: CPU sched 06:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991833254.56G/439.78G) 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255299.84 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 225529.98 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225529.984253 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029769.86 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 202976.99 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428506.970080 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826792.87 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 182679.29 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611186.257325 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644113.59 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 164411.36 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775597.615846 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479702.23 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 147970.22 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923567.838514 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331732.00 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 133173.20 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056741.038915 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198558.80 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 119855.88 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176596.919277 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078702.92 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 107870.29 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284467.211602 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970832.63 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 97083.26 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381550.474694 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873749.37 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 87374.94 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468925.411478 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786374.43 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 78637.44 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547562.854583 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707736.99 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 70773.70 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618336.553377 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636963.29 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 63696.33 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1682032.882293 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573266.96 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 57326.70 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739359.578316 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515940.26 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 51594.03 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790953.604737 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464346.24 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 46434.62 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837388.228516 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417911.61 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 41791.16 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879179.389918 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376120.45 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 37612.05 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916791.435179 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338508.41 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 33850.84 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950642.275914 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304657.57 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 30465.76 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981108.032575 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274191.81 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 27419.18 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008527.213570 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246772.63 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 24677.26 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033204.476466 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222095.37 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 22209.54 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055414.013072 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199885.83 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 19988.58 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075402.596018 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179897.25 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 17989.72 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093392.320669 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161907.52 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 16190.75 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109583.072855 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145716.77 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 14571.68 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124154.749822 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131145.09 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 13114.51 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137269.259093 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118030.58 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 11803.06 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149072.317437 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106227.53 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 10622.75 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159695.069946 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95604.77 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 9560.48 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169255.547204 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86044.30 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 8604.43 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177859.976736 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77439.87 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 7743.99 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185603.963316 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69695.88 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 6969.59 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192573.551237 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62726.29 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 6272.63 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198846.180366 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56453.66 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 5645.37 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204491.546582 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50808.30 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 5080.83 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209572.376177 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45727.47 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 4572.75 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214145.122812 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41154.72 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 4115.47 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218260.594784 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37039.25 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3703.92 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221964.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33335.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225564.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29735.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229164.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26135.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232764.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22535.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236364.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18935.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239964.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15335.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243564.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11735.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247164.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8135.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250764.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4535.32 sec) 06:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254364.519558 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 935.32 sec) 06:31:00 [GPUGRID] [rr_sim] 2255299.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991833254.56G/439.78G) 06:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851800.76 06:31:00 [---] pick_jobs_to_run() start 06:31:00 [---] pick_jobs_to_run() end 06:31:00 [---] [rr_sim] end 06:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:31:00 [---] [cpu_sched_debug] preliminary job list: 06:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:31:00 [---] [cpu_sched_debug] final job list: 06:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:31:00 [---] [cpu_sched_debug] enforce_run_list: end 06:31:00 [---] choose_project(): 23460.000000 06:31:00 [iThena.Measurements] doing RPC to get work 06:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412139.09 cpu 2156912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:31:00 [---] Got 0 tasks 06:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:32:00 [---] [rr_sim] doing sim: CPU sched 06:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991805664.21G/439.78G) 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255237.11 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 225523.71 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225523.710565 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029713.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 202971.34 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428495.050073 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826742.06 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 182674.21 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611169.255630 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644067.85 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 164406.79 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775576.040632 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479661.07 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 147966.11 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923542.147133 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331694.96 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 133169.50 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056711.642984 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198525.46 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 119852.55 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176564.189251 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078672.92 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 107867.29 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284431.480890 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970805.62 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 97080.56 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381512.043366 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873725.06 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 87372.51 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468884.549594 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786352.56 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 78635.26 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547519.805199 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707717.30 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 70771.73 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618291.535244 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636945.57 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 63694.56 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681986.092284 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573251.01 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 57325.10 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739311.193620 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515925.91 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 51592.59 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790903.784823 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464333.32 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 46433.33 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837337.116905 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417899.99 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 41790.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879127.115779 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376109.99 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 37611.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916738.114766 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338498.99 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 33849.90 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950588.013854 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304649.09 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 30464.91 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981052.923033 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274184.18 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 27418.42 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008471.341295 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246765.76 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 24676.58 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033147.917730 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222089.19 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 22208.92 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055356.836521 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199880.27 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 19988.03 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075344.863434 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179892.24 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 17989.22 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093334.087655 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161903.02 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 16190.30 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109524.389454 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145712.72 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 14571.27 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124095.661073 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131141.44 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 13114.14 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137209.805531 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118027.30 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 11802.73 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149012.535542 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106224.57 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 10622.46 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159634.992553 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95602.11 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 9560.21 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169195.203862 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86041.90 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 8604.19 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177799.394040 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77437.71 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 7743.77 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185543.165201 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69693.94 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 6969.39 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192512.559246 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62724.55 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 6272.45 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198785.013886 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56452.09 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 5645.21 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204430.223062 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50806.88 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 5080.69 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209510.911320 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45726.19 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 4572.62 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214083.530753 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41153.57 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 4115.36 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218198.888242 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37038.22 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3703.82 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221902.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33334.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225502.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29734.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229102.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26134.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232702.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22534.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236302.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18934.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239902.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15334.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243502.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11734.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247102.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8134.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250702.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4534.40 sec) 06:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254302.709982 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 934.40 sec) 06:32:00 [GPUGRID] [rr_sim] 2255237.11: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991805664.21G/439.78G) 06:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851798.02 06:32:00 [---] pick_jobs_to_run() start 06:32:00 [---] pick_jobs_to_run() end 06:32:00 [---] [rr_sim] end 06:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:32:00 [---] [cpu_sched_debug] preliminary job list: 06:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:32:00 [---] [cpu_sched_debug] final job list: 06:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:32:00 [---] [cpu_sched_debug] enforce_run_list: end 06:32:00 [---] choose_project(): 23520.000000 06:32:00 [iThena.Measurements] doing RPC to get work 06:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412079.09 cpu 2156852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:32:00 [---] Got 0 tasks 06:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:33:00 [---] [rr_sim] doing sim: CPU sched 06:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991778073.85G/439.78G) 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255174.37 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 225517.44 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225517.436876 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029656.93 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 202965.69 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428483.130065 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826691.24 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 182669.12 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611152.253935 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1644022.11 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 164402.21 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775554.465418 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479619.90 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 147961.99 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923516.455753 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331657.91 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 133165.79 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056682.247054 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198492.12 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 119849.21 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176531.459225 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078642.91 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 107864.29 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284395.750179 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970778.62 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 97077.86 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381473.612037 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873700.76 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 87370.08 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468843.687710 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786330.68 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 78633.07 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547476.755815 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707697.61 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 70769.76 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618246.517110 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636927.85 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 63692.79 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681939.302276 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573235.07 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 57323.51 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739262.808924 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515911.56 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 51591.16 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790853.964908 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464320.40 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 46432.04 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837286.005294 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417888.36 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 41788.84 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879074.841641 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376099.53 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 37609.95 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916684.794353 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338489.57 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 33848.96 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950533.751794 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304640.62 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 30464.06 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980997.813491 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274176.56 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 27417.66 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008415.469019 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246758.90 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 24675.89 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033091.358993 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222083.01 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 22208.30 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055299.659970 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199874.71 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 19987.47 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075287.130850 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179887.24 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 17988.72 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093275.854641 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161898.51 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 16189.85 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109465.706053 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145708.66 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 14570.87 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124036.572324 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131137.80 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 13113.78 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137150.351968 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118024.02 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 11802.40 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148952.753648 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106221.62 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 10622.16 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159574.915160 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95599.45 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 9559.95 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169134.860520 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86039.51 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 8603.95 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177738.811344 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77435.56 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 7743.56 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185482.367086 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69692.00 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 6969.20 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192451.567254 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62722.80 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 6272.28 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198723.847405 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56450.52 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 5645.05 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204368.899541 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50805.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 5080.55 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209449.446463 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45724.92 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 4572.49 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214021.938693 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41152.43 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 4115.24 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218137.181700 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37037.19 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3703.72 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221840.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33333.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225440.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29733.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229040.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26133.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232640.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22533.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236240.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18933.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239840.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15333.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243440.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11733.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247040.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8133.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250640.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4533.47 sec) 06:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254240.900407 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 933.47 sec) 06:33:00 [GPUGRID] [rr_sim] 2255174.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991778073.85G/439.78G) 06:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851795.28 06:33:00 [---] pick_jobs_to_run() start 06:33:00 [---] pick_jobs_to_run() end 06:33:00 [---] [rr_sim] end 06:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:33:00 [---] [cpu_sched_debug] preliminary job list: 06:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:33:00 [---] [cpu_sched_debug] final job list: 06:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:33:00 [---] [cpu_sched_debug] enforce_run_list: end 06:33:00 [---] choose_project(): 23580.000000 06:33:00 [iThena.Measurements] doing RPC to get work 06:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 412019.09 cpu 2156792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:33:00 [---] Got 0 tasks 06:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:35:00 [---] [rr_sim] doing sim: CPU sched 06:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991750483.49G/439.78G) 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255111.63 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 225511.16 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225511.163188 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029600.47 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 202960.05 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428471.210058 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826640.42 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 182664.04 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611135.252240 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643976.38 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 164397.64 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775532.890204 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479578.74 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 147957.87 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923490.764372 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331620.87 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 133162.09 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056652.851123 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198458.78 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 119845.88 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176498.729199 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078612.90 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 107861.29 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284360.019467 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970751.61 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 97075.16 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381435.180709 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873676.45 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 87367.65 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468802.825826 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786308.81 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 78630.88 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547433.706432 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707677.93 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 70767.79 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618201.498977 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636910.13 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 63691.01 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681892.512267 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573219.12 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 57321.91 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739214.424229 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515897.21 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 51589.72 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790804.144994 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464307.49 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 46430.75 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837234.893683 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417876.74 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 41787.67 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879022.567503 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376089.06 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 37608.91 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916631.473941 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338480.16 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 33848.02 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950479.489735 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304632.14 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 30463.21 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980942.703949 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274168.93 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 27416.89 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008359.596743 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246752.04 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 24675.20 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033034.800257 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222076.83 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 22207.68 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055242.483419 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199869.15 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 19986.91 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075229.398265 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179882.23 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 17988.22 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093217.621627 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161894.01 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 16189.40 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109407.022653 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145704.61 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 14570.46 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123977.483575 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131134.15 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 13113.41 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137090.898406 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118020.73 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 11802.07 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148892.971754 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106218.66 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 10621.87 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159514.837767 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95596.79 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 9559.68 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169074.517178 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86037.11 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 8603.71 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177678.228648 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77433.40 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 7743.34 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185421.568972 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69690.06 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 6969.01 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192390.575263 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62721.06 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 6272.11 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198662.680925 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56448.95 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 5644.90 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204307.576020 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50804.06 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 5080.41 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209387.981607 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45723.65 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 4572.37 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213960.346634 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41151.29 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 4115.13 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218075.475159 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37036.16 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3703.62 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221779.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33332.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225379.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29732.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228979.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26132.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232579.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22532.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236179.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18932.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239779.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15332.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243379.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11732.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246979.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8132.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250579.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4532.54 sec) 06:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254179.090831 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 932.54 sec) 06:35:00 [GPUGRID] [rr_sim] 2255111.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991750483.49G/439.78G) 06:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851852.54 06:35:00 [---] pick_jobs_to_run() start 06:35:00 [---] pick_jobs_to_run() end 06:35:00 [---] [rr_sim] end 06:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:35:00 [---] [cpu_sched_debug] preliminary job list: 06:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 06:35:00 [---] [cpu_sched_debug] final job list: 06:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:35:00 [---] [cpu_sched_debug] enforce_run_list: end 06:35:00 [---] choose_project(): 23700.000000 06:35:00 [iThena.Measurements] doing RPC to get work 06:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411899.09 cpu 2156732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:35:00 [---] Got 0 tasks 06:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:36:00 [---] [rr_sim] doing sim: CPU sched 06:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991722893.14G/439.78G) 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2255048.89 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 225504.89 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225504.889500 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029544.01 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 202954.40 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428459.290050 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826589.60 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 182658.96 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611118.250545 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643930.64 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 164393.06 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775511.314990 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479537.58 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 147953.76 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923465.072991 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331583.82 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 133158.38 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056623.455192 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198425.44 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 119842.54 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176465.999173 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078582.90 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 107858.29 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284324.288756 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970724.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 97072.46 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381396.749380 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873652.15 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 87365.21 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468761.963942 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786286.93 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 78628.69 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547390.657048 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707658.24 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 70765.82 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618156.480843 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636892.41 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 63689.24 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681845.722259 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573203.17 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 57320.32 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739166.039533 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515882.86 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 51588.29 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790754.325079 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464294.57 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 46429.46 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837183.782071 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417865.11 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 41786.51 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878970.293364 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376078.60 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 37607.86 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916578.153528 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338470.74 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 33847.07 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950425.227675 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304623.67 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 30462.37 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980887.594408 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274161.30 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 27416.13 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008303.724467 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246745.17 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 24674.52 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032978.241520 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222070.65 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 22207.07 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055185.306868 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199863.59 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 19986.36 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075171.665681 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179877.23 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 17987.72 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093159.388613 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161889.51 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 16188.95 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109348.339252 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145700.56 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 14570.06 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123918.394827 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131130.50 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 13113.05 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137031.444844 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118017.45 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 11801.75 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148833.189859 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106215.71 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 10621.57 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159454.760373 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95594.13 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 9559.41 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169014.173836 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86034.72 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 8603.47 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177617.645952 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77431.25 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 7743.12 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185360.770857 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69688.12 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 6968.81 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192329.583271 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62719.31 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 6271.93 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198601.514444 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56447.38 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 5644.74 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204246.252500 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50802.64 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 5080.26 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209326.516750 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45722.38 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 4572.24 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213898.754575 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41150.14 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 4115.01 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218013.768617 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37035.13 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3703.51 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221717.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33331.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225317.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29731.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228917.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26131.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232517.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22531.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236117.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18931.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239717.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15331.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243317.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11731.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246917.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8131.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250517.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4531.61 sec) 06:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254117.281256 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 931.61 sec) 06:36:00 [GPUGRID] [rr_sim] 2255048.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991722893.14G/439.78G) 06:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851849.81 06:36:00 [---] pick_jobs_to_run() start 06:36:00 [---] pick_jobs_to_run() end 06:36:00 [---] [rr_sim] end 06:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:36:00 [---] [cpu_sched_debug] preliminary job list: 06:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:36:00 [---] [cpu_sched_debug] final job list: 06:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:36:00 [---] [cpu_sched_debug] enforce_run_list: end 06:36:00 [---] choose_project(): 23760.000000 06:36:00 [iThena.Measurements] doing RPC to get work 06:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411839.09 cpu 2156672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:36:00 [---] Got 0 tasks 06:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:37:00 [---] [rr_sim] doing sim: CPU sched 06:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991695302.78G/439.78G) 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254986.16 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 225498.62 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225498.615812 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029487.54 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 202948.75 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428447.370042 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826538.79 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 182653.88 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611101.248850 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643884.91 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 164388.49 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775489.739777 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479496.42 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 147949.64 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923439.381611 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331546.78 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 133154.68 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056594.059261 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198392.10 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 119839.21 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176433.269147 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078552.89 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 107855.29 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284288.558044 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970697.60 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 97069.76 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381358.318051 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873627.84 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 87362.78 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468721.102058 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786265.06 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 78626.51 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547347.607664 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707638.55 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 70763.86 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618111.462709 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636874.70 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 63687.47 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681798.932250 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573187.23 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 57318.72 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739117.654837 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515868.50 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 51586.85 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790704.505165 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464281.65 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 46428.17 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837132.670460 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417853.49 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 41785.35 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878918.019226 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376068.14 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 37606.81 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916524.833115 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338461.33 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 33846.13 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950370.965615 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304615.19 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 30461.52 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980832.484866 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274153.67 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 27415.37 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008247.852191 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246738.31 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 24673.83 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032921.682783 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222064.48 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 22206.45 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055128.130317 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199858.03 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 19985.80 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075113.933097 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179872.23 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 17987.22 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093101.155599 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161885.00 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 16188.50 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109289.655851 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145696.50 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 14569.65 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123859.306078 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131126.85 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 13112.69 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136971.991282 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118014.17 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 11801.42 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148773.407965 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106212.75 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 10621.28 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159394.682980 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95591.48 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 9559.15 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168953.830494 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86032.33 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 8603.23 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177557.063256 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77429.09 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 7742.91 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185299.972743 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69686.19 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 6968.62 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192268.591280 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62717.57 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 6271.76 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198540.347964 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56445.81 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 5644.58 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204184.928979 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50801.23 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 5080.12 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209265.051893 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45721.11 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 4572.11 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213837.162515 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41149.00 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 4114.90 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217952.062076 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37034.10 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3703.41 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221655.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33330.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225255.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29730.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228855.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26130.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232455.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22530.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236055.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18930.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239655.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15330.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243255.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11730.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246855.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8130.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250455.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4530.69 sec) 06:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2254055.471680 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 930.69 sec) 06:37:00 [GPUGRID] [rr_sim] 2254986.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991695302.78G/439.78G) 06:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851847.07 06:37:00 [---] pick_jobs_to_run() start 06:37:00 [---] pick_jobs_to_run() end 06:37:00 [---] [rr_sim] end 06:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:37:00 [---] [cpu_sched_debug] preliminary job list: 06:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:37:00 [---] [cpu_sched_debug] final job list: 06:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:37:00 [---] [cpu_sched_debug] enforce_run_list: end 06:37:00 [---] choose_project(): 23820.000000 06:37:00 [iThena.Measurements] doing RPC to get work 06:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411779.09 cpu 2156612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:37:00 [---] Got 0 tasks 06:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:38:00 [---] [rr_sim] doing sim: CPU sched 06:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991667712.42G/439.78G) 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254923.42 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 225492.34 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225492.342124 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029431.08 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 202943.11 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428435.450035 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826487.97 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 182648.80 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611084.247155 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643839.17 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 164383.92 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775468.164563 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479455.26 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 147945.53 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923413.690230 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331509.73 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 133150.97 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056564.663331 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198358.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 119835.88 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176400.539121 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078522.88 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 107852.29 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284252.827333 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970670.59 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 97067.06 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381319.886723 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873603.53 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 87360.35 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468680.240174 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786243.18 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 78624.32 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547304.558280 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707618.86 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 70761.89 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618066.444576 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636856.98 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 63685.70 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681752.142242 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573171.28 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 57317.13 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739069.270141 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515854.15 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 51585.42 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790654.685251 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464268.74 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 46426.87 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837081.558849 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417841.86 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 41784.19 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878865.745088 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376057.68 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 37605.77 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916471.512702 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338451.91 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 33845.19 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950316.703556 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304606.72 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 30460.67 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980777.375324 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274146.05 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 27414.60 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008191.979915 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246731.44 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 24673.14 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032865.124047 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222058.30 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 22205.83 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055070.953766 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199852.47 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 19985.25 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075056.200513 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179867.22 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 17986.72 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093042.922585 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161880.50 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 16188.05 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109230.972450 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145692.45 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 14569.24 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123800.217329 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131123.20 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 13112.32 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136912.537719 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118010.88 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 11801.09 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148713.626071 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106209.80 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 10620.98 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159334.605587 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95588.82 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 9558.88 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168893.487152 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86029.93 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 8602.99 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177496.480560 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77426.94 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 7742.69 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185239.174628 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69684.25 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 6968.42 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192207.599289 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62715.82 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 6271.58 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198479.181483 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56444.24 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 5644.42 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204123.605459 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50799.82 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 5079.98 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209203.587036 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45719.83 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 4571.98 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213775.570456 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41147.85 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 4114.79 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217890.355534 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37033.07 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3703.31 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221593.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33329.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225193.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29729.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228793.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26129.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232393.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22529.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235993.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18929.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239593.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15329.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243193.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11729.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246793.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8129.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250393.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4529.76 sec) 06:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253993.662104 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 929.76 sec) 06:38:00 [GPUGRID] [rr_sim] 2254923.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991667712.42G/439.78G) 06:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851844.33 06:38:00 [---] pick_jobs_to_run() start 06:38:00 [---] pick_jobs_to_run() end 06:38:00 [---] [rr_sim] end 06:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:38:00 [---] [cpu_sched_debug] preliminary job list: 06:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:38:00 [---] [cpu_sched_debug] final job list: 06:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:38:00 [---] [cpu_sched_debug] enforce_run_list: end 06:38:00 [---] choose_project(): 23880.000000 06:38:00 [iThena.Measurements] doing RPC to get work 06:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411719.09 cpu 2156552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:38:00 [---] Got 0 tasks 06:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:39:00 [---] [rr_sim] doing sim: CPU sched 06:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991640122.07G/439.78G) 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254860.68 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 225486.07 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225486.068435 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029374.62 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 202937.46 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428423.530027 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826437.15 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 182643.72 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611067.245460 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643793.44 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 164379.34 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775446.589349 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479414.10 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 147941.41 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923387.998849 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331472.69 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 133147.27 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056535.267400 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198325.42 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 119832.54 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176367.809095 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078492.88 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 107849.29 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284217.096621 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970643.59 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 97064.36 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381281.455394 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873579.23 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 87357.92 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468639.378290 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786221.31 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 78622.13 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547261.508896 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707599.18 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 70759.92 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1618021.426442 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636839.26 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 63683.93 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681705.352233 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573155.33 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 57315.53 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1739020.885445 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515839.80 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 51583.98 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790604.865336 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464255.82 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 46425.58 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837030.447238 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417830.24 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 41783.02 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878813.470949 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376047.21 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 37604.72 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916418.192290 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338442.49 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 33844.25 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950262.441496 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304598.24 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 30459.82 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980722.265782 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274138.42 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 27413.84 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008136.107639 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246724.58 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 24672.46 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032808.565310 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222052.12 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 22205.21 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055013.777215 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199846.91 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 19984.69 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074998.467929 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179862.22 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 17986.22 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092984.689571 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161875.99 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 16187.60 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109172.289049 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145688.40 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 14568.84 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123741.128580 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131119.56 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 13111.96 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136853.084157 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118007.60 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 11800.76 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148653.844177 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106206.84 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 10620.68 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159274.528194 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95586.16 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 9558.62 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168833.143810 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86027.54 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 8602.75 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177435.897864 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77424.79 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 7742.48 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185178.376513 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69682.31 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 6968.23 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192146.607297 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62714.08 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 6271.41 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198418.015003 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56442.67 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 5644.27 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204062.281938 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50798.40 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 5079.84 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209142.122179 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45718.56 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 4571.86 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213713.978397 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41146.71 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 4114.67 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217828.648993 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37032.04 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3703.20 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221531.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33328.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225131.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29728.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228731.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26128.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232331.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22528.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235931.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18928.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239531.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15328.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243131.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11728.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246731.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8128.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250331.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4528.83 sec) 06:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253931.852529 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 928.83 sec) 06:39:00 [GPUGRID] [rr_sim] 2254860.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991640122.07G/439.78G) 06:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851841.60 06:39:00 [---] pick_jobs_to_run() start 06:39:00 [---] pick_jobs_to_run() end 06:39:00 [---] [rr_sim] end 06:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:39:00 [---] [cpu_sched_debug] preliminary job list: 06:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:39:00 [---] [cpu_sched_debug] final job list: 06:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:39:00 [---] [cpu_sched_debug] enforce_run_list: end 06:39:00 [---] choose_project(): 23940.000000 06:39:00 [iThena.Measurements] doing RPC to get work 06:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411659.09 cpu 2156492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:39:00 [---] Got 0 tasks 06:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:40:00 [---] [rr_sim] doing sim: CPU sched 06:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991612531.71G/439.78G) 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254797.95 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 225479.79 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225479.794747 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029318.15 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 202931.82 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428411.610020 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826386.34 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 182638.63 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611050.243765 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643747.70 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 164374.77 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775425.014135 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479372.93 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 147937.29 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923362.307469 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331435.64 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 133143.56 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056505.871469 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198292.08 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 119829.21 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176335.079069 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078462.87 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 107846.29 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284181.365909 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970616.58 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 97061.66 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381243.024066 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873554.92 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 87355.49 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468598.516406 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786199.43 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 78619.94 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547218.459513 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707579.49 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 70757.95 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617976.408309 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636821.54 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 63682.15 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681658.562225 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573139.39 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 57313.94 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738972.500749 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515825.45 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 51582.54 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790555.045422 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464242.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 46424.29 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836979.335627 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417818.61 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 41781.86 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878761.196811 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376036.75 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 37603.68 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916364.871877 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338433.08 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 33843.31 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950208.179436 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304589.77 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 30458.98 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980667.156240 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274130.79 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 27413.08 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008080.235363 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246717.71 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 24671.77 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032752.006574 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222045.94 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 22204.59 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054956.600664 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199841.35 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 19984.13 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074940.735344 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179857.21 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 17985.72 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092926.456557 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161871.49 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 16187.15 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109113.605648 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145684.34 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 14568.43 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123682.039831 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131115.91 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 13111.59 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136793.630595 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118004.32 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 11800.43 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148594.062282 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106203.89 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 10620.39 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159214.450801 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95583.50 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 9558.35 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168772.800468 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86025.15 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 8602.51 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177375.315168 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77422.63 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 7742.26 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185117.578399 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69680.37 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 6968.04 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192085.615306 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62712.33 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 6271.23 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198356.848522 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56441.10 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 5644.11 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204000.958417 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50796.99 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 5079.70 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209080.657323 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45717.29 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 4571.73 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213652.386338 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41145.56 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 4114.56 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217766.942451 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37031.01 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3703.10 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221470.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33327.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225070.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29727.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228670.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26127.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232270.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22527.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235870.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18927.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239470.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15327.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243070.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11727.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246670.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8127.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250270.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4527.90 sec) 06:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253870.042953 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 927.90 sec) 06:40:00 [GPUGRID] [rr_sim] 2254797.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991612531.71G/439.78G) 06:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851838.86 06:40:00 [---] pick_jobs_to_run() start 06:40:00 [---] pick_jobs_to_run() end 06:40:00 [---] [rr_sim] end 06:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:40:00 [---] [cpu_sched_debug] preliminary job list: 06:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:40:00 [---] [cpu_sched_debug] final job list: 06:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:40:00 [---] [cpu_sched_debug] enforce_run_list: end 06:40:00 [---] choose_project(): 24000.000000 06:40:00 [iThena.Measurements] doing RPC to get work 06:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411599.09 cpu 2156432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:40:00 [---] Got 0 tasks 06:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:41:00 [---] [rr_sim] doing sim: CPU sched 06:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991584941.36G/439.78G) 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254735.21 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 225473.52 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225473.521059 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029261.69 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 202926.17 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428399.690012 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826335.52 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 182633.55 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611033.242070 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643701.97 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 164370.20 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775403.438922 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479331.77 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 147933.18 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923336.616088 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331398.59 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 133139.86 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056476.475538 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198258.74 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 119825.87 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176302.349043 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078432.86 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 107843.29 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284145.635198 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970589.58 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 97058.96 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381204.592737 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873530.62 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 87353.06 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468557.654522 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786177.56 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 78617.76 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547175.410129 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707559.80 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 70755.98 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617931.390175 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636803.82 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 63680.38 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681611.772216 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573123.44 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 57312.34 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738924.116054 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515811.09 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 51581.11 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790505.225507 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464229.99 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 46423.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836928.224015 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417806.99 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 41780.70 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878708.922673 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376026.29 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 37602.63 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916311.551464 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338423.66 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 33842.37 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950153.917377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304581.29 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 30458.13 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980612.046698 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274123.16 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 27412.32 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008024.363087 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246710.85 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 24671.08 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032695.447837 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222039.76 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 22203.98 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054899.424112 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199835.79 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 19983.58 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074883.002760 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179852.21 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 17985.22 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092868.223543 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161866.99 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 16186.70 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109054.922248 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145680.29 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 14568.03 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123622.951082 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131112.26 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 13111.23 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136734.177032 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118001.03 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 11800.10 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148534.280388 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106200.93 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 10620.09 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159154.373408 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95580.84 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 9558.08 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168712.457126 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86022.75 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 8602.28 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177314.732472 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77420.48 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 7742.05 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185056.780284 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69678.43 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 6967.84 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192024.623315 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62710.59 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 6271.06 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198295.682042 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56439.53 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 5643.95 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203939.634897 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50795.58 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 5079.56 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209019.192466 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45716.02 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 4571.60 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213590.794278 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41144.42 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 4114.44 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217705.235909 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37029.97 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3703.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221408.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33326.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225008.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29726.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228608.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26126.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232208.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22526.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235808.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18926.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239408.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15326.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243008.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11726.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246608.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8126.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250208.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4526.98 sec) 06:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253808.233377 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 926.98 sec) 06:41:00 [GPUGRID] [rr_sim] 2254735.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991584941.36G/439.78G) 06:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851836.12 06:41:00 [---] pick_jobs_to_run() start 06:41:00 [---] pick_jobs_to_run() end 06:41:00 [---] [rr_sim] end 06:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:41:00 [---] [cpu_sched_debug] preliminary job list: 06:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:41:00 [---] [cpu_sched_debug] final job list: 06:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:41:00 [---] [cpu_sched_debug] enforce_run_list: end 06:41:00 [---] choose_project(): 24060.000000 06:41:00 [iThena.Measurements] doing RPC to get work 06:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411539.09 cpu 2156372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:41:00 [---] Got 0 tasks 06:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:42:00 [---] [rr_sim] doing sim: CPU sched 06:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991557351.00G/439.78G) 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254672.47 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 225467.25 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225467.247371 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029205.23 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 202920.52 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428387.770004 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826284.70 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 182628.47 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 611016.240375 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643656.23 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 164365.62 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775381.863708 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479290.61 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 147929.06 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923310.924708 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331361.55 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 133136.15 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056447.079608 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198225.39 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 119822.54 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176269.619017 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078402.85 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 107840.29 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284109.904486 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970562.57 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 97056.26 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381166.161408 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873506.31 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 87350.63 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468516.792638 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786155.68 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 78615.57 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547132.360745 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707540.11 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 70754.01 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617886.372041 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636786.10 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 63678.61 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681564.982208 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573107.49 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 57310.75 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738875.731358 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515796.74 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 51579.67 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790455.405593 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464217.07 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 46421.71 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836877.112404 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417795.36 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 41779.54 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878656.648534 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376015.83 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 37601.58 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916258.231052 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338414.24 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 33841.42 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950099.655317 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304572.82 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 30457.28 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980556.937156 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274115.54 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 27411.55 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007968.490811 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246703.98 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 24670.40 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032638.889101 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222033.58 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 22203.36 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054842.247561 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199830.23 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 19983.02 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074825.270176 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179847.20 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 17984.72 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092809.990529 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161862.48 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 16186.25 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108996.238847 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145676.23 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 14567.62 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123563.862333 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131108.61 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 13110.86 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136674.723470 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117997.75 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 11799.78 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148474.498494 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106197.98 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 10619.80 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159094.296015 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95578.18 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 9557.82 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168652.113784 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86020.36 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 8602.04 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177254.149776 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77418.32 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 7741.83 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184995.982170 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69676.49 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 6967.65 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191963.631323 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62708.84 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 6270.88 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198234.515562 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56437.96 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 5643.80 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203878.311376 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50794.16 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 5079.42 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208957.727609 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45714.75 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 4571.47 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213529.202219 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41143.27 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 4114.33 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217643.529368 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37028.94 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3702.89 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221346.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33326.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224946.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29726.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228546.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26126.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232146.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22526.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235746.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18926.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239346.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15326.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242946.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11726.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246546.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8126.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250146.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4526.05 sec) 06:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253746.423802 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 926.05 sec) 06:42:00 [GPUGRID] [rr_sim] 2254672.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991557351.00G/439.78G) 06:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851833.39 06:42:00 [---] pick_jobs_to_run() start 06:42:00 [---] pick_jobs_to_run() end 06:42:00 [---] [rr_sim] end 06:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:42:00 [---] [cpu_sched_debug] preliminary job list: 06:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:42:00 [---] [cpu_sched_debug] final job list: 06:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:42:00 [---] [cpu_sched_debug] enforce_run_list: end 06:42:00 [---] choose_project(): 24120.000000 06:42:00 [iThena.Measurements] doing RPC to get work 06:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411479.09 cpu 2156312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:42:00 [---] Got 0 tasks 06:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:43:00 [---] [rr_sim] doing sim: CPU sched 06:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991529760.64G/439.78G) 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254609.74 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 225460.97 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225460.973682 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029148.76 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 202914.88 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428375.849997 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826233.89 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 182623.39 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610999.238679 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643610.50 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 164361.05 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775360.288494 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479249.45 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 147924.94 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923285.233327 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331324.50 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 133132.45 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056417.683677 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198192.05 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 119819.21 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176236.888992 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078372.85 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 107837.28 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284074.173775 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970535.56 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 97053.56 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381127.730080 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873482.01 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 87348.20 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468475.930754 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786133.81 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 78613.38 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547089.311361 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707520.43 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 70752.04 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617841.353908 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636768.38 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 63676.84 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681518.192199 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573091.54 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 57309.15 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738827.346662 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515782.39 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 51578.24 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790405.585678 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464204.15 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 46420.42 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836826.000793 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417783.74 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 41778.37 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878604.374396 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376005.36 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 37600.54 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916204.910639 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338404.83 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 33840.48 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950045.393257 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304564.34 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 30456.43 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980501.827614 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274107.91 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 27410.79 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007912.618535 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246697.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 24669.71 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032582.330364 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222027.41 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 22202.74 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054785.071010 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199824.67 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 19982.47 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074767.537592 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179842.20 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 17984.22 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092751.757515 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161857.98 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 16185.80 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108937.555446 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145672.18 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 14567.22 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123504.773584 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131104.96 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 13110.50 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136615.269908 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117994.47 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 11799.45 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148414.716600 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106195.02 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 10619.50 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159034.218622 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95575.52 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 9557.55 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168591.770442 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86017.97 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 8601.80 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177193.567081 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77416.17 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 7741.62 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184935.184055 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69674.55 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 6967.46 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191902.639332 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62707.10 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 6270.71 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198173.349081 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56436.39 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 5643.64 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203816.987855 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50792.75 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 5079.27 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208896.262752 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45713.47 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 4571.35 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213467.610160 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41142.13 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 4114.21 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217581.822826 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37027.91 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3702.79 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221284.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33325.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224884.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29725.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228484.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26125.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232084.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22525.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235684.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18925.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239284.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15325.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242884.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11725.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246484.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8125.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250084.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4525.12 sec) 06:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253684.614226 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 925.12 sec) 06:43:00 [GPUGRID] [rr_sim] 2254609.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991529760.64G/439.78G) 06:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851830.65 06:43:00 [---] pick_jobs_to_run() start 06:43:00 [---] pick_jobs_to_run() end 06:43:00 [---] [rr_sim] end 06:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:43:00 [---] [cpu_sched_debug] preliminary job list: 06:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:43:00 [---] [cpu_sched_debug] final job list: 06:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:43:00 [---] [cpu_sched_debug] enforce_run_list: end 06:43:00 [---] choose_project(): 24180.000000 06:43:00 [iThena.Measurements] doing RPC to get work 06:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411419.09 cpu 2156252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:43:00 [---] Got 0 tasks 06:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:44:00 [---] [rr_sim] doing sim: CPU sched 06:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991502170.29G/439.78G) 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254547.00 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 225454.70 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225454.699994 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029092.30 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 202909.23 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428363.929989 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826183.07 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 182618.31 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610982.236984 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643564.76 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 164356.48 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775338.713280 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479208.29 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 147920.83 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923259.541946 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331287.46 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 133128.75 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056388.287746 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198158.71 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 119815.87 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176204.158966 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078342.84 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 107834.28 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284038.443063 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970508.56 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 97050.86 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381089.298751 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873457.70 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 87345.77 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468435.068870 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786111.93 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 78611.19 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547046.261978 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707500.74 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 70750.07 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617796.335774 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636750.66 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 63675.07 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681471.402191 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573075.60 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 57307.56 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738778.961966 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515768.04 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 51576.80 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790355.765764 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464191.23 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 46419.12 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836774.889182 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417772.11 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 41777.21 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878552.100258 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375994.90 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 37599.49 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916151.590226 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338395.41 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 33839.54 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949991.131198 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304555.87 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 30455.59 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980446.718072 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274100.28 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 27410.03 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007856.746259 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246690.25 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 24669.03 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032525.771628 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222021.23 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 22202.12 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054727.894459 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199819.11 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 19981.91 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074709.805007 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179837.19 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 17983.72 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092693.524501 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161853.48 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 16185.35 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108878.872045 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145668.13 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 14566.81 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123445.684835 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131101.32 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 13110.13 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136555.816346 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117991.18 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 11799.12 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148354.934705 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106192.07 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 10619.21 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158974.141229 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95572.86 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 9557.29 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168531.427100 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86015.57 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 8601.56 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177132.984385 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77414.02 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 7741.40 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184874.385940 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69672.61 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 6967.26 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191841.647341 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62705.35 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 6270.54 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198112.182601 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56434.82 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 5643.48 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203755.664335 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50791.34 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 5079.13 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208834.797896 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45712.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 4571.22 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213406.018100 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41140.98 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 4114.10 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217520.116285 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37026.88 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3702.69 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221222.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33324.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224822.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29724.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228422.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26124.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232022.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22524.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235622.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18924.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239222.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15324.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242822.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11724.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246422.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8124.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250022.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4524.20 sec) 06:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253622.804650 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 924.20 sec) 06:44:00 [GPUGRID] [rr_sim] 2254547.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991502170.29G/439.78G) 06:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851827.91 06:44:00 [---] pick_jobs_to_run() start 06:44:00 [---] pick_jobs_to_run() end 06:44:00 [---] [rr_sim] end 06:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:44:00 [---] [cpu_sched_debug] preliminary job list: 06:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:44:00 [---] [cpu_sched_debug] final job list: 06:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:44:00 [---] [cpu_sched_debug] enforce_run_list: end 06:44:00 [---] choose_project(): 24240.000000 06:44:00 [iThena.Measurements] doing RPC to get work 06:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411359.09 cpu 2156192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:44:00 [---] Got 0 tasks 06:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:45:00 [---] [rr_sim] doing sim: CPU sched 06:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991474579.93G/439.78G) 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254484.26 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 225448.43 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225448.426306 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2029035.84 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 202903.58 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428352.009981 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826132.25 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 182613.23 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610965.235289 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643519.03 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 164351.90 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775317.138066 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479167.12 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 147916.71 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923233.850566 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331250.41 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 133125.04 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056358.891815 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198125.37 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 119812.54 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176171.428940 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078312.83 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 107831.28 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1284002.712352 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970481.55 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 97048.16 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381050.867423 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873433.40 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 87343.34 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468394.206986 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786090.06 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 78609.01 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1547003.212594 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707481.05 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 70748.11 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617751.317640 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636732.95 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 63673.29 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681424.612182 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573059.65 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 57305.97 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738730.577270 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515753.69 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 51575.37 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790305.945849 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464178.32 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 46417.83 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836723.777570 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417760.49 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 41776.05 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878499.826119 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375984.44 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 37598.44 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916098.269813 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338385.99 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 33838.60 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949936.869138 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304547.39 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 30454.74 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980391.608530 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274092.65 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 27409.27 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007800.873983 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246683.39 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 24668.34 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032469.212891 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222015.05 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 22201.51 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054670.717908 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199813.55 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 19981.35 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074652.072423 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179832.19 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 17983.22 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092635.291487 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161848.97 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 16184.90 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108820.188644 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145664.07 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 14566.41 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123386.596086 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131097.67 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 13109.77 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136496.362783 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117987.90 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 11798.79 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148295.152811 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106189.11 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 10618.91 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158914.063836 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95570.20 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 9557.02 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168471.083758 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86013.18 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 8601.32 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177072.401689 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77411.86 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 7741.19 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184813.587826 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69670.68 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 6967.07 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191780.655349 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62703.61 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 6270.36 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198051.016120 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56433.25 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 5643.32 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203694.340814 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50789.92 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 5078.99 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208773.333039 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45710.93 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 4571.09 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213344.426041 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41139.84 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 4113.98 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217458.409743 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37025.85 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3702.59 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221160.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33323.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224760.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29723.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228360.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26123.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231960.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22523.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235560.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18923.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239160.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15323.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242760.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11723.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246360.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8123.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249960.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4523.27 sec) 06:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253560.995075 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 923.27 sec) 06:45:00 [GPUGRID] [rr_sim] 2254484.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991474579.93G/439.78G) 06:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851825.18 06:45:00 [---] pick_jobs_to_run() start 06:45:00 [---] pick_jobs_to_run() end 06:45:00 [---] [rr_sim] end 06:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:45:00 [---] [cpu_sched_debug] preliminary job list: 06:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:45:00 [---] [cpu_sched_debug] final job list: 06:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:45:00 [---] [cpu_sched_debug] enforce_run_list: end 06:45:00 [---] choose_project(): 24300.000000 06:45:00 [iThena.Measurements] doing RPC to get work 06:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411299.09 cpu 2156132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:45:00 [---] Got 0 tasks 06:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:46:00 [---] [rr_sim] doing sim: CPU sched 06:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991446989.57G/439.78G) 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254421.53 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 225442.15 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225442.152618 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028979.37 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 202897.94 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428340.089974 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826081.44 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 182608.14 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610948.233594 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643473.29 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 164347.33 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775295.562853 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479125.96 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 147912.60 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923208.159185 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331213.37 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 133121.34 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056329.495884 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198092.03 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 119809.20 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176138.698914 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078282.83 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 107828.28 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283966.981640 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970454.54 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 97045.45 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1381012.436094 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873409.09 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 87340.91 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468353.345102 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786068.18 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 78606.82 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546960.163210 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707461.36 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 70746.14 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617706.299507 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636715.23 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 63671.52 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681377.822174 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573043.70 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 57304.37 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738682.192574 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515739.33 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 51573.93 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790256.125935 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464165.40 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 46416.54 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836672.665959 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417748.86 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 41774.89 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878447.551981 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375973.97 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 37597.40 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916044.949401 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338376.58 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 33837.66 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949882.607078 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304538.92 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 30453.89 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980336.498988 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274085.03 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 27408.50 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007745.001707 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246676.52 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 24667.65 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032412.654154 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222008.87 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 22200.89 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054613.541357 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199807.98 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 19980.80 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074594.339839 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179827.19 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 17982.72 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092577.058473 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161844.47 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 16184.45 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108761.505243 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145660.02 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 14566.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123327.507337 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131094.02 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 13109.40 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136436.909221 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117984.62 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 11798.46 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148235.370917 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106186.16 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 10618.62 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158853.986443 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95567.54 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 9556.75 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168410.740416 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86010.79 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 8601.08 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177011.818993 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77409.71 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 7740.97 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184752.789711 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69668.74 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 6966.87 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191719.663358 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62701.86 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 6270.19 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197989.849640 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56431.68 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 5643.17 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203633.017294 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50788.51 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 5078.85 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208711.868182 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45709.66 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 4570.97 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213282.833982 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41138.69 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 4113.87 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217396.703201 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37024.82 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3702.48 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221099.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33322.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224699.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29722.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228299.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26122.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231899.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22522.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235499.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18922.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239099.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15322.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242699.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11722.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246299.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8122.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249899.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4522.34 sec) 06:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253499.185499 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 922.34 sec) 06:46:00 [GPUGRID] [rr_sim] 2254421.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991446989.57G/439.78G) 06:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851822.44 06:46:00 [---] pick_jobs_to_run() start 06:46:00 [---] pick_jobs_to_run() end 06:46:00 [---] [rr_sim] end 06:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:46:00 [---] [cpu_sched_debug] preliminary job list: 06:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:46:00 [---] [cpu_sched_debug] final job list: 06:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:46:00 [---] [cpu_sched_debug] enforce_run_list: end 06:46:00 [---] choose_project(): 24360.000000 06:46:00 [iThena.Measurements] doing RPC to get work 06:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 584039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411239.09 cpu 2156072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:46:00 [---] Got 0 tasks 06:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:47:00 [---] [rr_sim] doing sim: CPU sched 06:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991419399.22G/439.78G) 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254358.79 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 225435.88 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225435.878930 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028922.91 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 202892.29 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428328.169966 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1826030.62 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 182603.06 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610931.231899 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643427.56 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 164342.76 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775273.987639 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479084.80 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 147908.48 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923182.467805 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331176.32 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 133117.63 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056300.099954 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198058.69 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 119805.87 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176105.968888 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078252.82 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 107825.28 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283931.250929 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970427.54 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 97042.75 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380974.004765 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873384.78 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 87338.48 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468312.483219 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786046.31 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 78604.63 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546917.113826 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707441.68 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 70744.17 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617661.281373 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636697.51 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 63669.75 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681331.032165 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573027.76 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 57302.78 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738633.807879 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515724.98 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 51572.50 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790206.306020 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464152.48 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 46415.25 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836621.554348 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417737.23 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 41773.72 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878395.277843 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375963.51 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 37596.35 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915991.628988 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338367.16 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 33836.72 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949828.345019 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304530.44 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 30453.04 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980281.389446 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274077.40 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 27407.74 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007689.129431 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246669.66 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 24666.97 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032356.095418 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222002.69 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 22200.27 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054556.364806 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199802.42 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 19980.24 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074536.607255 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179822.18 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 17982.22 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092518.825459 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161839.96 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 16184.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108702.821843 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145655.97 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 14565.60 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123268.418588 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131090.37 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 13109.04 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136377.455659 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117981.33 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 11798.13 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148175.589022 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106183.20 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 10618.32 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158793.909050 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95564.88 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 9556.49 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168350.397074 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86008.39 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 8600.84 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176951.236297 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77407.55 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 7740.76 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184691.991596 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69666.80 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 6966.68 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191658.671366 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62700.12 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 6270.01 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197928.683159 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56430.11 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 5643.01 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203571.693773 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50787.10 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 5078.71 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208650.403325 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45708.39 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 4570.84 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213221.241922 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41137.55 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 4113.75 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217334.996660 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37023.79 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3702.38 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221037.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33321.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224637.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29721.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228237.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26121.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231837.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22521.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235437.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18921.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239037.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15321.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242637.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11721.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246237.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8121.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249837.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4521.41 sec) 06:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253437.375923 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 921.41 sec) 06:47:00 [GPUGRID] [rr_sim] 2254358.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991419399.22G/439.78G) 06:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851819.70 06:47:00 [---] pick_jobs_to_run() start 06:47:00 [---] pick_jobs_to_run() end 06:47:00 [---] [rr_sim] end 06:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:47:00 [---] [cpu_sched_debug] preliminary job list: 06:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:47:00 [---] [cpu_sched_debug] final job list: 06:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:47:00 [---] [cpu_sched_debug] enforce_run_list: end 06:47:00 [---] choose_project(): 24420.000000 06:47:00 [iThena.Measurements] doing RPC to get work 06:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411179.09 cpu 2156012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2156012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2156012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:47:00 [---] Got 0 tasks 06:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:48:00 [---] [rr_sim] doing sim: CPU sched 06:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991391808.86G/439.78G) 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254296.05 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 225429.61 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225429.605241 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028866.45 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 202886.64 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428316.249959 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825979.80 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 182597.98 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610914.230204 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643381.82 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 164338.18 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775252.412425 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479043.64 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 147904.36 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923156.776424 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331139.28 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 133113.93 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056270.704023 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1198025.35 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 119802.53 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176073.238862 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078222.81 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 107822.28 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283895.520217 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970400.53 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 97040.05 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380935.573437 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873360.48 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 87336.05 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468271.621335 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786024.43 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 78602.44 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546874.064442 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707421.99 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 70742.20 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617616.263240 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636679.79 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 63667.98 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681284.242157 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573011.81 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 57301.18 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738585.423183 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515710.63 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 51571.06 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790156.486106 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464139.57 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 46413.96 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836570.442737 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417725.61 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 41772.56 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878343.003704 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375953.05 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 37595.30 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915938.308575 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338357.74 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 33835.77 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949774.082959 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304521.97 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 30452.20 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980226.279905 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274069.77 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 27406.98 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007633.257155 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246662.80 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 24666.28 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032299.536681 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221996.52 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 22199.65 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054499.188255 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199796.86 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 19979.69 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074478.874670 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179817.18 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 17981.72 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092460.592445 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161835.46 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 16183.55 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108644.138442 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145651.91 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 14565.19 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123209.329839 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131086.72 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 13108.67 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136318.002096 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117978.05 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 11797.81 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148115.807128 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106180.25 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 10618.02 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158733.831657 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95562.22 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 9556.22 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168290.053732 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86006.00 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 8600.60 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176890.653601 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77405.40 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 7740.54 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184631.193482 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69664.86 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 6966.49 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191597.679375 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62698.37 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 6269.84 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197867.516679 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56428.54 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 5642.85 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203510.370252 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50785.68 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 5078.57 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208588.938469 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45707.11 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 4570.71 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213159.649863 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41136.40 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 4113.64 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217273.290118 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37022.76 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3702.28 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220975.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33320.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224575.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29720.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228175.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26120.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231775.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22520.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235375.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18920.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238975.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15320.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242575.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11720.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246175.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8120.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249775.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4520.49 sec) 06:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253375.566348 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 920.49 sec) 06:48:00 [GPUGRID] [rr_sim] 2254296.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991391808.86G/439.78G) 06:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851816.97 06:48:00 [---] pick_jobs_to_run() start 06:48:00 [---] pick_jobs_to_run() end 06:48:00 [---] [rr_sim] end 06:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:48:00 [---] [cpu_sched_debug] preliminary job list: 06:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:48:00 [---] [cpu_sched_debug] final job list: 06:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:48:00 [---] [cpu_sched_debug] enforce_run_list: end 06:48:00 [---] choose_project(): 24480.000000 06:48:00 [iThena.Measurements] doing RPC to get work 06:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411119.09 cpu 2155952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:48:00 [---] Got 0 tasks 06:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:49:00 [---] [rr_sim] doing sim: CPU sched 06:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991364218.51G/439.78G) 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254233.32 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 225423.33 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225423.331553 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028809.98 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 202881.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428304.329951 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825928.99 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 182592.90 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610897.228509 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643336.09 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 164333.61 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775230.837211 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1479002.48 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 147900.25 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923131.085043 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331102.23 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 133110.22 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056241.308092 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197992.01 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 119799.20 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176040.508836 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078192.81 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 107819.28 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283859.789506 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970373.53 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 97037.35 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380897.142108 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873336.17 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 87333.62 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468230.759451 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786002.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 78600.26 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546831.015059 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707402.30 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 70740.23 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617571.245106 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636662.07 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 63666.21 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681237.452149 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572995.86 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 57299.59 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738537.038487 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515696.28 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 51569.63 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790106.666191 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464126.65 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 46412.66 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836519.331125 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417713.98 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 41771.40 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878290.729566 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375942.59 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 37594.26 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915884.988163 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338348.33 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 33834.83 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949719.820899 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304513.49 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 30451.35 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980171.170363 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274062.15 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 27406.21 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007577.384880 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246655.93 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 24665.59 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032242.977945 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221990.34 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 22199.03 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054442.011703 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199791.30 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 19979.13 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074421.142086 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179812.17 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 17981.22 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092402.359431 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161830.96 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 16183.10 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108585.455041 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145647.86 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 14564.79 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123150.241090 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131083.07 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 13108.31 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136258.548534 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117974.77 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 11797.48 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148056.025234 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106177.29 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 10617.73 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158673.754264 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95559.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 9555.96 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168229.710390 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86003.61 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 8600.36 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176830.070905 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77403.24 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 7740.32 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184570.395367 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69662.92 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 6966.29 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191536.687384 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62696.63 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 6269.66 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197806.350198 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56426.97 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 5642.70 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203449.046732 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50784.27 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 5078.43 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208527.473612 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45705.84 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 4570.58 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213098.057804 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41135.26 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 4113.53 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217211.583577 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37021.73 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3702.17 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220913.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33319.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224513.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29719.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228113.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26119.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231713.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22519.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235313.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18919.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238913.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15319.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242513.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11719.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246113.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8119.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249713.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4519.56 sec) 06:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253313.756772 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 919.56 sec) 06:49:00 [GPUGRID] [rr_sim] 2254233.32: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991364218.51G/439.78G) 06:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851814.23 06:49:00 [---] pick_jobs_to_run() start 06:49:00 [---] pick_jobs_to_run() end 06:49:00 [---] [rr_sim] end 06:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:49:00 [---] [cpu_sched_debug] preliminary job list: 06:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:49:00 [---] [cpu_sched_debug] final job list: 06:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:49:00 [---] [cpu_sched_debug] enforce_run_list: end 06:49:00 [---] choose_project(): 24540.000000 06:49:00 [iThena.Measurements] doing RPC to get work 06:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 411059.09 cpu 2155892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:49:00 [---] Got 0 tasks 06:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:50:00 [---] [rr_sim] doing sim: CPU sched 06:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991336628.15G/439.78G) 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254170.58 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 225417.06 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225417.057865 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028753.52 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 202875.35 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428292.409943 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825878.17 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 182587.82 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610880.226814 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643290.35 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 164329.04 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775209.261998 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478961.32 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 147896.13 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923105.393663 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331065.18 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 133106.52 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056211.912161 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197958.67 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 119795.87 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1176007.778810 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078162.80 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 107816.28 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283824.058794 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970346.52 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 97034.65 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380858.710780 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873311.87 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 87331.19 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468189.897567 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785980.68 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 78598.07 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546787.965675 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707382.61 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 70738.26 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617526.226972 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636644.35 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 63664.44 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681190.662140 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572979.92 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 57297.99 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738488.653791 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515681.92 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 51568.19 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790056.846277 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464113.73 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 46411.37 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836468.219514 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417702.36 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 41770.24 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878238.455428 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375932.12 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 37593.21 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915831.667750 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338338.91 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 33833.89 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949665.558840 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304505.02 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 30450.50 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980116.060821 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274054.52 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 27405.45 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007521.512604 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246649.07 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 24664.91 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032186.419208 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221984.16 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 22198.42 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054384.835152 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199785.74 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 19978.57 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074363.409502 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179807.17 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 17980.72 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092344.126417 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161826.45 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 16182.65 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108526.771640 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145643.81 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 14564.38 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123091.152341 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131079.43 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 13107.94 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136199.094972 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117971.48 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 11797.15 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147996.243340 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106174.34 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 10617.43 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158613.676871 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95556.90 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 9555.69 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168169.367048 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86001.21 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 8600.12 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176769.488209 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77401.09 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 7740.11 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184509.597253 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69660.98 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 6966.10 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191475.695392 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62694.88 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 6269.49 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197745.183718 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56425.39 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 5642.54 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203387.723211 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50782.86 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 5078.29 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208466.008755 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45704.57 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 4570.46 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213036.465744 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41134.11 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 4113.41 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217149.877035 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37020.70 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3702.07 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220851.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33318.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224451.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29718.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228051.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26118.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231651.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22518.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235251.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18918.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238851.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15318.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242451.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11718.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246051.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8118.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249651.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4518.63 sec) 06:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253251.947196 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 918.63 sec) 06:50:00 [GPUGRID] [rr_sim] 2254170.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991336628.15G/439.78G) 06:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851811.49 06:50:00 [---] pick_jobs_to_run() start 06:50:00 [---] pick_jobs_to_run() end 06:50:00 [---] [rr_sim] end 06:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:50:00 [---] [cpu_sched_debug] preliminary job list: 06:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:50:00 [---] [cpu_sched_debug] final job list: 06:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:50:00 [---] [cpu_sched_debug] enforce_run_list: end 06:50:00 [---] choose_project(): 24600.000000 06:50:00 [iThena.Measurements] doing RPC to get work 06:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410999.09 cpu 2155832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:50:00 [---] Got 0 tasks 06:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:51:00 [---] [rr_sim] doing sim: CPU sched 06:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991309037.79G/439.78G) 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254107.84 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 225410.78 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225410.784177 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028697.06 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 202869.71 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428280.489936 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825827.35 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 182582.74 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610863.225119 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643244.62 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 164324.46 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775187.686784 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478920.15 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 147892.02 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923079.702282 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1331028.14 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 133102.81 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056182.516231 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197925.33 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 119792.53 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175975.048784 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078132.79 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 107813.28 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283788.328083 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970319.51 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 97031.95 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380820.279451 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873287.56 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 87328.76 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468149.035683 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785958.81 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 78595.88 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546744.916291 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707362.93 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 70736.29 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617481.208839 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636626.63 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 63662.66 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681143.872132 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572963.97 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 57296.40 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738440.269095 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515667.57 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 51566.76 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790007.026362 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464100.82 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 46410.08 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836417.107903 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417690.73 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 41769.07 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878186.181289 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375921.66 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 37592.17 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915778.347337 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338329.49 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 33832.95 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949611.296780 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304496.54 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 30449.65 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980060.951279 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274046.89 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 27404.69 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007465.640328 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246642.20 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 24664.22 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032129.860472 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221977.98 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 22197.80 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054327.658601 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199780.18 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 19978.02 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074305.676918 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179802.16 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 17980.22 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092285.893403 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161821.95 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 16182.19 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108468.088239 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145639.75 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 14563.98 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123032.063592 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131075.78 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 13107.58 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136139.641410 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117968.20 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 11796.82 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147936.461445 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106171.38 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 10617.14 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158553.599478 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95554.24 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 9555.42 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168109.023706 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85998.82 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 8599.88 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176708.905513 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77398.94 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 7739.89 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184448.799138 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69659.04 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 6965.90 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191414.703401 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62693.14 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 6269.31 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197684.017238 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56423.82 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 5642.38 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203326.399691 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50781.44 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 5078.14 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208404.543898 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45703.30 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 4570.33 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212974.873685 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41132.97 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 4113.30 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217088.170493 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37019.67 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3701.97 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220790.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33317.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224390.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29717.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227990.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26117.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231590.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22517.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235190.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18917.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238790.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15317.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242390.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11717.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245990.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8117.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249590.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4517.70 sec) 06:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253190.137621 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 917.70 sec) 06:51:00 [GPUGRID] [rr_sim] 2254107.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991309037.79G/439.78G) 06:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851808.75 06:51:00 [---] pick_jobs_to_run() start 06:51:00 [---] pick_jobs_to_run() end 06:51:00 [---] [rr_sim] end 06:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:51:00 [---] [cpu_sched_debug] preliminary job list: 06:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:51:00 [---] [cpu_sched_debug] final job list: 06:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:51:00 [---] [cpu_sched_debug] enforce_run_list: end 06:51:00 [---] choose_project(): 24660.000000 06:51:00 [iThena.Measurements] doing RPC to get work 06:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410939.09 cpu 2155772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:51:00 [---] Got 0 tasks 06:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:52:00 [---] [rr_sim] doing sim: CPU sched 06:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991281447.44G/439.78G) 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2254045.10 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 225404.51 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225404.510489 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028640.59 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 202864.06 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428268.569928 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825776.53 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 182577.65 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610846.223424 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643198.88 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 164319.89 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775166.111570 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478878.99 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 147887.90 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923054.010901 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330991.09 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 133099.11 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056153.120300 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197891.98 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 119789.20 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175942.318758 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078102.79 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 107810.28 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283752.597371 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970292.51 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 97029.25 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380781.848122 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873263.26 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 87326.33 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468108.173799 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785936.93 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 78593.69 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546701.866907 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707343.24 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 70734.32 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617436.190705 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636608.91 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 63660.89 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681097.082123 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572948.02 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 57294.80 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738391.884399 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515653.22 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 51565.32 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789957.206448 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464087.90 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 46408.79 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836365.996292 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417679.11 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 41767.91 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878133.907151 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375911.20 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 37591.12 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915725.026924 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338320.08 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 33832.01 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949557.034720 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304488.07 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 30448.81 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980005.841737 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274039.26 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 27403.93 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007409.768052 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246635.34 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 24663.53 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032073.301735 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221971.80 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 22197.18 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054270.482050 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199774.62 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 19977.46 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074247.944334 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179797.16 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 17979.72 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092227.660389 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161817.44 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 16181.74 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108409.404838 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145635.70 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 14563.57 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122972.974843 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131072.13 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 13107.21 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136080.187847 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117964.92 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 11796.49 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147876.679551 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106168.43 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 10616.84 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158493.522084 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95551.58 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 9555.16 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168048.680365 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85996.42 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 8599.64 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176648.322817 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77396.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 7739.68 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184388.001023 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69657.10 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 6965.71 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191353.711410 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62691.39 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 6269.14 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197622.850757 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56422.25 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 5642.23 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203265.076170 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50780.03 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 5078.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208343.079041 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45702.03 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 4570.20 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212913.281626 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41131.82 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 4113.18 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217026.463952 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37018.64 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3701.86 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220728.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33316.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224328.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29716.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227928.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26116.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231528.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22516.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235128.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18916.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238728.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15316.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242328.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11716.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245928.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8116.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249528.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4516.78 sec) 06:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253128.328045 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 916.78 sec) 06:52:00 [GPUGRID] [rr_sim] 2254045.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991281447.44G/439.78G) 06:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851806.02 06:52:00 [---] pick_jobs_to_run() start 06:52:00 [---] pick_jobs_to_run() end 06:52:00 [---] [rr_sim] end 06:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:52:00 [---] [cpu_sched_debug] preliminary job list: 06:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:52:00 [---] [cpu_sched_debug] final job list: 06:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:52:00 [---] [cpu_sched_debug] enforce_run_list: end 06:52:00 [---] choose_project(): 24720.000000 06:52:00 [iThena.Measurements] doing RPC to get work 06:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410879.09 cpu 2155712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:52:00 [---] Got 0 tasks 06:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:53:00 [---] [rr_sim] doing sim: CPU sched 06:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991253857.08G/439.78G) 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253982.37 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 225398.24 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225398.236800 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028584.13 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 202858.41 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428256.649921 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825725.72 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 182572.57 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610829.221729 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643153.15 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 164315.31 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775144.536356 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478837.83 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 147883.78 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923028.319521 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330954.05 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 133095.40 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056123.724369 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197858.64 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 119785.86 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175909.588732 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078072.78 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 107807.28 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283716.866660 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970265.50 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 97026.55 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380743.416794 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873238.95 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 87323.90 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468067.311915 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785915.06 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 78591.51 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546658.817524 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707323.55 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 70732.36 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617391.172572 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636591.20 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 63659.12 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681050.292115 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572932.08 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 57293.21 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738343.499703 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515638.87 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 51563.89 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789907.386533 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464074.98 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 46407.50 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836314.884680 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417667.48 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 41766.75 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878081.633013 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375900.73 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 37590.07 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915671.706512 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338310.66 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 33831.07 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949502.772661 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304479.60 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 30447.96 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979950.732195 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274031.64 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 27403.16 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007353.895776 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246628.47 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 24662.85 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032016.742999 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221965.63 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 22196.56 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054213.305499 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199769.06 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 19976.91 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074190.211749 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179792.16 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 17979.22 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092169.427375 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161812.94 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 16181.29 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108350.721438 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145631.65 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 14563.16 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122913.886094 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131068.48 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 13106.85 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136020.734285 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117961.63 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 11796.16 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147816.897657 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106165.47 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 10616.55 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158433.444691 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95548.92 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 9554.89 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167988.337023 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85994.03 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 8599.40 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176587.740121 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77394.63 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 7739.46 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184327.202909 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69655.17 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 6965.52 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191292.719418 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62689.65 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 6268.96 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197561.684277 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56420.68 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 5642.07 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203203.752649 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50778.62 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 5077.86 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208281.614185 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45700.75 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 4570.08 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212851.689567 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41130.68 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 4113.07 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216964.757410 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37017.61 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3701.76 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220666.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33315.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224266.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29715.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227866.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26115.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231466.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22515.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235066.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18915.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238666.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15315.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242266.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11715.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245866.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8115.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249466.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4515.85 sec) 06:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253066.518469 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 915.85 sec) 06:53:00 [GPUGRID] [rr_sim] 2253982.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991253857.08G/439.78G) 06:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851803.28 06:53:00 [---] pick_jobs_to_run() start 06:53:00 [---] pick_jobs_to_run() end 06:53:00 [---] [rr_sim] end 06:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:53:00 [---] [cpu_sched_debug] preliminary job list: 06:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:53:00 [---] [cpu_sched_debug] final job list: 06:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:53:00 [---] [cpu_sched_debug] enforce_run_list: end 06:53:00 [---] choose_project(): 24780.000000 06:53:00 [iThena.Measurements] doing RPC to get work 06:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410819.09 cpu 2155652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:53:00 [---] Got 0 tasks 06:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:54:00 [---] [rr_sim] doing sim: CPU sched 06:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991226266.72G/439.78G) 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253919.63 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 225391.96 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225391.963112 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028527.67 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 202852.77 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428244.729913 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825674.90 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 182567.49 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610812.220034 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643107.41 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 164310.74 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775122.961142 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478796.67 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 147879.67 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 923002.628140 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330917.00 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 133091.70 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056094.328438 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197825.30 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 119782.53 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175876.858707 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078042.77 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 107804.28 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283681.135948 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970238.50 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 97023.85 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380704.985465 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873214.65 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 87321.46 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1468026.450031 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785893.18 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 78589.32 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546615.768140 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707303.86 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 70730.39 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617346.154438 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636573.48 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 63657.35 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1681003.502106 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572916.13 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 57291.61 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738295.115008 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515624.52 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 51562.45 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789857.566619 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464062.06 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 46406.21 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836263.773069 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417655.86 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 41765.59 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878029.358874 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375890.27 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 37589.03 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915618.386099 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338301.25 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 33830.12 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949448.510601 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304471.12 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 30447.11 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979895.622653 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274024.01 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 27402.40 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007298.023500 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246621.61 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 24662.16 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031960.184262 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221959.45 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 22195.94 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054156.128948 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199763.50 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 19976.35 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074132.479165 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179787.15 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 17978.72 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092111.194361 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161808.44 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 16180.84 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108292.038037 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145627.59 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 14562.76 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122854.797345 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131064.83 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 13106.48 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135961.280723 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117958.35 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 11795.84 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147757.115762 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106162.52 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 10616.25 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158373.367298 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95546.26 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 9554.63 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167927.993681 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85991.64 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 8599.16 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176527.157425 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77392.47 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 7739.25 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184266.404794 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69653.23 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 6965.32 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191231.727427 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62687.90 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 6268.79 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197500.517796 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56419.11 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 5641.91 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203142.429129 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50777.20 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 5077.72 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208220.149328 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45699.48 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 4569.95 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212790.097507 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41129.53 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 4112.95 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216903.050869 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37016.58 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3701.66 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220604.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33314.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224204.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29714.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227804.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26114.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231404.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22514.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235004.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18914.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238604.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15314.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242204.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11714.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245804.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8114.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249404.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4514.92 sec) 06:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2253004.708894 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 914.92 sec) 06:54:00 [GPUGRID] [rr_sim] 2253919.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991226266.72G/439.78G) 06:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851800.54 06:54:00 [---] pick_jobs_to_run() start 06:54:00 [---] pick_jobs_to_run() end 06:54:00 [---] [rr_sim] end 06:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:54:00 [---] [cpu_sched_debug] preliminary job list: 06:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:54:00 [---] [cpu_sched_debug] final job list: 06:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:54:00 [---] [cpu_sched_debug] enforce_run_list: end 06:54:00 [---] choose_project(): 24840.000000 06:54:00 [iThena.Measurements] doing RPC to get work 06:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410759.09 cpu 2155592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:54:00 [---] Got 0 tasks 06:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:55:00 [---] [rr_sim] doing sim: CPU sched 06:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991198676.37G/439.78G) 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253856.89 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 225385.69 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225385.689424 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028471.20 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 202847.12 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428232.809905 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825624.08 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 182562.41 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610795.218339 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643061.68 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 164306.17 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775101.385929 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478755.51 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 147875.55 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922976.936760 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330879.96 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 133088.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056064.932508 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197791.96 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 119779.20 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175844.128681 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1078012.77 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 107801.28 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283645.405236 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970211.49 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 97021.15 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380666.554137 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873190.34 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 87319.03 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467985.588147 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785871.31 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 78587.13 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546572.718756 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707284.18 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 70728.42 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617301.136304 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636555.76 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 63655.58 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680956.712098 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572900.18 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 57290.02 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738246.730312 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515610.16 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 51561.02 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789807.746704 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464049.15 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 46404.91 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836212.661458 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417644.23 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 41764.42 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877977.084736 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375879.81 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 37587.98 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915565.065686 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338291.83 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 33829.18 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949394.248541 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304462.65 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 30446.26 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979840.513111 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274016.38 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 27401.64 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007242.151224 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246614.74 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 24661.47 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031903.625525 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221953.27 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 22195.33 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054098.952397 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199757.94 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 19975.79 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074074.746581 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179782.15 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 17978.21 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092052.961347 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161803.93 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 16180.39 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108233.354636 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145623.54 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 14562.35 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122795.708596 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131061.19 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 13106.12 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135901.827160 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117955.07 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 11795.51 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147697.333868 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106159.56 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 10615.96 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158313.289905 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95543.60 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 9554.36 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167867.650339 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85989.24 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 8598.92 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176466.574729 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77390.32 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 7739.03 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184205.606680 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69651.29 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 6965.13 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191170.735435 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62686.16 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 6268.62 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197439.351316 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56417.54 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 5641.75 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203081.105608 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50775.79 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 5077.58 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208158.684471 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45698.21 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 4569.82 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212728.505448 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41128.39 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 4112.84 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216841.344327 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37015.55 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3701.55 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220542.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33313.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224142.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29713.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227742.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26113.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231342.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22513.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234942.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18913.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238542.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15313.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242142.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11713.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245742.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8113.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249342.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4513.99 sec) 06:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252942.899318 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 913.99 sec) 06:55:00 [GPUGRID] [rr_sim] 2253856.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991198676.37G/439.78G) 06:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851797.81 06:55:00 [---] pick_jobs_to_run() start 06:55:00 [---] pick_jobs_to_run() end 06:55:00 [---] [rr_sim] end 06:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:55:00 [---] [cpu_sched_debug] preliminary job list: 06:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:55:00 [---] [cpu_sched_debug] final job list: 06:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:55:00 [---] [cpu_sched_debug] enforce_run_list: end 06:55:00 [---] choose_project(): 24900.000000 06:55:00 [iThena.Measurements] doing RPC to get work 06:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410699.09 cpu 2155532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:55:00 [---] Got 0 tasks 06:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:56:00 [---] [rr_sim] doing sim: CPU sched 06:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991171086.01G/439.78G) 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253794.16 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 225379.42 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225379.415736 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028414.74 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 202841.47 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428220.889898 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825573.27 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 182557.33 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610778.216644 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1643015.94 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 164301.59 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775079.810715 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478714.35 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 147871.43 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922951.245379 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330842.91 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 133084.29 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056035.536577 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197758.62 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 119775.86 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175811.398655 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077982.76 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 107798.28 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283609.674525 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970184.48 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 97018.45 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380628.122808 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873166.03 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 87316.60 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467944.726263 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785849.43 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 78584.94 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546529.669372 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707264.49 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 70726.45 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617256.118171 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636538.04 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 63653.80 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680909.922089 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572884.24 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 57288.42 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738198.345616 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515595.81 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 51559.58 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789757.926790 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464036.23 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 46403.62 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836161.549847 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417632.61 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 41763.26 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877924.810598 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375869.35 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 37586.93 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915511.745274 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338282.41 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 33828.24 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949339.986482 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304454.17 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 30445.42 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979785.403569 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274008.75 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 27400.88 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007186.278948 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246607.88 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 24660.79 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031847.066789 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221947.09 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 22194.71 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054041.775846 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199752.38 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 19975.24 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074017.013997 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179777.14 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 17977.71 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091994.728333 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161799.43 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 16179.94 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108174.671235 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145619.49 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 14561.95 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122736.619847 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131057.54 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 13105.75 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135842.373598 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117951.78 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 11795.18 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147637.551974 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106156.61 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 10615.66 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158253.212512 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95540.94 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 9554.09 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167807.306997 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85986.85 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 8598.69 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176405.992033 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77388.17 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 7738.82 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184144.808565 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69649.35 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 6964.93 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191109.743444 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62684.41 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 6268.44 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197378.184835 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56415.97 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 5641.60 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203019.782087 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50774.38 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 5077.44 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208097.219614 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45696.94 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 4569.69 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212666.913389 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41127.24 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 4112.72 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216779.637785 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37014.52 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3701.45 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220481.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33313.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224081.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29713.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227681.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26113.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231281.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22513.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234881.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18913.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238481.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15313.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242081.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11713.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245681.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8113.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249281.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4513.07 sec) 06:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252881.089742 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 913.07 sec) 06:56:00 [GPUGRID] [rr_sim] 2253794.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991171086.01G/439.78G) 06:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851795.07 06:56:00 [---] pick_jobs_to_run() start 06:56:00 [---] pick_jobs_to_run() end 06:56:00 [---] [rr_sim] end 06:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:56:00 [---] [cpu_sched_debug] preliminary job list: 06:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:56:00 [---] [cpu_sched_debug] final job list: 06:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:56:00 [---] [cpu_sched_debug] enforce_run_list: end 06:56:00 [---] choose_project(): 24960.000000 06:56:00 [iThena.Measurements] doing RPC to get work 06:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410639.09 cpu 2155472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:56:00 [---] Got 0 tasks 06:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:57:00 [---] [rr_sim] doing sim: CPU sched 06:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991143495.66G/439.78G) 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253731.42 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 225373.14 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225373.142047 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028358.28 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 202835.83 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428208.969890 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825522.45 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 182552.25 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610761.214949 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642970.21 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 164297.02 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775058.235501 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478673.18 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 147867.32 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922925.553998 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330805.87 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 133080.59 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1056006.140646 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197725.28 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 119772.53 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175778.668629 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077952.75 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 107795.28 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283573.943813 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970157.48 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 97015.75 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380589.691479 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873141.73 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 87314.17 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467903.864379 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785827.56 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 78582.76 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546486.619988 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707244.80 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 70724.48 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617211.100037 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636520.32 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 63652.03 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680863.132081 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572868.29 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 57286.83 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738149.960920 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515581.46 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 51558.15 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789708.106876 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464023.31 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 46402.33 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836110.438235 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417620.98 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 41762.10 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877872.536459 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375858.88 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 37585.89 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915458.424861 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338273.00 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 33827.30 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949285.724422 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304445.70 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 30444.57 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979730.294027 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274001.13 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 27400.11 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007130.406672 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246601.01 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 24660.10 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031790.508052 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221940.91 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 22194.09 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053984.599294 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199746.82 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 19974.68 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073959.281412 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179772.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 17977.21 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091936.495319 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161794.93 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 16179.49 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108115.987834 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145615.43 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 14561.54 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122677.531098 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131053.89 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 13105.39 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135782.920036 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117948.50 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 11794.85 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147577.770080 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106153.65 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 10615.37 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158193.135119 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95538.29 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 9553.83 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167746.963655 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85984.46 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 8598.45 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176345.409337 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77386.01 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 7738.60 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184084.010450 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69647.41 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 6964.74 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191048.751453 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62682.67 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 6268.27 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197317.018355 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56414.40 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 5641.44 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202958.458567 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50772.96 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 5077.30 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208035.754758 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45695.67 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 4569.57 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212605.321329 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41126.10 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 4112.61 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216717.931244 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37013.49 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3701.35 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220419.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33312.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224019.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29712.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227619.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26112.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231219.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22512.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234819.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18912.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238419.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15312.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242019.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11712.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245619.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8112.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249219.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4512.14 sec) 06:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252819.280167 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 912.14 sec) 06:57:00 [GPUGRID] [rr_sim] 2253731.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991143495.66G/439.78G) 06:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851792.33 06:57:00 [---] pick_jobs_to_run() start 06:57:00 [---] pick_jobs_to_run() end 06:57:00 [---] [rr_sim] end 06:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:57:00 [---] [cpu_sched_debug] preliminary job list: 06:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:57:00 [---] [cpu_sched_debug] final job list: 06:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:57:00 [---] [cpu_sched_debug] enforce_run_list: end 06:57:00 [---] choose_project(): 25020.000000 06:57:00 [iThena.Measurements] doing RPC to get work 06:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410579.09 cpu 2155412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:57:00 [---] Got 0 tasks 06:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:58:00 [---] [rr_sim] doing sim: CPU sched 06:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991115905.30G/439.78G) 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253668.68 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 225366.87 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225366.868359 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028301.82 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 202830.18 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428197.049883 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825471.63 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 182547.16 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610744.213253 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642924.47 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 164292.45 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775036.660287 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478632.02 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 147863.20 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922899.862618 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330768.82 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 133076.88 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055976.744715 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197691.94 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 119769.19 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175745.938603 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077922.74 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 107792.27 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283538.213102 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970130.47 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 97013.05 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380551.260151 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873117.42 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 87311.74 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467863.002495 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785805.68 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 78580.57 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546443.570605 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707225.11 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 70722.51 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617166.081903 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636502.60 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 63650.26 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680816.342072 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572852.34 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 57285.23 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738101.576224 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515567.11 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 51556.71 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789658.286961 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464010.40 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 46401.04 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836059.326624 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417609.36 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 41760.94 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877820.262321 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375848.42 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 37584.84 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915405.104448 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338263.58 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 33826.36 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949231.462362 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304437.22 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 30443.72 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979675.184485 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273993.50 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 27399.35 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007074.534396 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246594.15 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 24659.41 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031733.949316 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221934.73 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 22193.47 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053927.422743 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199741.26 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 19974.13 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073901.548828 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179767.13 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 17976.71 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091878.262305 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161790.42 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 16179.04 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108057.304433 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145611.38 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 14561.14 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122618.442349 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131050.24 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 13105.02 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135723.466474 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117945.22 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 11794.52 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147517.988185 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106150.70 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 10615.07 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158133.057726 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95535.63 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 9553.56 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167686.620313 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85982.06 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 8598.21 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176284.826641 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77383.86 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 7738.39 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184023.212336 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69645.47 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 6964.55 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190987.759461 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62680.92 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 6268.09 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197255.851874 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56412.83 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 5641.28 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202897.135046 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50771.55 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 5077.15 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207974.289901 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45694.39 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 4569.44 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212543.729270 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41124.95 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 4112.50 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216656.224702 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37012.46 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3701.25 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220357.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33311.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223957.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29711.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227557.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26111.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231157.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22511.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234757.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18911.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238357.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15311.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241957.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11711.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245557.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8111.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249157.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4511.21 sec) 06:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252757.470591 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 911.21 sec) 06:58:00 [GPUGRID] [rr_sim] 2253668.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991115905.30G/439.78G) 06:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851789.60 06:58:00 [---] pick_jobs_to_run() start 06:58:00 [---] pick_jobs_to_run() end 06:58:00 [---] [rr_sim] end 06:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:58:00 [---] [cpu_sched_debug] preliminary job list: 06:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:58:00 [---] [cpu_sched_debug] final job list: 06:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:58:00 [---] [cpu_sched_debug] enforce_run_list: end 06:58:00 [---] choose_project(): 25080.000000 06:58:00 [iThena.Measurements] doing RPC to get work 06:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410519.09 cpu 2155352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:58:00 [---] Got 0 tasks 06:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 06:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 06:59:00 [---] [rr_sim] doing sim: CPU sched 06:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991088314.94G/439.78G) 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253605.95 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 225360.59 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225360.594671 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028245.35 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 202824.54 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428185.129875 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825420.82 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 182542.08 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610727.211558 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642878.74 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 164287.87 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 775015.085074 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478590.86 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 147859.09 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922874.171237 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330731.78 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 133073.18 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055947.348784 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197658.60 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 119765.86 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175713.208577 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077892.74 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 107789.27 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283502.482390 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970103.46 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 97010.35 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380512.828822 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873093.12 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 87309.31 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467822.140611 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785783.81 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 78578.38 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546400.521221 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707205.43 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 70720.54 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617121.063770 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636484.88 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 63648.49 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680769.552064 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572836.39 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 57283.64 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738053.191528 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515552.76 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 51555.28 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789608.467047 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463997.48 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 46399.75 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836008.215013 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417597.73 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 41759.77 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877767.988183 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375837.96 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 37583.80 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915351.784035 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338254.16 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 33825.42 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949177.200303 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304428.75 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 30442.87 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979620.074944 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273985.87 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 27398.59 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007018.662120 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246587.28 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 24658.73 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031677.390579 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221928.56 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 22192.86 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053870.246192 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199735.70 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 19973.57 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073843.816244 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179762.13 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 17976.21 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091820.029291 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161785.92 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 16178.59 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107998.621033 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145607.33 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 14560.73 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122559.353600 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131046.59 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 13104.66 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135664.012911 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117941.93 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 11794.19 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147458.206291 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106147.74 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 10614.77 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158072.980333 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95532.97 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 9553.30 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167626.276971 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85979.67 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 8597.97 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176224.243945 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77381.70 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 7738.17 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183962.414221 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69643.53 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 6964.35 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190926.767470 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62679.18 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 6267.92 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197194.685394 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56411.26 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 5641.13 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202835.811526 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50770.14 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 5077.01 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207912.825044 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45693.12 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 4569.31 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212482.137211 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41123.81 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 4112.38 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216594.518161 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37011.43 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3701.14 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220295.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33310.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223895.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29710.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227495.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26110.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231095.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22510.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234695.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18910.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238295.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15310.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241895.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11710.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245495.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8110.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249095.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4510.29 sec) 06:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252695.661016 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 910.29 sec) 06:59:00 [GPUGRID] [rr_sim] 2253605.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991088314.94G/439.78G) 06:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851786.86 06:59:00 [---] pick_jobs_to_run() start 06:59:00 [---] pick_jobs_to_run() end 06:59:00 [---] [rr_sim] end 06:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 06:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 06:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 06:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 06:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 06:59:00 [---] [cpu_sched_debug] preliminary job list: 06:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:59:00 [---] [cpu_sched_debug] final job list: 06:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 06:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 06:59:00 [---] [cpu_sched_debug] enforce_run_list: end 06:59:00 [---] choose_project(): 25140.000000 06:59:00 [iThena.Measurements] doing RPC to get work 06:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410459.09 cpu 2155292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 06:59:00 [---] Got 0 tasks 07:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:00:00 [---] [rr_sim] doing sim: CPU sched 07:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991060724.59G/439.78G) 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253543.21 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 225354.32 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225354.320983 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028188.89 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 202818.89 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428173.209867 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825370.00 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 182537.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610710.209863 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642833.00 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 164283.30 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774993.509860 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478549.70 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 147854.97 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922848.479857 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330694.73 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 133069.47 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055917.952854 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197625.26 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 119762.53 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175680.478551 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077862.73 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 107786.27 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283466.751679 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970076.46 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 97007.65 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380474.397494 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873068.81 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 87306.88 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467781.278727 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785761.93 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 78576.19 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546357.471837 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707185.74 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 70718.57 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617076.045636 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636467.16 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 63646.72 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680722.762055 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572820.45 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 57282.04 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1738004.806833 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515538.40 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 51553.84 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789558.647132 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463984.56 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 46398.46 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835957.103402 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417586.11 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 41758.61 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877715.714044 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375827.50 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 37582.75 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915298.463623 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338244.75 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 33824.47 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949122.938243 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304420.27 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 30442.03 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979564.965402 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273978.24 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 27397.82 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006962.789844 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246580.42 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 24658.04 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031620.831843 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221922.38 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 22192.24 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053813.069641 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199730.14 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 19973.01 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073786.083660 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179757.13 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 17975.71 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091761.796277 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161781.41 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 16178.14 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107939.937632 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145603.27 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 14560.33 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122500.264851 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131042.94 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 13104.29 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135604.559349 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117938.65 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 11793.87 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147398.424397 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106144.79 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 10614.48 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158012.902940 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95530.31 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 9553.03 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167565.933629 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85977.28 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 8597.73 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176163.661249 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77379.55 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 7737.95 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183901.616107 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69641.59 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 6964.16 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190865.775479 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62677.43 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 6267.74 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197133.518914 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56409.69 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 5640.97 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202774.488005 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50768.72 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 5076.87 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207851.360187 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45691.85 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 4569.18 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212420.545151 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41122.66 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 4112.27 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216532.811619 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37010.40 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3701.04 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220233.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33309.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223833.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29709.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227433.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26109.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231033.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22509.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234633.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18909.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238233.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15309.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241833.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11709.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245433.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8109.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249033.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4509.36 sec) 07:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252633.851440 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 909.36 sec) 07:00:00 [GPUGRID] [rr_sim] 2253543.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991060724.59G/439.78G) 07:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851784.12 07:00:00 [---] pick_jobs_to_run() start 07:00:00 [---] pick_jobs_to_run() end 07:00:00 [---] [rr_sim] end 07:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:00:00 [---] [cpu_sched_debug] preliminary job list: 07:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:00:00 [---] [cpu_sched_debug] final job list: 07:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:00:00 [---] [cpu_sched_debug] enforce_run_list: end 07:00:00 [---] choose_project(): 25200.000000 07:00:00 [iThena.Measurements] doing RPC to get work 07:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 583199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410399.09 cpu 2155232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:00:00 [---] Got 0 tasks 07:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:04:00 [---] [rr_sim] doing sim: CPU sched 07:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991033134.23G/439.78G) 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253480.47 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 225348.05 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225348.047295 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028132.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 202813.24 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428161.289860 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825319.18 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 182531.92 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610693.208168 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642787.26 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 164278.73 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774971.934646 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478508.54 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 147850.85 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922822.788476 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330657.68 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 133065.77 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055888.556923 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197591.92 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 119759.19 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175647.748525 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077832.72 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 107783.27 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283431.020967 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970049.45 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 97004.95 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380435.966165 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873044.51 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 87304.45 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467740.416843 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785740.06 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 78574.01 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546314.422453 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707166.05 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 70716.61 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617031.027503 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636449.45 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 63644.94 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680675.972047 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572804.50 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 57280.45 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737956.422137 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515524.05 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 51552.41 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789508.827218 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463971.65 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 46397.16 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835905.991790 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417574.48 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 41757.45 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877663.439906 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375817.03 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 37581.70 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915245.143210 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338235.33 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 33823.53 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949068.676184 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304411.80 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 30441.18 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979509.855860 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273970.62 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 27397.06 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006906.917568 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246573.56 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 24657.36 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031564.273106 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221916.20 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 22191.62 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053755.893090 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199724.58 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 19972.46 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073728.351076 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179752.12 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 17975.21 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091703.563263 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161776.91 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 16177.69 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107881.254231 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145599.22 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 14559.92 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122441.176102 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131039.30 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 13103.93 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135545.105787 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117935.37 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 11793.54 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147338.642503 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106141.83 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 10614.18 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157952.825547 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95527.65 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 9552.76 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167505.590287 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85974.88 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 8597.49 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176103.078553 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77377.39 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 7737.74 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183840.817992 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69639.65 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 6963.97 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190804.783487 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62675.69 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 6267.57 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197072.352433 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56408.12 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 5640.81 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202713.164484 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50767.31 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 5076.73 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207789.895331 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45690.58 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 4569.06 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212358.953092 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41121.52 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 4112.15 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216471.105077 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37009.37 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3700.94 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220172.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33308.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223772.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29708.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227372.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26108.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230972.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22508.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234572.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18908.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238172.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15308.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241772.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11708.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245372.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8108.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248972.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4508.43 sec) 07:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252572.041864 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 908.43 sec) 07:04:00 [GPUGRID] [rr_sim] 2253480.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991033134.23G/439.78G) 07:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851961.39 07:04:00 [---] pick_jobs_to_run() start 07:04:00 [---] pick_jobs_to_run() end 07:04:00 [---] [rr_sim] end 07:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:04:00 [---] [cpu_sched_debug] preliminary job list: 07:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:04:00 [---] [cpu_sched_debug] final job list: 07:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:04:00 [---] [cpu_sched_debug] enforce_run_list: end 07:04:00 [---] choose_project(): 25440.000000 07:04:00 [iThena.Measurements] doing RPC to get work 07:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410159.09 cpu 2155172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:04:00 [---] Got 0 tasks 07:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:05:00 [---] [rr_sim] doing sim: CPU sched 07:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991033134.23G/439.78G) 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253480.47 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 225348.05 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225348.047295 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028132.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 202813.24 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428161.289860 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825319.18 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 182531.92 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610693.208168 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642787.26 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 164278.73 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774971.934646 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478508.54 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 147850.85 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922822.788476 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330657.68 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 133065.77 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055888.556923 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197591.92 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 119759.19 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175647.748525 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077832.72 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 107783.27 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283431.020967 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970049.45 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 97004.95 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380435.966165 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873044.51 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 87304.45 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467740.416843 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785740.06 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 78574.01 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546314.422453 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707166.05 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 70716.61 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1617031.027503 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636449.45 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 63644.94 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680675.972047 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572804.50 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 57280.45 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737956.422137 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515524.05 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 51552.41 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789508.827218 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463971.65 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 46397.16 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835905.991790 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417574.48 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 41757.45 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877663.439906 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375817.03 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 37581.70 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915245.143210 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338235.33 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 33823.53 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949068.676184 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304411.80 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 30441.18 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979509.855860 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273970.62 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 27397.06 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006906.917568 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246573.56 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 24657.36 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031564.273106 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221916.20 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 22191.62 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053755.893090 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199724.58 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 19972.46 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073728.351076 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179752.12 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 17975.21 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091703.563263 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161776.91 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 16177.69 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107881.254231 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145599.22 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 14559.92 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122441.176102 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131039.30 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 13103.93 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135545.105787 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117935.37 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 11793.54 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147338.642503 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106141.83 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 10614.18 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157952.825547 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95527.65 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 9552.76 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167505.590287 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85974.88 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 8597.49 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176103.078553 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77377.39 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 7737.74 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183840.817992 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69639.65 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 6963.97 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190804.783487 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62675.69 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 6267.57 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197072.352433 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56408.12 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 5640.81 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202713.164484 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50767.31 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 5076.73 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207789.895331 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45690.58 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 4569.06 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212358.953092 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41121.52 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 4112.15 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216471.105077 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37009.37 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3700.94 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220172.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33308.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223772.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29708.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227372.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26108.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230972.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22508.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234572.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18908.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238172.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15308.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241772.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11708.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245372.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8108.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248972.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4508.43 sec) 07:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252572.041864 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 908.43 sec) 07:05:00 [GPUGRID] [rr_sim] 2253480.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991033134.23G/439.78G) 07:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852021.39 07:05:00 [---] pick_jobs_to_run() start 07:05:00 [---] pick_jobs_to_run() end 07:05:00 [---] [rr_sim] end 07:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:05:00 [---] [cpu_sched_debug] preliminary job list: 07:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:05:00 [---] [cpu_sched_debug] final job list: 07:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:05:00 [---] [cpu_sched_debug] enforce_run_list: end 07:05:00 [---] choose_project(): 25500.000000 07:05:00 [iThena.Measurements] doing RPC to get work 07:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410099.09 cpu 2155172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:05:00 [---] Got 0 tasks 07:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:06:00 [---] [rr_sim] doing sim: CPU sched 07:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991005543.88G/439.78G) 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253417.74 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 225341.77 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225341.773606 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028075.96 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 202807.60 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428149.369852 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825268.37 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 182526.84 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610676.206473 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642741.53 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 164274.15 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774950.359432 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478467.38 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 147846.74 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922797.097095 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330620.64 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 133062.06 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055859.160992 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197558.58 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 119755.86 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175615.018499 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077802.72 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 107780.27 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283395.290256 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970022.45 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 97002.24 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380397.534836 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873020.20 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 87302.02 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467699.554959 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785718.18 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 78571.82 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546271.373070 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707146.36 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 70714.64 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616986.009369 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636431.73 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 63643.17 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680629.182038 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572788.55 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 57278.86 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737908.037441 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515509.70 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 51550.97 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789459.007303 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463958.73 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 46395.87 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835854.880179 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417562.86 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 41756.29 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877611.165768 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375806.57 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 37580.66 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915191.822797 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338225.91 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 33822.59 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949014.414124 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304403.32 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 30440.33 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979454.746318 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273962.99 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 27396.30 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006851.045292 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246566.69 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 24656.67 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031507.714369 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221910.02 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 22191.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053698.716539 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199719.02 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 19971.90 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073670.618491 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179747.12 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 17974.71 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091645.330249 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161772.41 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 16177.24 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107822.570830 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145595.17 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 14559.52 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122382.087353 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131035.65 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 13103.56 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135485.652224 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117932.08 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 11793.21 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147278.860608 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106138.88 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 10613.89 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157892.748154 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95524.99 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 9552.50 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167445.246945 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85972.49 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 8597.25 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176042.495857 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77375.24 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 7737.52 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183780.019877 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69637.72 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 6963.77 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190743.791496 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62673.94 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 6267.39 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197011.185953 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56406.55 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 5640.66 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202651.840964 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50765.90 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 5076.59 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207728.430474 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45689.31 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 4568.93 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212297.361033 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41120.38 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 4112.04 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216409.398536 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37008.34 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3700.83 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220110.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33307.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223710.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29707.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227310.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26107.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230910.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22507.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234510.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18907.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238110.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15307.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241710.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11707.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245310.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8107.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248910.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4507.50 sec) 07:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252510.232289 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 907.50 sec) 07:06:00 [GPUGRID] [rr_sim] 2253417.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991005543.88G/439.78G) 07:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852018.65 07:06:00 [---] pick_jobs_to_run() start 07:06:00 [---] pick_jobs_to_run() end 07:06:00 [---] [rr_sim] end 07:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:06:00 [---] [cpu_sched_debug] preliminary job list: 07:06:00 [---] [cpu_sched_debug] final job list: 07:06:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 07:06:00 [---] [work_fetch] Request work fetch: CPUs idle 07:06:00 [---] [cpu_sched_debug] enforce_run_list: end 07:06:00 [---] choose_project(): 25560.000000 07:06:00 [iThena.Measurements] doing RPC to get work 07:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 410039.09 cpu 2155112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:06:00 [---] Got 0 tasks 07:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:07:00 [---] [rr_sim] doing sim: CPU sched 07:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (991005543.88G/439.78G) 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253417.74 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 225341.77 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225341.773606 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028075.96 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 202807.60 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428149.369852 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825268.37 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 182526.84 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610676.206473 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642741.53 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 164274.15 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774950.359432 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478467.38 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 147846.74 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922797.097095 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330620.64 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 133062.06 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055859.160992 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197558.58 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 119755.86 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175615.018499 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077802.72 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 107780.27 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283395.290256 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 970022.45 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 97002.24 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380397.534836 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873020.20 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 87302.02 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467699.554959 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785718.18 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 78571.82 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546271.373070 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707146.36 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 70714.64 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616986.009369 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636431.73 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 63643.17 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680629.182038 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572788.55 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 57278.86 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737908.037441 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515509.70 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 51550.97 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789459.007303 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463958.73 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 46395.87 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835854.880179 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417562.86 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 41756.29 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877611.165768 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375806.57 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 37580.66 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915191.822797 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338225.91 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 33822.59 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949014.414124 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304403.32 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 30440.33 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979454.746318 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273962.99 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 27396.30 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006851.045292 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246566.69 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 24656.67 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031507.714369 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221910.02 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 22191.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053698.716539 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199719.02 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 19971.90 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073670.618491 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179747.12 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 17974.71 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091645.330249 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161772.41 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 16177.24 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107822.570830 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145595.17 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 14559.52 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122382.087353 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131035.65 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 13103.56 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135485.652224 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117932.08 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 11793.21 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147278.860608 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106138.88 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 10613.89 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157892.748154 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95524.99 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 9552.50 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167445.246945 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85972.49 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 8597.25 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176042.495857 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77375.24 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 7737.52 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183780.019877 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69637.72 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 6963.77 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190743.791496 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62673.94 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 6267.39 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197011.185953 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56406.55 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 5640.66 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202651.840964 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50765.90 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 5076.59 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207728.430474 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45689.31 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 4568.93 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212297.361033 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41120.38 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 4112.04 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216409.398536 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37008.34 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3700.83 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220110.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33307.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223710.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29707.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227310.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26107.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230910.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22507.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234510.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18907.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238110.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15307.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241710.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11707.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245310.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8107.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248910.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4507.50 sec) 07:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252510.232289 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 907.50 sec) 07:07:00 [GPUGRID] [rr_sim] 2253417.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (991005543.88G/439.78G) 07:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852078.65 07:07:00 [---] pick_jobs_to_run() start 07:07:00 [---] pick_jobs_to_run() end 07:07:00 [---] [rr_sim] end 07:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:07:00 [---] [cpu_sched_debug] preliminary job list: 07:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:07:00 [---] [cpu_sched_debug] final job list: 07:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:07:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 991005543.88G 07:07:00 [---] [cpu_sched_debug] enforce_run_list: end 07:07:00 [---] choose_project(): 25620.000000 07:07:00 [iThena.Measurements] doing RPC to get work 07:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409979.09 cpu 2155112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:07:00 [---] Got 0 tasks 07:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:08:00 [---] [rr_sim] doing sim: CPU sched 07:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990977953.52G/439.78G) 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253355.00 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 225335.50 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225335.499918 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2028019.50 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 202801.95 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428137.449844 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825217.55 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 182521.75 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610659.204778 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642695.79 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 164269.58 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774928.784218 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478426.21 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 147842.62 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922771.405715 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330583.59 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 133058.36 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055829.765061 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197525.23 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 119752.52 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175582.288473 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077772.71 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 107777.27 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283359.559544 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969995.44 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 96999.54 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380359.103508 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872995.90 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 87299.59 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467658.693075 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785696.31 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 78569.63 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546228.323686 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707126.68 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 70712.67 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616940.991235 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636414.01 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 63641.40 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680582.392030 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572772.61 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 57277.26 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737859.652745 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515495.35 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 51549.53 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789409.187389 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463945.81 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 46394.58 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835803.768568 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417551.23 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 41755.12 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877558.891629 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375796.11 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 37579.61 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915138.502385 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338216.50 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 33821.65 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948960.152064 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304394.85 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 30439.48 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979399.636776 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273955.36 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 27395.54 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006795.173016 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246559.83 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 24655.98 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031451.155633 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221903.84 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 22190.38 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053641.539988 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199713.46 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 19971.35 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073612.885907 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179742.11 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 17974.21 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091587.097235 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161767.90 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 16176.79 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107763.887429 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145591.11 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 14559.11 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122322.998604 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131032.00 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 13103.20 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135426.198662 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117928.80 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 11792.88 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147219.078714 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106135.92 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 10613.59 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157832.670761 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95522.33 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 9552.23 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167384.903603 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85970.10 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 8597.01 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175981.913161 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77373.09 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 7737.31 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183719.221763 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69635.78 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 6963.58 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190682.799505 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62672.20 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 6267.22 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196950.019472 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56404.98 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 5640.50 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202590.517443 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50764.48 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 5076.45 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207666.965617 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45688.03 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 4568.80 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212235.768973 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41119.23 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 4111.92 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216347.691994 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37007.31 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3700.73 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220048.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33306.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223648.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29706.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227248.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26106.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230848.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22506.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234448.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18906.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238048.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15306.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241648.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11706.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245248.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8106.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248848.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4506.58 sec) 07:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252448.422713 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 906.58 sec) 07:08:00 [GPUGRID] [rr_sim] 2253355.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990977953.52G/439.78G) 07:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852075.91 07:08:00 [---] pick_jobs_to_run() start 07:08:00 [---] pick_jobs_to_run() end 07:08:00 [---] [rr_sim] end 07:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:08:00 [---] [cpu_sched_debug] preliminary job list: 07:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 07:08:00 [---] [cpu_sched_debug] final job list: 07:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:08:00 [---] [cpu_sched_debug] enforce_run_list: end 07:08:00 [---] choose_project(): 25680.000000 07:08:00 [iThena.Measurements] doing RPC to get work 07:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409919.09 cpu 2155052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2155052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2155052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:08:00 [---] Got 0 tasks 07:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:09:00 [---] [rr_sim] doing sim: CPU sched 07:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990950363.16G/439.78G) 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253292.26 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 225329.23 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225329.226230 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027963.04 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 202796.30 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428125.529837 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825166.73 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 182516.67 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610642.203083 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642650.06 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 164265.01 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774907.209005 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478385.05 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 147838.51 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922745.714334 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330546.55 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 133054.65 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055800.369131 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197491.89 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 119749.19 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175549.558447 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077742.70 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 107774.27 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283323.828833 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969968.43 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 96996.84 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380320.672179 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872971.59 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 87297.16 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467617.831191 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785674.43 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 78567.44 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546185.274302 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707106.99 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 70710.70 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616895.973102 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636396.29 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 63639.63 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680535.602022 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572756.66 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 57275.67 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737811.268049 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515480.99 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 51548.10 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789359.367474 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463932.89 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 46393.29 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835752.656957 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417539.61 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 41753.96 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877506.617491 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375785.64 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 37578.56 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915085.181972 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338207.08 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 33820.71 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948905.890005 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304386.37 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 30438.64 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979344.527234 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273947.74 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 27394.77 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006739.300741 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246552.96 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 24655.30 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031394.596896 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221897.67 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 22189.77 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053584.363437 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199707.90 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 19970.79 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073555.153323 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179737.11 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 17973.71 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091528.864221 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161763.40 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 16176.34 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107705.204028 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145587.06 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 14558.71 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122263.909855 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131028.35 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 13102.84 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135366.745100 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117925.52 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 11792.55 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147159.296820 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106132.97 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 10613.30 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157772.593368 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95519.67 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 9551.97 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167324.560261 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85967.70 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 8596.77 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175921.330465 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77370.93 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 7737.09 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183658.423648 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69633.84 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 6963.38 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190621.807513 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62670.45 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 6267.05 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196888.852992 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56403.41 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 5640.34 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202529.193923 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50763.07 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 5076.31 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207605.500760 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45686.76 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 4568.68 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212174.176914 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41118.09 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 4111.81 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216285.985453 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37006.28 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3700.63 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219986.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33305.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223586.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29705.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227186.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26105.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230786.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22505.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234386.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18905.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237986.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15305.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241586.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11705.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245186.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8105.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248786.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4505.65 sec) 07:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252386.613137 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 905.65 sec) 07:09:00 [GPUGRID] [rr_sim] 2253292.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990950363.16G/439.78G) 07:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852073.18 07:09:00 [---] pick_jobs_to_run() start 07:09:00 [---] pick_jobs_to_run() end 07:09:00 [---] [rr_sim] end 07:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:09:00 [---] [cpu_sched_debug] preliminary job list: 07:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:09:00 [---] [cpu_sched_debug] final job list: 07:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:09:00 [---] [cpu_sched_debug] enforce_run_list: end 07:09:00 [---] choose_project(): 25740.000000 07:09:00 [iThena.Measurements] doing RPC to get work 07:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409859.09 cpu 2154992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:09:00 [---] Got 0 tasks 07:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:10:00 [---] [rr_sim] doing sim: CPU sched 07:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990922772.81G/439.78G) 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253229.53 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 225322.95 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225322.952542 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027906.57 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 202790.66 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428113.609829 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825115.92 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 182511.59 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610625.201388 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642604.32 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 164260.43 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774885.633791 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478343.89 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 147834.39 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922720.022954 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330509.50 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 133050.95 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055770.973200 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197458.55 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 119745.86 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175516.828422 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077712.70 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 107771.27 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283288.098121 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969941.43 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 96994.14 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380282.240851 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872947.28 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 87294.73 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467576.969307 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785652.56 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 78565.26 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546142.224918 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707087.30 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 70708.73 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616850.954968 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636378.57 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 63637.86 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680488.812013 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572740.71 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 57274.07 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737762.883353 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515466.64 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 51546.66 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789309.547560 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463919.98 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 46392.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835701.545345 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417527.98 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 41752.80 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877454.343353 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375775.18 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 37577.52 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915031.861559 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338197.66 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 33819.77 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948851.627945 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304377.90 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 30437.79 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979289.417692 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273940.11 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 27394.01 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006683.428465 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246546.10 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 24654.61 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031338.038160 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221891.49 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 22189.15 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053527.186886 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199702.34 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 19970.23 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073497.420739 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179732.10 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 17973.21 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091470.631206 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161758.89 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 16175.89 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107646.520628 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145583.00 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 14558.30 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122204.821106 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131024.70 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 13102.47 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135307.291538 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117922.23 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 11792.22 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147099.514925 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106130.01 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 10613.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157712.515975 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95517.01 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 9551.70 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167264.216919 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85965.31 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 8596.53 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175860.747769 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77368.78 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 7736.88 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183597.625533 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69631.90 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 6963.19 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190560.815522 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62668.71 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 6266.87 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196827.686511 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56401.84 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 5640.18 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202467.870402 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50761.66 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 5076.17 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207544.035903 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45685.49 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 4568.55 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212112.584855 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41116.94 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 4111.69 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216224.278911 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37005.25 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3700.52 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219924.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33304.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223524.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29704.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227124.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26104.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230724.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22504.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234324.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18904.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237924.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15304.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241524.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11704.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245124.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8104.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248724.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4504.72 sec) 07:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252324.803562 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 904.72 sec) 07:10:00 [GPUGRID] [rr_sim] 2253229.53: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990922772.81G/439.78G) 07:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852070.44 07:10:00 [---] pick_jobs_to_run() start 07:10:00 [---] pick_jobs_to_run() end 07:10:00 [---] [rr_sim] end 07:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:10:00 [---] [cpu_sched_debug] preliminary job list: 07:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:10:00 [---] [cpu_sched_debug] final job list: 07:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:10:00 [---] [cpu_sched_debug] enforce_run_list: end 07:10:00 [---] choose_project(): 25800.000000 07:10:00 [iThena.Measurements] doing RPC to get work 07:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409799.09 cpu 2154932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:10:00 [---] Got 0 tasks 07:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:11:00 [---] [rr_sim] doing sim: CPU sched 07:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990895182.45G/439.78G) 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253166.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 225316.68 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225316.678853 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027850.11 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 202785.01 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428101.689822 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825065.10 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 182506.51 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610608.199693 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642558.59 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 164255.86 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774864.058577 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478302.73 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 147830.27 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922694.331573 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330472.46 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 133047.25 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055741.577269 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197425.21 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 119742.52 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175484.098396 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077682.69 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 107768.27 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283252.367410 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969914.42 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 96991.44 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380243.809522 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872922.98 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 87292.30 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467536.107423 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785630.68 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 78563.07 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546099.175535 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707067.61 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 70706.76 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616805.936835 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636360.85 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 63636.09 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680442.022005 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572724.77 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 57272.48 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737714.498658 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515452.29 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 51545.23 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789259.727645 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463907.06 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 46390.71 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835650.433734 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417516.35 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 41751.64 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877402.069214 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375764.72 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 37576.47 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914978.541146 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338188.25 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 33818.82 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948797.365885 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304369.42 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 30436.94 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979234.308150 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273932.48 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 27393.25 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006627.556189 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246539.23 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 24653.92 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031281.479423 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221885.31 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 22188.53 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053470.010334 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199696.78 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 19969.68 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073439.688154 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179727.10 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 17972.71 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091412.398192 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161754.39 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 16175.44 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107587.837227 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145578.95 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 14557.90 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122145.732358 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131021.06 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 13102.11 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135247.837975 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117918.95 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 11791.90 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147039.733031 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106127.06 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 10612.71 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157652.438582 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95514.35 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 9551.43 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167203.873577 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85962.91 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 8596.29 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175800.165073 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77366.62 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 7736.66 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183536.827419 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69629.96 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 6963.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190499.823530 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62666.97 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 6266.70 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196766.520031 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56400.27 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 5640.03 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202406.546881 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50760.24 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 5076.02 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207482.571047 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45684.22 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 4568.42 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212050.992795 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41115.80 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 4111.58 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216162.572369 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37004.22 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3700.42 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219862.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33303.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223462.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29703.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227062.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26103.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230662.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22503.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234262.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18903.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237862.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15303.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241462.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11703.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245062.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8103.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248662.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4503.79 sec) 07:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252262.993986 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 903.79 sec) 07:11:00 [GPUGRID] [rr_sim] 2253166.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990895182.45G/439.78G) 07:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852067.70 07:11:00 [---] pick_jobs_to_run() start 07:11:00 [---] pick_jobs_to_run() end 07:11:00 [---] [rr_sim] end 07:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:11:00 [---] [cpu_sched_debug] preliminary job list: 07:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:11:00 [---] [cpu_sched_debug] final job list: 07:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:11:00 [---] [cpu_sched_debug] enforce_run_list: end 07:11:00 [---] choose_project(): 25860.000000 07:11:00 [iThena.Measurements] doing RPC to get work 07:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409739.09 cpu 2154872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:11:00 [---] Got 0 tasks 07:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:12:00 [---] [rr_sim] doing sim: CPU sched 07:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990867592.09G/439.78G) 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253104.05 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 225310.41 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225310.405165 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027793.65 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 202779.36 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428089.769814 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1825014.28 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 182501.43 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610591.197998 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642512.85 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 164251.29 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774842.483363 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478261.57 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 147826.16 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922668.640192 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330435.41 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 133043.54 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055712.181338 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197391.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 119739.19 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175451.368370 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077652.68 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 107765.27 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283216.636698 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969887.41 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 96988.74 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380205.378193 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872898.67 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 87289.87 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467495.245539 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785608.81 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 78560.88 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546056.126151 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707047.93 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 70704.79 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616760.918701 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636343.13 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 63634.31 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680395.231996 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572708.82 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 57270.88 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737666.113962 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515437.94 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 51543.79 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789209.907731 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463894.14 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 46389.41 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835599.322123 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417504.73 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 41750.47 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877349.795076 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375754.26 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 37575.43 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914925.220734 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338178.83 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 33817.88 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948743.103826 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304360.95 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 30436.09 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979179.198608 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273924.85 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 27392.49 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006571.683913 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246532.37 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 24653.24 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031224.920687 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221879.13 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 22187.91 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053412.833783 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199691.22 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 19969.12 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073381.955570 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179722.10 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 17972.21 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091354.165178 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161749.89 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 16174.99 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107529.153826 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145574.90 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 14557.49 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122086.643609 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131017.41 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 13101.74 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135188.384413 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117915.67 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 11791.57 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146979.951137 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106124.10 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 10612.41 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157592.361188 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95511.69 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 9551.17 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167143.530235 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85960.52 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 8596.05 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175739.582377 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77364.47 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 7736.45 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183476.029304 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69628.02 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 6962.80 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190438.831539 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62665.22 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 6266.52 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196705.353550 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56398.70 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 5639.87 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202345.223361 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50758.83 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 5075.88 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207421.106190 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45682.95 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 4568.29 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211989.400736 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41114.65 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 4111.47 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216100.865828 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37003.19 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3700.32 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219801.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33302.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223401.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29702.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227001.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26102.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230601.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22502.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234201.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18902.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237801.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15302.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241401.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11702.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245001.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8102.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248601.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4502.87 sec) 07:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252201.184410 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 902.87 sec) 07:12:00 [GPUGRID] [rr_sim] 2253104.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990867592.09G/439.78G) 07:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852064.96 07:12:00 [---] pick_jobs_to_run() start 07:12:00 [---] pick_jobs_to_run() end 07:12:00 [---] [rr_sim] end 07:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:12:00 [---] [cpu_sched_debug] preliminary job list: 07:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:12:00 [---] [cpu_sched_debug] final job list: 07:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:12:00 [---] [cpu_sched_debug] enforce_run_list: end 07:12:00 [---] choose_project(): 25920.000000 07:12:00 [iThena.Measurements] doing RPC to get work 07:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409679.09 cpu 2154812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:12:00 [---] Got 0 tasks 07:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:13:00 [---] [rr_sim] doing sim: CPU sched 07:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990840001.74G/439.78G) 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2253041.31 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 225304.13 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225304.131477 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027737.18 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 202773.72 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428077.849806 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824963.46 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 182496.35 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610574.196303 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642467.12 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 164246.71 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774820.908150 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478220.41 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 147822.04 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922642.948812 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330398.37 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 133039.84 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055682.785408 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197358.53 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 119735.85 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175418.638344 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077622.68 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 107762.27 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283180.905986 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969860.41 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 96986.04 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380166.946865 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872874.37 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 87287.44 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467454.383655 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785586.93 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 78558.69 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1546013.076767 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707028.24 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 70702.82 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616715.900567 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636325.41 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 63632.54 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680348.441988 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572692.87 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 57269.29 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737617.729266 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515423.59 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 51542.36 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789160.087816 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463881.23 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 46388.12 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835548.210512 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417493.10 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 41749.31 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877297.520938 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375743.79 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 37574.38 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914871.900321 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338169.41 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 33816.94 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948688.841766 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304352.47 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 30435.25 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979124.089066 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273917.23 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 27391.72 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006515.811637 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246525.50 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 24652.55 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031168.361950 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221872.95 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 22187.30 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053355.657232 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199685.66 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 19968.57 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073324.222986 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179717.09 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 17971.71 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091295.932164 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161745.38 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 16174.54 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107470.470425 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145570.84 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 14557.08 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122027.554860 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131013.76 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 13101.38 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135128.930851 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117912.38 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 11791.24 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146920.169243 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106121.15 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 10612.11 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157532.283795 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95509.03 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 9550.90 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167083.186893 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85958.13 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 8595.81 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175678.999681 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77362.32 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 7736.23 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183415.231190 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69626.08 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 6962.61 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190377.839548 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62663.48 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 6266.35 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196644.187070 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56397.13 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 5639.71 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202283.899840 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50757.41 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 5075.74 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207359.641333 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45681.67 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 4568.17 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211927.808677 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41113.51 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 4111.35 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216039.159286 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37002.16 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3700.22 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219739.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33301.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223339.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29701.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226939.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26101.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230539.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22501.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234139.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18901.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237739.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15301.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241339.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11701.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244939.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8101.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248539.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4501.94 sec) 07:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252139.374835 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 901.94 sec) 07:13:00 [GPUGRID] [rr_sim] 2253041.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990840001.74G/439.78G) 07:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852062.23 07:13:00 [---] pick_jobs_to_run() start 07:13:00 [---] pick_jobs_to_run() end 07:13:00 [---] [rr_sim] end 07:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:13:00 [---] [cpu_sched_debug] preliminary job list: 07:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:13:00 [---] [cpu_sched_debug] final job list: 07:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:13:00 [---] [cpu_sched_debug] enforce_run_list: end 07:13:00 [---] choose_project(): 25980.000000 07:13:00 [iThena.Measurements] doing RPC to get work 07:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409619.09 cpu 2154752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:13:00 [---] Got 0 tasks 07:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:14:00 [---] [rr_sim] doing sim: CPU sched 07:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990812411.38G/439.78G) 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252978.58 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 225297.86 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225297.857789 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027680.72 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 202768.07 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428065.929799 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824912.65 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 182491.26 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610557.194608 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642421.38 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 164242.14 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774799.332936 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478179.24 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 147817.92 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922617.257431 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330361.32 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 133036.13 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055653.389477 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197325.19 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 119732.52 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175385.908318 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077592.67 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 107759.27 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283145.175275 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969833.40 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 96983.34 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380128.515536 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872850.06 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 87285.01 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467413.521771 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785565.06 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 78556.51 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545970.027383 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707008.55 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 70700.86 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616670.882434 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636307.70 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 63630.77 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680301.651979 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572676.93 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 57267.69 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737569.344570 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515409.23 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 51540.92 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789110.267902 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463868.31 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 46386.83 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835497.098901 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417481.48 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 41748.15 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877245.246799 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375733.33 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 37573.33 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914818.579908 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338160.00 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 33816.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948634.579706 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304344.00 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 30434.40 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979068.979524 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273909.60 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 27390.96 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006459.939361 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246518.64 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 24651.86 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031111.803214 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221866.77 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 22186.68 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053298.480681 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199680.10 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 19968.01 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073266.490402 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179712.09 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 17971.21 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091237.699150 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161740.88 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 16174.09 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107411.787024 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145566.79 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 14556.68 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121968.466111 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131010.11 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 13101.01 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135069.477288 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117909.10 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 11790.91 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146860.387348 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106118.19 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 10611.82 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157472.206402 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95506.37 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 9550.64 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167022.843551 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85955.73 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 8595.57 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175618.416985 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77360.16 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 7736.02 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183354.433075 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69624.14 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 6962.41 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190316.847556 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62661.73 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 6266.17 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196583.020590 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56395.56 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 5639.56 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202222.576319 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50756.00 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 5075.60 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207298.176476 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45680.40 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 4568.04 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211866.216618 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41112.36 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 4111.24 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215977.452745 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37001.13 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3700.11 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219677.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33301.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223277.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29701.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226877.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26101.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230477.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22501.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234077.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18901.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237677.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15301.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241277.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11701.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244877.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8101.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248477.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4501.01 sec) 07:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252077.565259 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 901.01 sec) 07:14:00 [GPUGRID] [rr_sim] 2252978.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990812411.38G/439.78G) 07:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852059.49 07:14:00 [---] pick_jobs_to_run() start 07:14:00 [---] pick_jobs_to_run() end 07:14:00 [---] [rr_sim] end 07:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:14:00 [---] [cpu_sched_debug] preliminary job list: 07:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:14:00 [---] [cpu_sched_debug] final job list: 07:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:14:00 [---] [cpu_sched_debug] enforce_run_list: end 07:14:00 [---] choose_project(): 26040.000000 07:14:00 [iThena.Measurements] doing RPC to get work 07:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409559.09 cpu 2154692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:14:00 [---] Got 0 tasks 07:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:15:00 [---] [rr_sim] doing sim: CPU sched 07:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990784821.03G/439.78G) 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252915.84 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 225291.58 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225291.584101 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027624.26 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 202762.43 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428054.009791 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824861.83 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 182486.18 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610540.192913 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642375.65 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 164237.56 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774777.757722 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478138.08 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 147813.81 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922591.566050 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330324.27 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 133032.43 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055623.993546 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197291.85 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 119729.18 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175353.178292 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077562.66 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 107756.27 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283109.444563 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969806.40 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 96980.64 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380090.084208 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872825.76 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 87282.58 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467372.659888 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785543.18 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 78554.32 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545926.977999 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706988.86 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 70698.89 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616625.864300 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636289.98 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 63629.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680254.861971 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572660.98 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 57266.10 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737520.959874 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515394.88 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 51539.49 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789060.447987 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463855.39 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 46385.54 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835445.987289 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417469.85 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 41746.99 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877192.972661 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375722.87 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 37572.29 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914765.259495 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338150.58 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 33815.06 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948580.317647 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304335.52 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 30433.55 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979013.869983 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273901.97 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 27390.20 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006404.067085 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246511.77 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 24651.18 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031055.244477 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221860.60 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 22186.06 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053241.304130 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199674.54 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 19967.45 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073208.757818 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179707.08 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 17970.71 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091179.466136 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161736.37 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 16173.64 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107353.103623 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145562.74 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 14556.27 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121909.377362 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131006.46 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 13100.65 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135010.023726 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117905.82 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 11790.58 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146800.605454 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106115.24 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 10611.52 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157412.129009 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95503.71 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 9550.37 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166962.500209 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85953.34 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 8595.33 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175557.834289 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77358.01 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 7735.80 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183293.634960 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69622.21 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 6962.22 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190255.855565 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62659.99 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 6266.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196521.854109 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56393.99 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 5639.40 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202161.252799 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50754.59 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 5075.46 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207236.711620 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45679.13 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 4567.91 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211804.624558 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41111.22 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 4111.12 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215915.746203 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37000.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3700.01 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219615.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33300.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223215.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29700.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226815.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26100.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230415.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22500.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234015.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18900.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237615.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15300.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241215.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11700.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244815.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8100.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248415.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4500.09 sec) 07:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2252015.755683 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 900.09 sec) 07:15:00 [GPUGRID] [rr_sim] 2252915.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990784821.03G/439.78G) 07:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852056.75 07:15:00 [---] pick_jobs_to_run() start 07:15:00 [---] pick_jobs_to_run() end 07:15:00 [---] [rr_sim] end 07:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:15:00 [---] [cpu_sched_debug] preliminary job list: 07:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:15:00 [---] [cpu_sched_debug] final job list: 07:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:15:00 [---] [cpu_sched_debug] enforce_run_list: end 07:15:00 [---] choose_project(): 26100.000000 07:15:00 [iThena.Measurements] doing RPC to get work 07:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409499.09 cpu 2154632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:15:00 [---] Got 0 tasks 07:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:16:00 [---] [rr_sim] doing sim: CPU sched 07:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990757230.67G/439.78G) 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252853.10 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 225285.31 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225285.310412 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027567.79 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 202756.78 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428042.089784 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824811.01 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 182481.10 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610523.191218 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642329.91 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 164232.99 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774756.182508 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478096.92 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 147809.69 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922565.874670 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330287.23 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 133028.72 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055594.597615 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197258.51 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 119725.85 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175320.448266 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077532.66 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 107753.27 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283073.713852 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969779.39 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 96977.94 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380051.652879 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872801.45 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 87280.15 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467331.798004 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785521.31 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 78552.13 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545883.928616 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706969.18 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 70696.92 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616580.846166 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636272.26 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 63627.23 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680208.071962 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572645.03 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 57264.50 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737472.575178 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515380.53 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 51538.05 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789010.628073 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463842.48 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 46384.25 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835394.875678 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417458.23 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 41745.82 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877140.698523 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375712.41 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 37571.24 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914711.939083 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338141.17 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 33814.12 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948526.055587 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304327.05 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 30432.70 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978958.760441 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273894.34 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 27389.43 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006348.194809 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246504.91 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 24650.49 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030998.685740 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221854.42 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 22185.44 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053184.127579 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199668.98 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 19966.90 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073151.025233 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179702.08 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 17970.21 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091121.233122 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161731.87 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 16173.19 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107294.420223 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145558.68 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 14555.87 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121850.288613 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131002.82 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 13100.28 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134950.570164 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117902.53 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 11790.25 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146740.823560 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106112.28 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 10611.23 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157352.051616 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95501.05 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 9550.11 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166902.156867 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85950.95 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 8595.09 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175497.251593 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77355.85 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 7735.59 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183232.836846 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69620.27 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 6962.03 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190194.863574 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62658.24 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 6265.82 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196460.687629 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56392.42 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 5639.24 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202099.929278 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50753.17 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 5075.32 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207175.246763 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45677.86 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 4567.79 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211743.032499 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41110.07 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 4111.01 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215854.039661 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36999.06 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3699.91 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219553.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33299.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223153.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29699.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226753.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26099.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230353.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22499.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233953.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18899.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237553.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15299.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241153.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11699.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244753.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8099.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248353.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4499.16 sec) 07:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251953.946108 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 899.16 sec) 07:16:00 [GPUGRID] [rr_sim] 2252853.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990757230.67G/439.78G) 07:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852054.02 07:16:00 [---] pick_jobs_to_run() start 07:16:00 [---] pick_jobs_to_run() end 07:16:00 [---] [rr_sim] end 07:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:16:00 [---] [cpu_sched_debug] preliminary job list: 07:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:16:00 [---] [cpu_sched_debug] final job list: 07:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:16:00 [---] [cpu_sched_debug] enforce_run_list: end 07:16:00 [---] choose_project(): 26160.000000 07:16:00 [iThena.Measurements] doing RPC to get work 07:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409439.09 cpu 2154572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:16:00 [---] Got 0 tasks 07:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:17:00 [---] [rr_sim] doing sim: CPU sched 07:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990729640.31G/439.78G) 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252790.37 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 225279.04 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225279.036724 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027511.33 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 202751.13 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428030.169776 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824760.20 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 182476.02 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610506.189523 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642284.18 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 164228.42 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774734.607294 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478055.76 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 147805.58 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922540.183289 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330250.18 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 133025.02 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055565.201684 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197225.17 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 119722.52 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175287.718240 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077502.65 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 107750.26 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283037.983140 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969752.38 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 96975.24 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1380013.221551 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872777.15 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 87277.71 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467290.936120 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785499.43 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 78549.94 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545840.879232 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706949.49 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 70694.95 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616535.828033 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636254.54 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 63625.45 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680161.281954 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572629.09 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 57262.91 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737424.190483 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515366.18 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 51536.62 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788960.808158 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463829.56 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 46382.96 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835343.764067 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417446.60 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 41744.66 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877088.424384 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375701.94 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 37570.19 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914658.618670 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338131.75 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 33813.17 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948471.793527 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304318.57 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 30431.86 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978903.650899 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273886.72 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 27388.67 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006292.322533 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246498.04 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 24649.80 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030942.127004 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221848.24 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 22184.82 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053126.951028 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199663.42 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 19966.34 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073093.292649 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179697.07 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 17969.71 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091063.000108 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161727.37 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 16172.74 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107235.736822 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145554.63 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 14555.46 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121791.199864 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130999.17 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 13099.92 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134891.116602 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117899.25 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 11789.93 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146681.041666 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106109.33 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 10610.93 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157291.974223 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95498.39 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 9549.84 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166841.813525 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85948.55 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 8594.86 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175436.668897 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77353.70 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 7735.37 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183172.038731 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69618.33 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 6961.83 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190133.871582 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62656.50 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 6265.65 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196399.521148 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56390.85 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 5639.08 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202038.605758 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50751.76 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 5075.18 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207113.781906 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45676.59 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 4567.66 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211681.440440 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41108.93 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 4110.89 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215792.333120 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36998.03 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3699.80 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219492.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33298.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223092.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29698.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226692.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26098.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230292.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22498.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233892.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18898.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237492.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15298.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241092.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11698.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244692.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8098.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248292.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4498.23 sec) 07:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251892.136532 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 898.23 sec) 07:17:00 [GPUGRID] [rr_sim] 2252790.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990729640.31G/439.78G) 07:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852051.28 07:17:00 [---] pick_jobs_to_run() start 07:17:00 [---] pick_jobs_to_run() end 07:17:00 [---] [rr_sim] end 07:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:17:00 [---] [cpu_sched_debug] preliminary job list: 07:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:17:00 [---] [cpu_sched_debug] final job list: 07:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:17:00 [---] [cpu_sched_debug] enforce_run_list: end 07:17:00 [---] choose_project(): 26220.000000 07:17:00 [iThena.Measurements] doing RPC to get work 07:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409379.09 cpu 2154512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:17:00 [---] Got 0 tasks 07:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:18:00 [---] [rr_sim] doing sim: CPU sched 07:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990702049.96G/439.78G) 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252727.63 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 225272.76 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225272.763036 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027454.87 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 202745.49 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428018.249768 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824709.38 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 182470.94 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610489.187827 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642238.44 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 164223.84 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774713.032081 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1478014.60 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 147801.46 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922514.491909 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330213.14 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 133021.31 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055535.805754 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197191.82 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 119719.18 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175254.988214 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077472.64 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 107747.26 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1283002.252429 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969725.38 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 96972.54 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379974.790222 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872752.84 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 87275.28 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467250.074236 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785477.56 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 78547.76 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545797.829848 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706929.80 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 70692.98 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616490.809899 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636236.82 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 63623.68 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680114.491945 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572613.14 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 57261.31 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737375.805787 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515351.82 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 51535.18 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788910.988244 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463816.64 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 46381.66 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835292.652456 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417434.98 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 41743.50 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877036.150246 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375691.48 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 37569.15 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914605.298257 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338122.33 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 33812.23 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948417.531468 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304310.10 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 30431.01 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978848.541357 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273879.09 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 27387.91 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006236.450257 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246491.18 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 24649.12 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030885.568267 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221842.06 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 22184.21 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053069.774477 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199657.86 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 19965.79 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073035.560065 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179692.07 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 17969.21 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091004.767094 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161722.86 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 16172.29 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107177.053421 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145550.58 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 14555.06 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121732.111115 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130995.52 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 13099.55 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134831.663039 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117895.97 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 11789.60 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146621.259771 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106106.37 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 10610.64 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157231.896830 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95495.73 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 9549.57 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166781.470183 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85946.16 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 8594.62 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175376.086201 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77351.54 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 7735.15 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183111.240617 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69616.39 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 6961.64 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190072.879591 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62654.75 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 6265.48 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196338.354668 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56389.28 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 5638.93 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201977.282237 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50750.35 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 5075.03 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207052.317049 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45675.31 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 4567.53 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211619.848380 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41107.78 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 4110.78 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215730.626578 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36997.00 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3699.70 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219430.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33297.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223030.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29697.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226630.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26097.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230230.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22497.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233830.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18897.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237430.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15297.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241030.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11697.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244630.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8097.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248230.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4497.30 sec) 07:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251830.326956 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 897.30 sec) 07:18:00 [GPUGRID] [rr_sim] 2252727.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990702049.96G/439.78G) 07:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852048.54 07:18:00 [---] pick_jobs_to_run() start 07:18:00 [---] pick_jobs_to_run() end 07:18:00 [---] [rr_sim] end 07:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:18:00 [---] [cpu_sched_debug] preliminary job list: 07:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:18:00 [---] [cpu_sched_debug] final job list: 07:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:18:00 [---] [cpu_sched_debug] enforce_run_list: end 07:18:00 [---] choose_project(): 26280.000000 07:18:00 [iThena.Measurements] doing RPC to get work 07:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409319.09 cpu 2154452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:18:00 [---] Got 0 tasks 07:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:19:00 [---] [rr_sim] doing sim: CPU sched 07:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990674459.60G/439.78G) 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252664.89 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 225266.49 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225266.489348 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027398.40 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 202739.84 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 428006.329761 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824658.56 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 182465.86 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610472.186132 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642192.71 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 164219.27 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774691.456867 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477973.44 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 147797.34 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922488.800528 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330176.09 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 133017.61 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055506.409823 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197158.48 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 119715.85 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175222.258188 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077442.64 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 107744.26 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282966.521717 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969698.37 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 96969.84 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379936.358893 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872728.53 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 87272.85 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467209.212352 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785455.68 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 78545.57 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545754.780464 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706910.11 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 70691.01 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616445.791766 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636219.10 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 63621.91 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680067.701937 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572597.19 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 57259.72 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737327.421091 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515337.47 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 51533.75 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788861.168330 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463803.73 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 46380.37 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835241.540844 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417423.35 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 41742.34 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876983.876108 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375681.02 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 37568.10 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914551.977845 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338112.92 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 33811.29 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948363.269408 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304301.62 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 30430.16 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978793.431815 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273871.46 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 27387.15 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006180.577981 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246484.32 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 24648.43 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030829.009531 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221835.88 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 22183.59 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053012.597925 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199652.30 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 19965.23 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072977.827481 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179687.07 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 17968.71 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090946.534080 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161718.36 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 16171.84 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107118.370020 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145546.52 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 14554.65 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121673.022366 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130991.87 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 13099.19 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134772.209477 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117892.68 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 11789.27 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146561.477877 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106103.42 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 10610.34 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157171.819437 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95493.07 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 9549.31 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166721.126841 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85943.77 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 8594.38 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175315.503505 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77349.39 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 7734.94 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183050.442502 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69614.45 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 6961.45 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190011.887600 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62653.01 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 6265.30 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196277.188187 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56387.71 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 5638.77 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201915.958716 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50748.93 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 5074.89 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206990.852192 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45674.04 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 4567.40 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211558.256321 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41106.64 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 4110.66 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215668.920037 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36995.97 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3699.60 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219368.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33296.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222968.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29696.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226568.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26096.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230168.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22496.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233768.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18896.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237368.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15296.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240968.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11696.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244568.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8096.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248168.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4496.38 sec) 07:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251768.517381 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 896.38 sec) 07:19:00 [GPUGRID] [rr_sim] 2252664.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990674459.60G/439.78G) 07:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852045.81 07:19:00 [---] pick_jobs_to_run() start 07:19:00 [---] pick_jobs_to_run() end 07:19:00 [---] [rr_sim] end 07:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:19:00 [---] [cpu_sched_debug] preliminary job list: 07:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:19:00 [---] [cpu_sched_debug] final job list: 07:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:19:00 [---] [cpu_sched_debug] enforce_run_list: end 07:19:00 [---] choose_project(): 26340.000000 07:19:00 [iThena.Measurements] doing RPC to get work 07:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 582059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409259.09 cpu 2154392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:19:00 [---] Got 0 tasks 07:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:20:00 [---] [rr_sim] doing sim: CPU sched 07:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990646869.24G/439.78G) 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252602.16 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 225260.22 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225260.215660 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027341.94 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 202734.19 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427994.409753 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824607.75 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 182460.77 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610455.184437 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642146.97 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 164214.70 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774669.881653 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477932.27 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 147793.23 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922463.109147 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330139.05 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 133013.90 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055477.013892 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197125.14 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 119712.51 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175189.528162 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077412.63 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 107741.26 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282930.791006 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969671.37 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 96967.14 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379897.927565 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872704.23 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 87270.42 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467168.350468 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785433.81 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 78543.38 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545711.731081 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706890.43 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 70689.04 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616400.773632 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636201.38 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 63620.14 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1680020.911928 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572581.24 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 57258.12 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737279.036395 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515323.12 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 51532.31 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788811.348415 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463790.81 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 46379.08 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835190.429233 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417411.73 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 41741.17 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876931.601969 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375670.55 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 37567.06 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914498.657432 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338103.50 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 33810.35 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948309.007348 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304293.15 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 30429.31 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978738.322273 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273863.83 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 27386.38 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006124.705705 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246477.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 24647.75 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030772.450794 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221829.71 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 22182.97 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052955.421374 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199646.74 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 19964.67 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072920.094896 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179682.06 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 17968.21 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090888.301066 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161713.86 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 16171.39 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107059.686619 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145542.47 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 14554.25 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121613.933617 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130988.22 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 13098.82 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134712.755915 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117889.40 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 11788.94 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146501.695983 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106100.46 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 10610.05 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157111.742044 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95490.41 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 9549.04 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166660.783499 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85941.37 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 8594.14 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175254.920809 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77347.24 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 7734.72 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182989.644387 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69612.51 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 6961.25 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189950.895608 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62651.26 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 6265.13 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196216.021707 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56386.13 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 5638.61 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201854.635196 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50747.52 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 5074.75 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206929.387336 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45672.77 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 4567.28 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211496.664262 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41105.49 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 4110.55 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215607.213495 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36994.94 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3699.49 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219306.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33295.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222906.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29695.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226506.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26095.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230106.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22495.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233706.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18895.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237306.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15295.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240906.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11695.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244506.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8095.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248106.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4495.45 sec) 07:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251706.707805 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 895.45 sec) 07:20:00 [GPUGRID] [rr_sim] 2252602.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990646869.24G/439.78G) 07:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852043.07 07:20:00 [---] pick_jobs_to_run() start 07:20:00 [---] pick_jobs_to_run() end 07:20:00 [---] [rr_sim] end 07:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:20:00 [---] [cpu_sched_debug] preliminary job list: 07:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:20:00 [---] [cpu_sched_debug] final job list: 07:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:20:00 [---] [cpu_sched_debug] enforce_run_list: end 07:20:00 [---] choose_project(): 26400.000000 07:20:00 [iThena.Measurements] doing RPC to get work 07:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409199.09 cpu 2154332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:20:00 [---] Got 0 tasks 07:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:21:00 [---] [rr_sim] doing sim: CPU sched 07:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990619278.89G/439.78G) 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252539.42 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 225253.94 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225253.941971 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027285.48 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 202728.55 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427982.489745 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824556.93 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 182455.69 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610438.182742 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642101.24 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 164210.12 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774648.306439 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477891.11 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 147789.11 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922437.417767 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330102.00 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 133010.20 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055447.617961 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197091.80 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 119709.18 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175156.798137 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077382.62 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 107738.26 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282895.060294 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969644.36 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 96964.44 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379859.496236 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872679.92 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 87267.99 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467127.488584 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785411.93 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 78541.19 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545668.681697 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706870.74 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 70687.07 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616355.755498 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636183.66 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 63618.37 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679974.121920 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572565.30 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 57256.53 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737230.651699 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515308.77 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 51530.88 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788761.528501 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463777.89 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 46377.79 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835139.317622 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417400.10 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 41740.01 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876879.327831 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375660.09 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 37566.01 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914445.337019 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338094.08 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 33809.41 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948254.745289 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304284.67 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 30428.47 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978683.212731 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273856.21 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 27385.62 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006068.833429 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246470.59 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 24647.06 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030715.892058 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221823.53 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 22182.35 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052898.244823 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199641.17 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 19964.12 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072862.362312 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179677.06 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 17967.71 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090830.068052 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161709.35 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 16170.94 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107001.003218 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145538.42 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 14553.84 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121554.844868 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130984.57 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 13098.46 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134653.302352 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117886.12 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 11788.61 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146441.914088 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106097.51 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 10609.75 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157051.664651 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95487.76 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 9548.78 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166600.440157 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85938.98 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 8593.90 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175194.338113 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77345.08 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 7734.51 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182928.846273 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69610.57 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 6961.06 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189889.903617 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62649.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 6264.95 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196154.855226 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56384.56 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 5638.46 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201793.311675 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50746.11 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 5074.61 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206867.922479 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45671.50 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 4567.15 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211435.072202 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41104.35 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 4110.43 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215545.506953 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36993.91 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3699.39 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219244.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33294.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222844.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29694.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226444.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26094.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230044.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22494.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233644.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18894.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237244.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15294.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240844.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11694.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244444.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8094.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248044.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4494.52 sec) 07:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251644.898229 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 894.52 sec) 07:21:00 [GPUGRID] [rr_sim] 2252539.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990619278.89G/439.78G) 07:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852040.33 07:21:00 [---] pick_jobs_to_run() start 07:21:00 [---] pick_jobs_to_run() end 07:21:00 [---] [rr_sim] end 07:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:21:00 [---] [cpu_sched_debug] preliminary job list: 07:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:21:00 [---] [cpu_sched_debug] final job list: 07:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:21:00 [---] [cpu_sched_debug] enforce_run_list: end 07:21:00 [---] choose_project(): 26460.000000 07:21:00 [iThena.Measurements] doing RPC to get work 07:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409139.09 cpu 2154272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:21:00 [---] Got 0 tasks 07:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:22:00 [---] [rr_sim] doing sim: CPU sched 07:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990591688.53G/439.78G) 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252476.68 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 225247.67 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225247.668283 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027229.01 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 202722.90 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427970.569738 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824506.11 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 182450.61 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610421.181047 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642055.50 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 164205.55 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774626.731226 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477849.95 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 147785.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922411.726386 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330064.96 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 133006.50 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055418.222031 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197058.46 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 119705.85 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175124.068111 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077352.61 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 107735.26 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282859.329583 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969617.35 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 96961.74 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379821.064908 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872655.62 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 87265.56 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467086.626700 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785390.06 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 78539.01 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545625.632313 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706851.05 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 70685.11 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616310.737365 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636165.95 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 63616.59 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679927.331911 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572549.35 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 57254.94 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737182.267003 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515294.42 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 51529.44 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788711.708586 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463764.97 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 46376.50 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835088.206011 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417388.48 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 41738.85 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876827.053693 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375649.63 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 37564.96 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914392.016606 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338084.67 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 33808.47 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948200.483229 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304276.20 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 30427.62 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978628.103189 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273848.58 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 27384.86 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006012.961153 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246463.72 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 24646.37 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030659.333321 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221817.35 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 22181.73 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052841.068272 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199635.61 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 19963.56 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072804.629728 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179672.05 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 17967.21 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090771.835038 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161704.85 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 16170.48 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106942.319818 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145534.36 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 14553.44 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121495.756119 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130980.93 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 13098.09 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134593.848790 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117882.83 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 11788.28 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146382.132194 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106094.55 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 10609.46 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156991.587258 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95485.10 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 9548.51 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166540.096815 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85936.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 8593.66 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175133.755417 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77342.93 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 7734.29 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182868.048158 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69608.63 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 6960.86 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189828.911625 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62647.77 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 6264.78 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196093.688746 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56382.99 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 5638.30 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201731.988155 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50744.69 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 5074.47 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206806.457622 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45670.23 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 4567.02 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211373.480143 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41103.20 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 4110.32 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215483.800412 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36992.88 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3699.29 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219183.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33293.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222783.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29693.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226383.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26093.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229983.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22493.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233583.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18893.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237183.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15293.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240783.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11693.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244383.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8093.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247983.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4493.59 sec) 07:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251583.088654 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 893.59 sec) 07:22:00 [GPUGRID] [rr_sim] 2252476.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990591688.53G/439.78G) 07:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852037.60 07:22:00 [---] pick_jobs_to_run() start 07:22:00 [---] pick_jobs_to_run() end 07:22:00 [---] [rr_sim] end 07:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:22:00 [---] [cpu_sched_debug] preliminary job list: 07:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:22:00 [---] [cpu_sched_debug] final job list: 07:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:22:00 [---] [cpu_sched_debug] enforce_run_list: end 07:22:00 [---] choose_project(): 26520.000000 07:22:00 [iThena.Measurements] doing RPC to get work 07:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409079.09 cpu 2154212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:22:00 [---] Got 0 tasks 07:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:23:00 [---] [rr_sim] doing sim: CPU sched 07:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990564098.18G/439.78G) 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252413.95 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 225241.39 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225241.394595 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027172.55 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 202717.26 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427958.649730 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824455.30 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 182445.53 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610404.179352 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1642009.77 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 164200.98 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774605.156012 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477808.79 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 147780.88 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922386.035006 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1330027.91 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 133002.79 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055388.826100 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1197025.12 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 119702.51 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175091.338085 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077322.61 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 107732.26 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282823.598871 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969590.35 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 96959.03 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379782.633579 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872631.31 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 87263.13 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467045.764816 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785368.18 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 78536.82 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545582.582929 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706831.36 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 70683.14 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616265.719231 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636148.23 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 63614.82 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679880.541903 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572533.40 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 57253.34 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737133.882308 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515280.06 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 51528.01 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788661.888672 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463752.06 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 46375.21 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835037.094399 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417376.85 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 41737.69 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876774.779554 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375639.17 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 37563.92 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914338.696194 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338075.25 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 33807.52 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948146.221169 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304267.72 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 30426.77 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978572.993647 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273840.95 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 27384.10 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005957.088877 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246456.86 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 24645.69 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030602.774585 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221811.17 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 22181.12 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052783.891721 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199630.05 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 19963.01 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072746.897144 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179667.05 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 17966.70 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090713.602024 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161700.34 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 16170.03 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106883.636417 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145530.31 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 14553.03 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121436.667370 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130977.28 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 13097.73 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134534.395228 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117879.55 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 11787.96 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146322.350300 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106091.60 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 10609.16 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156931.509865 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95482.44 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 9548.24 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166479.753473 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85934.19 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 8593.42 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175073.172721 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77340.77 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 7734.08 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182807.250044 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69606.70 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 6960.67 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189767.919634 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62646.03 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 6264.60 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196032.522266 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56381.42 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 5638.14 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201670.664634 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50743.28 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 5074.33 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206744.992765 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45668.95 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 4566.90 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211311.888084 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41102.06 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 4110.21 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215422.093870 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36991.85 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3699.19 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219121.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33292.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222721.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29692.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226321.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26092.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229921.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22492.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233521.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18892.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237121.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15292.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240721.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11692.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244321.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8092.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247921.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4492.67 sec) 07:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251521.279078 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 892.67 sec) 07:23:00 [GPUGRID] [rr_sim] 2252413.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990564098.18G/439.78G) 07:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852034.86 07:23:00 [---] pick_jobs_to_run() start 07:23:00 [---] pick_jobs_to_run() end 07:23:00 [---] [rr_sim] end 07:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:23:00 [---] [cpu_sched_debug] preliminary job list: 07:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:23:00 [---] [cpu_sched_debug] final job list: 07:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:23:00 [---] [cpu_sched_debug] enforce_run_list: end 07:23:00 [---] choose_project(): 26580.000000 07:23:00 [iThena.Measurements] doing RPC to get work 07:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 409019.09 cpu 2154152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:23:00 [---] Got 0 tasks 07:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:24:00 [---] [rr_sim] doing sim: CPU sched 07:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990536507.82G/439.78G) 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252351.21 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 225235.12 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225235.120907 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027116.09 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 202711.61 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427946.729723 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824404.48 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 182440.45 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610387.177657 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641964.03 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 164196.40 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774583.580798 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477767.63 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 147776.76 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922360.343625 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329990.87 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 132999.09 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055359.430169 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196991.78 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 119699.18 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175058.608059 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077292.60 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 107729.26 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282787.868160 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969563.34 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 96956.33 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379744.202250 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872607.01 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 87260.70 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1467004.902932 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785346.31 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 78534.63 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545539.533545 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706811.68 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 70681.17 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616220.701098 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636130.51 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 63613.05 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679833.751894 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572517.46 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 57251.75 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737085.497612 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515265.71 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 51526.57 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788612.068757 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463739.14 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 46373.91 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834985.982788 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417365.23 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 41736.52 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876722.505416 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375628.70 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 37562.87 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914285.375781 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338065.83 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 33806.58 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948091.959110 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304259.25 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 30425.92 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978517.884105 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273833.32 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 27383.33 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005901.216601 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246449.99 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 24645.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030546.215848 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221804.99 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 22180.50 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052726.715170 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199624.49 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 19962.45 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072689.164560 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179662.04 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 17966.20 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090655.369010 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161695.84 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 16169.58 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106824.953016 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145526.26 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 14552.63 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121377.578621 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130973.63 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 13097.36 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134474.941666 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117876.27 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 11787.63 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146262.568406 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106088.64 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 10608.86 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156871.432472 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95479.78 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 9547.98 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166419.410131 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85931.80 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 8593.18 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175012.590025 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77338.62 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 7733.86 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182746.451929 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69604.76 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 6960.48 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189706.927643 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62644.28 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 6264.43 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195971.355785 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56379.85 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 5637.99 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201609.341113 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50741.87 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 5074.19 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206683.527909 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45667.68 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 4566.77 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211250.296024 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41100.91 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 4110.09 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215360.387329 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36990.82 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3699.08 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219059.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33291.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222659.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29691.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226259.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26091.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229859.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22491.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233459.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18891.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237059.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15291.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240659.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11691.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244259.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8091.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247859.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4491.74 sec) 07:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251459.469502 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 891.74 sec) 07:24:00 [GPUGRID] [rr_sim] 2252351.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990536507.82G/439.78G) 07:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852032.12 07:24:00 [---] pick_jobs_to_run() start 07:24:00 [---] pick_jobs_to_run() end 07:24:00 [---] [rr_sim] end 07:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:24:00 [---] [cpu_sched_debug] preliminary job list: 07:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:24:00 [---] [cpu_sched_debug] final job list: 07:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:24:00 [---] [cpu_sched_debug] enforce_run_list: end 07:24:00 [---] choose_project(): 26640.000000 07:24:00 [iThena.Measurements] doing RPC to get work 07:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408959.09 cpu 2154092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:24:00 [---] Got 0 tasks 07:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:25:00 [---] [rr_sim] doing sim: CPU sched 07:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990508917.46G/439.78G) 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252288.47 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 225228.85 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225228.847218 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027059.62 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 202705.96 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427934.809715 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824353.66 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 182435.37 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610370.175962 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641918.30 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 164191.83 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774562.005584 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477726.47 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 147772.65 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922334.652244 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329953.82 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 132995.38 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055330.034238 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196958.44 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 119695.84 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1175025.878033 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077262.59 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 107726.26 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282752.137448 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969536.33 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 96953.63 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379705.770922 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872582.70 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 87258.27 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466964.041048 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785324.43 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 78532.44 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545496.484162 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706791.99 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 70679.20 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616175.682964 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636112.79 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 63611.28 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679786.961886 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572501.51 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 57250.15 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1737037.112916 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515251.36 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 51525.14 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788562.248843 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463726.22 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 46372.62 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834934.871177 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417353.60 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 41735.36 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876670.231278 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375618.24 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 37561.82 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914232.055368 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338056.42 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 33805.64 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948037.697050 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304250.78 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 30425.08 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978462.774563 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273825.70 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 27382.57 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005845.344326 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246443.13 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 24644.31 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030489.657111 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221798.82 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 22179.88 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052669.538619 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199618.93 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 19961.89 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072631.431975 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179657.04 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 17965.70 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090597.135996 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161691.34 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 16169.13 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106766.269615 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145522.20 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 14552.22 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121318.489872 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130969.98 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 13097.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134415.488103 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117872.98 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 11787.30 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146202.786511 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106085.69 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 10608.57 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156811.355079 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95477.12 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 9547.71 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166359.066789 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85929.41 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 8592.94 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174952.007329 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77336.46 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 7733.65 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182685.653814 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69602.82 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 6960.28 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189645.935651 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62642.54 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 6264.25 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195910.189305 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56378.28 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 5637.83 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201548.017593 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50740.45 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 5074.05 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206622.063052 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45666.41 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 4566.64 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211188.703965 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41099.77 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 4109.98 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215298.680787 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36989.79 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3698.98 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218997.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33290.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222597.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29690.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226197.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26090.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229797.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22490.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233397.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18890.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236997.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15290.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240597.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11690.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244197.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8090.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247797.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4490.81 sec) 07:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251397.659927 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 890.81 sec) 07:25:00 [GPUGRID] [rr_sim] 2252288.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990508917.46G/439.78G) 07:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852029.38 07:25:00 [---] pick_jobs_to_run() start 07:25:00 [---] pick_jobs_to_run() end 07:25:00 [---] [rr_sim] end 07:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:25:00 [---] [cpu_sched_debug] preliminary job list: 07:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:25:00 [---] [cpu_sched_debug] final job list: 07:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:25:00 [---] [cpu_sched_debug] enforce_run_list: end 07:25:00 [---] choose_project(): 26700.000000 07:25:00 [iThena.Measurements] doing RPC to get work 07:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408899.09 cpu 2154032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2154032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2154032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:25:00 [---] Got 0 tasks 07:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:26:00 [---] [rr_sim] doing sim: CPU sched 07:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990481327.11G/439.78G) 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252225.74 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 225222.57 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225222.573530 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2027003.16 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 202700.32 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427922.889707 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824302.85 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 182430.28 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610353.174267 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641872.56 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 164187.26 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774540.430370 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477685.30 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 147768.53 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922308.960864 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329916.77 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 132991.68 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055300.638308 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196925.10 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 119692.51 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174993.148007 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077232.59 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 107723.26 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282716.406737 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969509.33 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 96950.93 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379667.339593 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872558.40 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 87255.84 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466923.179164 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785302.56 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 78530.26 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545453.434778 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706772.30 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 70677.23 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616130.664830 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636095.07 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 63609.51 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679740.171878 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572485.56 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 57248.56 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736988.728220 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515237.01 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 51523.70 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788512.428928 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463713.31 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 46371.33 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834883.759566 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417341.98 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 41734.20 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876617.957139 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375607.78 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 37560.78 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914178.734956 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338047.00 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 33804.70 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947983.434990 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304242.30 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 30424.23 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978407.665021 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273818.07 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 27381.81 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005789.472050 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246436.26 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 24643.63 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030433.098375 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221792.64 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 22179.26 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052612.362068 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199613.37 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 19961.34 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072573.699391 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179652.04 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 17965.20 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090538.902982 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161686.83 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 16168.68 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106707.586214 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145518.15 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 14551.81 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121259.401123 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130966.33 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 13096.63 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134356.034541 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117869.70 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 11786.97 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146143.004617 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106082.73 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 10608.27 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156751.277686 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95474.46 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 9547.45 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166298.723447 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85927.01 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 8592.70 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174891.424633 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77334.31 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 7733.43 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182624.855700 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69600.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 6960.09 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189584.943660 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62640.79 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 6264.08 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195849.022824 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56376.71 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 5637.67 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201486.694072 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50739.04 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 5073.90 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206560.598195 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45665.14 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 4566.51 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211127.111906 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41098.62 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 4109.86 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215236.974245 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36988.76 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3698.88 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218935.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33289.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222535.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29689.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226135.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26089.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229735.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22489.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233335.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18889.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236935.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15289.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240535.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11689.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244135.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8089.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247735.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4489.88 sec) 07:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251335.850351 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 889.88 sec) 07:26:00 [GPUGRID] [rr_sim] 2252225.74: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990481327.11G/439.78G) 07:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852026.65 07:26:00 [---] pick_jobs_to_run() start 07:26:00 [---] pick_jobs_to_run() end 07:26:00 [---] [rr_sim] end 07:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:26:00 [---] [cpu_sched_debug] preliminary job list: 07:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:26:00 [---] [cpu_sched_debug] final job list: 07:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:26:00 [---] [cpu_sched_debug] enforce_run_list: end 07:26:00 [---] choose_project(): 26760.000000 07:26:00 [iThena.Measurements] doing RPC to get work 07:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408839.09 cpu 2153972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:26:00 [---] Got 0 tasks 07:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:27:00 [---] [rr_sim] doing sim: CPU sched 07:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990453736.75G/439.78G) 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252163.00 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 225216.30 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225216.299842 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026946.70 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 202694.67 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427910.969700 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824252.03 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 182425.20 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610336.172572 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641826.83 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 164182.68 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774518.855157 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477644.14 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 147764.41 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922283.269483 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329879.73 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 132987.97 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055271.242377 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196891.76 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 119689.18 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174960.417981 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077202.58 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 107720.26 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282680.676025 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969482.32 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 96948.23 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379628.908265 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872534.09 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 87253.41 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466882.317280 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785280.68 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 78528.07 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545410.385394 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706752.61 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 70675.26 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616085.646697 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636077.35 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 63607.74 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679693.381869 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572469.62 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 57246.96 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736940.343524 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515222.65 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 51522.27 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788462.609014 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463700.39 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 46370.04 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834832.647954 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417330.35 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 41733.04 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876565.683001 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375597.32 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 37559.73 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914125.414543 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338037.58 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 33803.76 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947929.172931 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304233.83 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 30423.38 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978352.555480 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273810.44 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 27381.04 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005733.599774 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246429.40 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 24642.94 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030376.539638 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221786.46 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 22178.65 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052555.185516 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199607.81 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 19960.78 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072515.966807 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179647.03 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 17964.70 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090480.669968 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161682.33 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 16168.23 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106648.902813 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145514.10 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 14551.41 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121200.312374 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130962.69 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 13096.27 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134296.580979 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117866.42 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 11786.64 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146083.222723 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106079.78 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 10607.98 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156691.200293 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95471.80 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 9547.18 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166238.380105 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85924.62 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 8592.46 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174830.841937 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77332.16 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 7733.22 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182564.057585 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69598.94 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 6959.89 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189523.951669 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62639.05 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 6263.90 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195787.856344 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56375.14 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 5637.51 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201425.370551 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50737.63 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 5073.76 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206499.133338 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45663.87 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 4566.39 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211065.519846 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41097.48 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 4109.75 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215175.267704 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36987.73 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3698.77 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218874.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33288.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222474.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29688.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226074.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26088.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229674.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22488.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233274.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18888.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236874.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15288.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240474.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11688.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244074.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8088.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247674.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4488.96 sec) 07:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251274.040775 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 888.96 sec) 07:27:00 [GPUGRID] [rr_sim] 2252163.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990453736.75G/439.78G) 07:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852023.91 07:27:00 [---] pick_jobs_to_run() start 07:27:00 [---] pick_jobs_to_run() end 07:27:00 [---] [rr_sim] end 07:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:27:00 [---] [cpu_sched_debug] preliminary job list: 07:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:27:00 [---] [cpu_sched_debug] final job list: 07:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:27:00 [---] [cpu_sched_debug] enforce_run_list: end 07:27:00 [---] choose_project(): 26820.000000 07:27:00 [iThena.Measurements] doing RPC to get work 07:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408779.09 cpu 2153912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:27:00 [---] Got 0 tasks 07:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:28:00 [---] [rr_sim] doing sim: CPU sched 07:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990426146.39G/439.78G) 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252100.26 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 225210.03 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225210.026154 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026890.24 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 202689.02 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427899.049692 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824201.21 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 182420.12 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610319.170877 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641781.09 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 164178.11 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774497.279943 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477602.98 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 147760.30 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922257.578102 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329842.68 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 132984.27 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055241.846446 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196858.42 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 119685.84 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174927.687955 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077172.57 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 107717.26 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282644.945313 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969455.32 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 96945.53 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379590.476936 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872509.78 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 87250.98 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466841.455396 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785258.81 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 78525.88 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545367.336010 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706732.93 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 70673.29 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1616040.628563 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636059.63 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 63605.96 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679646.591861 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572453.67 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 57245.37 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736891.958828 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515208.30 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 51520.83 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788412.789099 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463687.47 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 46368.75 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834781.536343 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417318.73 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 41731.87 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876513.408863 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375586.85 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 37558.69 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914072.094130 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338028.17 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 33802.82 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947874.910871 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304225.35 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 30422.54 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978297.445938 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273802.82 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 27380.28 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005677.727498 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246422.53 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 24642.25 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030319.980902 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221780.28 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 22178.03 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052498.008965 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199602.25 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 19960.23 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072458.234223 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179642.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 17964.20 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090422.436954 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161677.82 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 16167.78 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106590.219413 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145510.04 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 14551.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121141.223625 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130959.04 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 13095.90 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134237.127416 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117863.13 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 11786.31 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146023.440829 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106076.82 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 10607.68 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156631.122899 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95469.14 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 9546.91 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166178.036763 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85922.22 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 8592.22 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174770.259241 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77330.00 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 7733.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182503.259471 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69597.00 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 6959.70 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189462.959677 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62637.30 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 6263.73 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195726.689863 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56373.57 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 5637.36 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201364.047031 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50736.21 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 5073.62 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206437.668482 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45662.59 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 4566.26 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211003.927787 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41096.33 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 4109.63 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215113.561162 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36986.70 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3698.67 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218812.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33288.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222412.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29688.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226012.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26088.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229612.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22488.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233212.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18888.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236812.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15288.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240412.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11688.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244012.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8088.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247612.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4488.03 sec) 07:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251212.231200 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 888.03 sec) 07:28:00 [GPUGRID] [rr_sim] 2252100.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990426146.39G/439.78G) 07:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852021.17 07:28:00 [---] pick_jobs_to_run() start 07:28:00 [---] pick_jobs_to_run() end 07:28:00 [---] [rr_sim] end 07:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:28:00 [---] [cpu_sched_debug] preliminary job list: 07:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:28:00 [---] [cpu_sched_debug] final job list: 07:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:28:00 [---] [cpu_sched_debug] enforce_run_list: end 07:28:00 [---] choose_project(): 26880.000000 07:28:00 [iThena.Measurements] doing RPC to get work 07:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408719.09 cpu 2153852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:28:00 [---] Got 0 tasks 07:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:29:00 [---] [rr_sim] doing sim: CPU sched 07:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990398556.04G/439.78G) 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2252037.52 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 225203.75 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225203.752466 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026833.77 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 202683.38 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427887.129685 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824150.39 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 182415.04 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610302.169182 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641735.36 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 164173.54 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774475.704729 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477561.82 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 147756.18 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922231.886722 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329805.64 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 132980.56 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055212.450515 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196825.07 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 119682.51 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174894.957929 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077142.57 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 107714.26 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282609.214602 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969428.31 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 96942.83 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379552.045607 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872485.48 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 87248.55 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466800.593512 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785236.93 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 78523.69 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545324.286627 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706713.24 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 70671.32 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615995.610429 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636041.91 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 63604.19 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679599.801852 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572437.72 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 57243.77 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736843.574132 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515193.95 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 51519.40 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788362.969185 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463674.56 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 46367.46 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834730.424732 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417307.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 41730.71 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876461.134724 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375576.39 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 37557.64 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914018.773717 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338018.75 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 33801.88 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947820.648811 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304216.88 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 30421.69 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978242.336396 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273795.19 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 27379.52 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005621.855222 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246415.67 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 24641.57 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030263.422165 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221774.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 22177.41 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052440.832414 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199596.69 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 19959.67 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072400.501638 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179637.02 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 17963.70 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090364.203940 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161673.32 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 16167.33 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106531.536012 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145505.99 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 14550.60 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121082.134876 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130955.39 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 13095.54 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134177.673854 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117859.85 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 11785.99 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145963.658934 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106073.87 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 10607.39 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156571.045506 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95466.48 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 9546.65 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166117.693421 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85919.83 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 8591.98 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174709.676545 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77327.85 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 7732.78 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182442.461356 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69595.06 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 6959.51 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189401.967686 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62635.56 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 6263.56 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195665.523383 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56372.00 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 5637.20 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201302.723510 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50734.80 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 5073.48 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206376.203625 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45661.32 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 4566.13 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210942.335728 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41095.19 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 4109.52 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215051.854621 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36985.67 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3698.57 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218750.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33287.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222350.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29687.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225950.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26087.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229550.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22487.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233150.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18887.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236750.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15287.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240350.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11687.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243950.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8087.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247550.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4487.10 sec) 07:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251150.421624 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 887.10 sec) 07:29:00 [GPUGRID] [rr_sim] 2252037.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990398556.04G/439.78G) 07:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852018.44 07:29:00 [---] pick_jobs_to_run() start 07:29:00 [---] pick_jobs_to_run() end 07:29:00 [---] [rr_sim] end 07:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:29:00 [---] [cpu_sched_debug] preliminary job list: 07:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:29:00 [---] [cpu_sched_debug] final job list: 07:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:29:00 [---] [cpu_sched_debug] enforce_run_list: end 07:29:00 [---] choose_project(): 26940.000000 07:29:00 [iThena.Measurements] doing RPC to get work 07:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408659.09 cpu 2153792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:29:00 [---] Got 0 tasks 07:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:30:00 [---] [rr_sim] doing sim: CPU sched 07:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990370965.68G/439.78G) 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251974.79 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 225197.48 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225197.478777 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026777.31 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 202677.73 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427875.209677 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824099.58 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 182409.96 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610285.167487 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641689.62 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 164168.96 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774454.129515 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477520.66 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 147752.07 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922206.195341 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329768.59 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 132976.86 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055183.054584 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196791.73 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 119679.17 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174862.227903 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077112.56 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 107711.26 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282573.483890 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969401.30 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 96940.13 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379513.614279 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872461.17 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 87246.12 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466759.731628 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785215.06 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 78521.51 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545281.237243 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706693.55 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 70669.36 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615950.592296 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636024.20 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 63602.42 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679553.011844 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572421.78 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 57242.18 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736795.189437 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515179.60 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 51517.96 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788313.149270 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463661.64 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 46366.16 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834679.313121 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417295.47 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 41729.55 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876408.860586 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375565.93 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 37556.59 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913965.453305 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338009.33 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 33800.93 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947766.386752 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304208.40 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 30420.84 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978187.226854 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273787.56 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 27378.76 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005565.982946 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246408.80 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 24640.88 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030206.863429 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221767.92 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 22176.79 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052383.655863 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199591.13 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 19959.11 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072342.769054 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179632.02 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 17963.20 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090305.970926 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161668.82 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 16166.88 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106472.852611 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145501.94 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 14550.19 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121023.046127 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130951.74 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 13095.17 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134118.220292 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117856.57 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 11785.66 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145903.877040 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106070.91 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 10607.09 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156510.968113 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95463.82 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 9546.38 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166057.350079 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85917.44 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 8591.74 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174649.093849 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77325.69 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 7732.57 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182381.663241 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69593.12 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 6959.31 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189340.975695 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62633.81 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 6263.38 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195604.356902 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56370.43 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 5637.04 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201241.399990 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50733.39 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 5073.34 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206314.738768 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45660.05 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 4566.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210880.743669 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41094.04 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 4109.40 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214990.148079 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36984.64 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3698.46 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218688.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33286.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222288.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29686.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225888.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26086.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229488.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22486.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233088.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18886.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236688.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15286.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240288.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11686.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243888.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8086.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247488.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4486.18 sec) 07:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251088.612049 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 886.18 sec) 07:30:00 [GPUGRID] [rr_sim] 2251974.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990370965.68G/439.78G) 07:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852015.70 07:30:00 [---] pick_jobs_to_run() start 07:30:00 [---] pick_jobs_to_run() end 07:30:00 [---] [rr_sim] end 07:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:30:00 [---] [cpu_sched_debug] preliminary job list: 07:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:30:00 [---] [cpu_sched_debug] final job list: 07:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:30:00 [---] [cpu_sched_debug] enforce_run_list: end 07:30:00 [---] choose_project(): 27000.000000 07:30:00 [iThena.Measurements] doing RPC to get work 07:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408599.09 cpu 2153732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:30:00 [---] Got 0 tasks 07:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:31:00 [---] [rr_sim] doing sim: CPU sched 07:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990343375.33G/439.78G) 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251912.05 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 225191.21 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225191.205089 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026720.85 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 202672.08 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427863.289669 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1824048.76 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 182404.88 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610268.165792 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641643.89 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 164164.39 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774432.554302 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477479.50 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 147747.95 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922180.503961 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329731.55 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 132973.15 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055153.658654 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196758.39 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 119675.84 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174829.497877 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077082.55 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 107708.26 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282537.753179 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969374.30 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 96937.43 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379475.182950 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872436.87 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 87243.69 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466718.869744 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785193.18 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 78519.32 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545238.187859 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706673.86 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 70667.39 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615905.574162 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636006.48 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 63600.65 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679506.221835 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572405.83 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 57240.58 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736746.804741 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515165.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 51516.52 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788263.329356 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463648.72 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 46364.87 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834628.201509 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417283.85 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 41728.38 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876356.586448 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375555.46 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 37555.55 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913912.132892 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337999.92 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 33799.99 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947712.124692 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304199.93 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 30419.99 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978132.117312 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273779.93 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 27377.99 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005510.110670 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246401.94 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 24640.19 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030150.304692 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221761.75 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 22176.17 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052326.479312 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199585.57 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 19958.56 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072285.036470 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179627.01 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 17962.70 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090247.737912 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161664.31 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 16166.43 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106414.169210 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145497.88 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 14549.79 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120963.957378 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130948.09 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 13094.81 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134058.766730 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117853.28 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 11785.33 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145844.095146 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106067.96 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 10606.80 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156450.890720 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95461.16 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 9546.12 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165997.006737 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85915.04 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 8591.50 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174588.511153 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77323.54 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 7732.35 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182320.865127 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69591.19 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 6959.12 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189279.983703 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62632.07 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 6263.21 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195543.190422 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56368.86 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 5636.89 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201180.076469 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50731.97 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 5073.20 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206253.273911 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45658.78 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 4565.88 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210819.151609 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41092.90 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 4109.29 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214928.441537 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36983.61 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3698.36 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218626.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33285.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222226.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29685.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225826.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26085.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229426.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22485.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233026.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18885.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236626.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15285.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240226.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11685.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243826.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8085.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247426.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4485.25 sec) 07:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2251026.802473 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 885.25 sec) 07:31:00 [GPUGRID] [rr_sim] 2251912.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990343375.33G/439.78G) 07:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852012.96 07:31:00 [---] pick_jobs_to_run() start 07:31:00 [---] pick_jobs_to_run() end 07:31:00 [---] [rr_sim] end 07:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:31:00 [---] [cpu_sched_debug] preliminary job list: 07:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:31:00 [---] [cpu_sched_debug] final job list: 07:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:31:00 [---] [cpu_sched_debug] enforce_run_list: end 07:31:00 [---] choose_project(): 27060.000000 07:31:00 [iThena.Measurements] doing RPC to get work 07:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408539.09 cpu 2153672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:31:00 [---] Got 0 tasks 07:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:32:00 [---] [rr_sim] doing sim: CPU sched 07:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990315784.97G/439.78G) 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251849.31 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 225184.93 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225184.931401 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026664.38 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 202666.44 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427851.369662 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823997.94 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 182399.79 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610251.164097 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641598.15 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 164159.81 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774410.979088 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477438.33 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 147743.83 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922154.812580 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329694.50 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 132969.45 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055124.262723 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196725.05 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 119672.51 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174796.767852 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077052.55 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 107705.25 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282502.022467 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969347.29 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 96934.73 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379436.751622 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872412.56 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 87241.26 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466678.007860 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785171.31 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 78517.13 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545195.138475 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706654.18 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 70665.42 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615860.556029 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635988.76 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 63598.88 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679459.431827 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572389.88 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 57238.99 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736698.420045 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515150.89 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 51515.09 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788213.509441 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463635.80 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 46363.58 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834577.089898 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417272.22 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 41727.22 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876304.312309 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375545.00 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 37554.50 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913858.812479 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337990.50 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 33799.05 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947657.862632 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304191.45 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 30419.15 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978077.007770 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273772.31 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 27377.23 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005454.238394 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246395.08 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 24639.51 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030093.745955 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221755.57 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 22175.56 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052269.302761 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199580.01 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 19958.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072227.303886 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179622.01 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 17962.20 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090189.504898 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161659.81 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 16165.98 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106355.485809 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145493.83 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 14549.38 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120904.868629 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130944.45 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 13094.44 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133999.313167 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117850.00 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 11785.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145784.313251 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106065.00 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 10606.50 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156390.813327 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95458.50 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 9545.85 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165936.663395 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85912.65 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 8591.27 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174527.928457 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77321.39 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 7732.14 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182260.067012 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69589.25 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 6958.92 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189218.991712 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62630.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 6263.03 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195482.023942 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56367.29 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 5636.73 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201118.752948 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50730.56 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 5073.06 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206191.809054 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45657.50 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 4565.75 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210757.559550 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41091.75 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 4109.18 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214866.734996 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36982.58 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3698.26 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218564.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33284.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222164.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29684.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225764.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26084.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229364.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22484.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232964.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18884.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236564.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15284.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240164.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11684.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243764.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8084.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247364.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4484.32 sec) 07:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250964.992897 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 884.32 sec) 07:32:00 [GPUGRID] [rr_sim] 2251849.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990315784.97G/439.78G) 07:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852010.23 07:32:00 [---] pick_jobs_to_run() start 07:32:00 [---] pick_jobs_to_run() end 07:32:00 [---] [rr_sim] end 07:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:32:00 [---] [cpu_sched_debug] preliminary job list: 07:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:32:00 [---] [cpu_sched_debug] final job list: 07:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:32:00 [---] [cpu_sched_debug] enforce_run_list: end 07:32:00 [---] choose_project(): 27120.000000 07:32:00 [iThena.Measurements] doing RPC to get work 07:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408479.09 cpu 2153612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:32:00 [---] Got 0 tasks 07:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:33:00 [---] [rr_sim] doing sim: CPU sched 07:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990288194.61G/439.78G) 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251786.58 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 225178.66 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225178.657713 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026607.92 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 202660.79 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427839.449654 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823947.13 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 182394.71 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610234.162401 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641552.41 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 164155.24 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774389.403874 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477397.17 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 147739.72 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922129.121199 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329657.46 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 132965.75 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055094.866792 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196691.71 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 119669.17 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174764.037826 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1077022.54 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 107702.25 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282466.291756 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969320.29 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 96932.03 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379398.320293 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872388.26 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 87238.83 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466637.145976 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785149.43 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 78514.94 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545152.089091 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706634.49 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 70663.45 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615815.537895 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635971.04 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 63597.10 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679412.641818 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572373.94 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 57237.39 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736650.035349 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515136.54 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 51513.65 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788163.689527 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463622.89 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 46362.29 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834525.978287 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417260.60 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 41726.06 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876252.038171 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375534.54 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 37553.45 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913805.492067 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337981.09 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 33798.11 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947603.600573 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304182.98 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 30418.30 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978021.898228 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273764.68 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 27376.47 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005398.366118 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246388.21 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 24638.82 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030037.187219 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221749.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 22174.94 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052212.126210 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199574.45 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 19957.45 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072169.571301 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179617.01 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 17961.70 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090131.271884 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161655.31 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 16165.53 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106296.802408 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145489.77 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 14548.98 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120845.779880 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130940.80 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 13094.08 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133939.859605 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117846.72 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 11784.67 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145724.531357 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106062.05 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 10606.20 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156330.735934 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95455.84 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 9545.58 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165876.320053 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85910.26 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 8591.03 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174467.345761 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77319.23 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 7731.92 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182199.268897 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69587.31 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 6958.73 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189157.999720 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62628.58 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 6262.86 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195420.857461 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56365.72 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 5636.57 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201057.429428 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50729.15 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 5072.91 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206130.344198 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45656.23 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 4565.62 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210695.967491 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41090.61 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 4109.06 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214805.028454 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36981.55 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3698.15 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218503.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33283.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222103.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29683.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225703.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26083.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229303.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22483.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232903.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18883.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236503.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15283.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240103.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11683.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243703.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8083.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247303.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4483.39 sec) 07:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250903.183322 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 883.39 sec) 07:33:00 [GPUGRID] [rr_sim] 2251786.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990288194.61G/439.78G) 07:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852007.49 07:33:00 [---] pick_jobs_to_run() start 07:33:00 [---] pick_jobs_to_run() end 07:33:00 [---] [rr_sim] end 07:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:33:00 [---] [cpu_sched_debug] preliminary job list: 07:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:33:00 [---] [cpu_sched_debug] final job list: 07:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:33:00 [---] [cpu_sched_debug] enforce_run_list: end 07:33:00 [---] choose_project(): 27180.000000 07:33:00 [iThena.Measurements] doing RPC to get work 07:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408419.09 cpu 2153552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:33:00 [---] Got 0 tasks 07:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:34:00 [---] [rr_sim] doing sim: CPU sched 07:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990260604.26G/439.78G) 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251723.84 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 225172.38 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225172.384025 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026551.46 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 202655.15 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427827.529647 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823896.31 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 182389.63 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610217.160706 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641506.68 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 164150.67 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774367.828660 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477356.01 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 147735.60 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922103.429819 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329620.41 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 132962.04 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055065.470861 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196658.37 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 119665.84 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174731.307800 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076992.53 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 107699.25 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282430.561044 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969293.28 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 96929.33 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379359.888964 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872363.95 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 87236.40 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466596.284092 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785127.56 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 78512.76 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545109.039708 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706614.80 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 70661.48 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615770.519761 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635953.32 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 63595.33 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679365.851810 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572357.99 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 57235.80 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736601.650653 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515122.19 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 51512.22 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788113.869612 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463609.97 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 46361.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834474.866676 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417248.97 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 41724.90 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876199.764033 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375524.08 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 37552.41 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913752.171654 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337971.67 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 33797.17 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947549.338513 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304174.50 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 30417.45 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977966.788686 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273757.05 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 27375.71 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005342.493842 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246381.35 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 24638.13 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029980.628482 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221743.21 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 22174.32 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052154.949659 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199568.89 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 19956.89 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072111.838717 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179612.00 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 17961.20 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090073.038870 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161650.80 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 16165.08 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106238.119008 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145485.72 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 14548.57 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120786.691131 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130937.15 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 13093.71 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133880.406043 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117843.43 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 11784.34 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145664.749463 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106059.09 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 10605.91 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156270.658541 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95453.18 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 9545.32 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165815.976712 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85907.86 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 8590.79 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174406.763065 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77317.08 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 7731.71 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182138.470783 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69585.37 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 6958.54 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189097.007729 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62626.83 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 6262.68 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195359.690981 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56364.15 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 5636.41 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200996.105907 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50727.73 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 5072.77 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206068.879341 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45654.96 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 4565.50 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210634.375431 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41089.46 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 4108.95 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214743.321913 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36980.52 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3698.05 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218441.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33282.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222041.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29682.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225641.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26082.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229241.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22482.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232841.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18882.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236441.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15282.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240041.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11682.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243641.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8082.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247241.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4482.47 sec) 07:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250841.373746 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 882.47 sec) 07:34:00 [GPUGRID] [rr_sim] 2251723.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990260604.26G/439.78G) 07:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852004.75 07:34:00 [---] pick_jobs_to_run() start 07:34:00 [---] pick_jobs_to_run() end 07:34:00 [---] [rr_sim] end 07:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:34:00 [---] [cpu_sched_debug] preliminary job list: 07:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:34:00 [---] [cpu_sched_debug] final job list: 07:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:34:00 [---] [cpu_sched_debug] enforce_run_list: end 07:34:00 [---] choose_project(): 27240.000000 07:34:00 [iThena.Measurements] doing RPC to get work 07:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408359.09 cpu 2153492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:34:00 [---] Got 0 tasks 07:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:35:00 [---] [rr_sim] doing sim: CPU sched 07:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990233013.90G/439.78G) 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251661.10 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 225166.11 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225166.110336 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026494.99 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 202649.50 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427815.609639 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823845.49 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 182384.55 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610200.159011 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641460.94 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 164146.09 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774346.253446 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477314.85 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 147731.48 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922077.738438 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329583.36 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 132958.34 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055036.074931 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196625.03 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 119662.50 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174698.577774 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076962.53 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 107696.25 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282394.830333 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969266.27 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 96926.63 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379321.457636 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872339.65 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 87233.96 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466555.422208 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785105.68 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 78510.57 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545065.990324 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706595.11 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 70659.51 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615725.501628 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635935.60 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 63593.56 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679319.061801 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572342.04 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 57234.20 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736553.265957 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515107.84 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 51510.78 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788064.049698 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463597.05 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 46359.71 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834423.755064 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417237.35 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 41723.73 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876147.489894 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375513.61 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 37551.36 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913698.851241 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337962.25 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 33796.23 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947495.076453 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304166.03 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 30416.60 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977911.679144 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273749.42 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 27374.94 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005286.621566 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246374.48 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 24637.45 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029924.069746 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221737.03 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 22173.70 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052097.773108 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199563.33 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 19956.33 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072054.106133 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179607.00 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 17960.70 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090014.805856 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161646.30 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 16164.63 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106179.435607 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145481.67 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 14548.17 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120727.602382 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130933.50 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 13093.35 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133820.952480 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117840.15 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 11784.02 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145604.967569 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106056.14 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 10605.61 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156210.581148 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95450.52 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 9545.05 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165755.633370 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85905.47 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 8590.55 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174346.180369 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77314.92 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 7731.49 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182077.672668 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69583.43 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 6958.34 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189036.015738 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62625.09 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 6262.51 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195298.524500 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56362.58 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 5636.26 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200934.782386 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50726.32 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 5072.63 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206007.414484 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45653.69 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 4565.37 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210572.783372 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41088.32 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 4108.83 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214681.615371 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36979.49 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3697.95 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218379.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33281.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221979.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29681.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225579.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26081.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229179.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22481.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232779.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18881.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236379.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15281.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239979.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11681.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243579.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8081.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247179.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4481.54 sec) 07:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250779.564170 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 881.54 sec) 07:35:00 [GPUGRID] [rr_sim] 2251661.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990233013.90G/439.78G) 07:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852002.02 07:35:00 [---] pick_jobs_to_run() start 07:35:00 [---] pick_jobs_to_run() end 07:35:00 [---] [rr_sim] end 07:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:35:00 [---] [cpu_sched_debug] preliminary job list: 07:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:35:00 [---] [cpu_sched_debug] final job list: 07:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:35:00 [---] [cpu_sched_debug] enforce_run_list: end 07:35:00 [---] choose_project(): 27300.000000 07:35:00 [iThena.Measurements] doing RPC to get work 07:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408299.09 cpu 2153432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:35:00 [---] Got 0 tasks 07:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:36:00 [---] [rr_sim] doing sim: CPU sched 07:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990205423.55G/439.78G) 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251598.37 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 225159.84 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225159.836648 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026438.53 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 202643.85 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427803.689631 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823794.68 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 182379.47 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610183.157316 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641415.21 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 164141.52 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774324.678233 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477273.69 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 147727.37 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922052.047058 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329546.32 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 132954.63 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1055006.679000 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196591.69 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 119659.17 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174665.847748 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076932.52 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 107693.25 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282359.099621 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969239.27 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 96923.93 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379283.026307 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872315.34 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 87231.53 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466514.560324 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785083.81 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 78508.38 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1545022.940940 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706575.43 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 70657.54 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615680.483494 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635917.88 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 63591.79 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679272.271793 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572326.09 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 57232.61 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736504.881262 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515093.49 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 51509.35 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788014.229784 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463584.14 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 46358.41 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834372.643453 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417225.72 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 41722.57 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876095.215756 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375503.15 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 37550.32 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913645.530828 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337952.84 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 33795.28 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947440.814394 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304157.55 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 30415.76 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977856.569602 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273741.80 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 27374.18 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005230.749290 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246367.62 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 24636.76 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029867.511009 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221730.86 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 22173.09 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052040.596556 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199557.77 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 19955.78 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071996.373549 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179601.99 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 17960.20 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089956.572842 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161641.79 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 16164.18 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106120.752206 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145477.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 14547.76 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120668.513633 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130929.85 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 13092.99 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133761.498918 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117836.87 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 11783.69 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145545.185674 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106053.18 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 10605.32 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156150.503755 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95447.86 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 9544.79 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165695.290028 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85903.08 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 8590.31 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174285.597673 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77312.77 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 7731.28 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182016.874554 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69581.49 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 6958.15 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188975.023746 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62623.34 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 6262.33 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195237.358020 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56361.01 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 5636.10 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200873.458866 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50724.91 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 5072.49 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205945.949627 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45652.42 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 4565.24 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210511.191313 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41087.18 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 4108.72 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214619.908829 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36978.46 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3697.85 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218317.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33280.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221917.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29680.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225517.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26080.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229117.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22480.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232717.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18880.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236317.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15280.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239917.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11680.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243517.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8080.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247117.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4480.61 sec) 07:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250717.754595 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 880.61 sec) 07:36:00 [GPUGRID] [rr_sim] 2251598.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990205423.55G/439.78G) 07:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851999.28 07:36:00 [---] pick_jobs_to_run() start 07:36:00 [---] pick_jobs_to_run() end 07:36:00 [---] [rr_sim] end 07:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:36:00 [---] [cpu_sched_debug] preliminary job list: 07:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:36:00 [---] [cpu_sched_debug] final job list: 07:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:36:00 [---] [cpu_sched_debug] enforce_run_list: end 07:36:00 [---] choose_project(): 27360.000000 07:36:00 [iThena.Measurements] doing RPC to get work 07:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 581039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408239.09 cpu 2153372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:36:00 [---] Got 0 tasks 07:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:37:00 [---] [rr_sim] doing sim: CPU sched 07:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990177833.19G/439.78G) 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251535.63 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 225153.56 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225153.562960 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026382.07 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 202638.21 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427791.769624 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823743.86 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 182374.39 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610166.155621 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641369.47 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 164136.95 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774303.103019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477232.53 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 147723.25 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922026.355677 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329509.27 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 132950.93 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054977.283069 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196558.35 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 119655.83 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174633.117722 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076902.51 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 107690.25 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282323.368910 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969212.26 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 96921.23 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379244.594979 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872291.03 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 87229.10 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466473.698441 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785061.93 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 78506.19 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544979.891556 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706555.74 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 70655.57 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615635.465361 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635900.16 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 63590.02 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679225.481784 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572310.15 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 57231.01 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736456.496566 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515079.13 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 51507.91 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787964.409869 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463571.22 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 46357.12 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834321.531842 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417214.10 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 41721.41 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876042.941618 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375492.69 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 37549.27 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913592.210416 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337943.42 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 33794.34 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947386.552334 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304149.08 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 30414.91 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977801.460060 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273734.17 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 27373.42 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005174.877014 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246360.75 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 24636.08 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029810.952273 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221724.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 22172.47 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051983.420005 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199552.21 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 19955.22 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071938.640965 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179596.99 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 17959.70 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089898.339828 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161637.29 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 16163.73 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106062.068805 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145473.56 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 14547.36 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120609.424884 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130926.20 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 13092.62 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133702.045356 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117833.58 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 11783.36 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145485.403780 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106050.23 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 10605.02 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156090.426362 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95445.20 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 9544.52 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165634.946686 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85900.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 8590.07 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174225.014977 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77310.61 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 7731.06 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181956.076439 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69579.55 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 6957.96 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188914.031755 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62621.60 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 6262.16 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195176.191539 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56359.44 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 5635.94 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200812.135345 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50723.49 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 5072.35 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205884.484771 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45651.14 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 4565.11 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210449.599253 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41086.03 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 4108.60 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214558.202288 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36977.43 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3697.74 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218255.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33279.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221855.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29679.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225455.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26079.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229055.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22479.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232655.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18879.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236255.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15279.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239855.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11679.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243455.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8079.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247055.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4479.68 sec) 07:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250655.945019 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 879.68 sec) 07:37:00 [GPUGRID] [rr_sim] 2251535.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990177833.19G/439.78G) 07:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851996.54 07:37:00 [---] pick_jobs_to_run() start 07:37:00 [---] pick_jobs_to_run() end 07:37:00 [---] [rr_sim] end 07:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:37:00 [---] [cpu_sched_debug] preliminary job list: 07:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:37:00 [---] [cpu_sched_debug] final job list: 07:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:37:00 [---] [cpu_sched_debug] enforce_run_list: end 07:37:00 [---] choose_project(): 27420.000000 07:37:00 [iThena.Measurements] doing RPC to get work 07:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408179.09 cpu 2153312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:37:00 [---] Got 0 tasks 07:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:38:00 [---] [rr_sim] doing sim: CPU sched 07:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990150242.83G/439.78G) 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251472.89 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 225147.29 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225147.289272 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026325.60 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 202632.56 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427779.849616 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823693.04 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 182369.30 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610149.153926 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641323.74 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 164132.37 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774281.527805 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477191.36 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 147719.14 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 922000.664296 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329472.23 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 132947.22 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054947.887138 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196525.01 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 119652.50 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174600.387696 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076872.51 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 107687.25 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282287.638198 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969185.25 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 96918.53 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379206.163650 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872266.73 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 87226.67 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466432.836557 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785040.06 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 78504.01 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544936.842173 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706536.05 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 70653.61 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615590.447227 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635882.45 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 63588.24 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679178.691776 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572294.20 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 57229.42 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736408.111870 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515064.78 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 51506.48 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787914.589955 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463558.30 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 46355.83 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834270.420231 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417202.47 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 41720.25 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875990.667479 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375482.23 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 37548.22 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913538.890003 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337934.00 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 33793.40 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947332.290274 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304140.60 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 30414.06 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977746.350519 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273726.54 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 27372.65 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005119.004738 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246353.89 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 24635.39 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029754.393536 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221718.50 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 22171.85 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051926.243454 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199546.65 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 19954.66 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071880.908380 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179591.98 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 17959.20 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089840.106814 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161632.79 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 16163.28 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106003.385404 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145469.51 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 14546.95 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120550.336135 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130922.56 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 13092.26 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133642.591794 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117830.30 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 11783.03 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145425.621886 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106047.27 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 10604.73 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156030.348969 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95442.54 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 9544.25 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165574.603344 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85898.29 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 8589.83 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174164.432281 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77308.46 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 7730.85 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181895.278324 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69577.61 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 6957.76 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188853.039764 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62619.85 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 6261.99 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195115.025059 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56357.87 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 5635.79 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200750.811825 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50722.08 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 5072.21 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205823.019914 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45649.87 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 4564.99 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210388.007194 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41084.89 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 4108.49 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214496.495746 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36976.40 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3697.64 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218194.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33278.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221794.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29678.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225394.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26078.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228994.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22478.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232594.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18878.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236194.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15278.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239794.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11678.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243394.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8078.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246994.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4478.76 sec) 07:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250594.135443 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 878.76 sec) 07:38:00 [GPUGRID] [rr_sim] 2251472.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990150242.83G/439.78G) 07:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851993.81 07:38:00 [---] pick_jobs_to_run() start 07:38:00 [---] pick_jobs_to_run() end 07:38:00 [---] [rr_sim] end 07:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:38:00 [---] [cpu_sched_debug] preliminary job list: 07:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:38:00 [---] [cpu_sched_debug] final job list: 07:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:38:00 [---] [cpu_sched_debug] enforce_run_list: end 07:38:00 [---] choose_project(): 27480.000000 07:38:00 [iThena.Measurements] doing RPC to get work 07:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408119.09 cpu 2153252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:38:00 [---] Got 0 tasks 07:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:39:00 [---] [rr_sim] doing sim: CPU sched 07:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990122652.48G/439.78G) 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251410.16 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 225141.02 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225141.015583 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026269.14 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 202626.91 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427767.929608 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823642.23 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 182364.22 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610132.152231 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641278.00 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 164127.80 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774259.952591 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477150.20 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 147715.02 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921974.972916 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329435.18 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 132943.52 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054918.491208 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196491.66 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 119649.17 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174567.657670 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076842.50 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 107684.25 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282251.907487 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969158.25 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 96915.82 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379167.732321 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872242.42 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 87224.24 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466391.974673 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785018.18 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 78501.82 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544893.792789 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706516.36 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 70651.64 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615545.429093 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635864.73 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 63586.47 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679131.901767 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572278.25 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 57227.83 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736359.727174 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515050.43 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 51505.04 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787864.770040 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463545.39 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 46354.54 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834219.308620 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417190.85 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 41719.08 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875938.393341 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375471.76 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 37547.18 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913485.569590 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337924.59 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 33792.46 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947278.028215 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304132.13 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 30413.21 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977691.240977 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273718.91 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 27371.89 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005063.132462 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246347.02 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 24634.70 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029697.834800 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221712.32 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 22171.23 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051869.066903 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199541.09 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 19954.11 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071823.175796 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179586.98 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 17958.70 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089781.873800 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161628.28 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 16162.83 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105944.702003 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145465.45 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 14546.55 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120491.247386 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130918.91 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 13091.89 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133583.138231 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117827.02 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 11782.70 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145365.839992 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106044.32 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 10604.43 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155970.271576 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95439.88 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 9543.99 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165514.260002 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85895.90 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 8589.59 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174103.849585 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77306.31 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 7730.63 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181834.480210 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69575.68 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 6957.57 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188792.047772 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62618.11 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 6261.81 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195053.858578 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56356.30 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 5635.63 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200689.488304 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50720.67 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 5072.07 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205761.555057 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45648.60 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 4564.86 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210326.415135 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41083.74 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 4108.37 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214434.789205 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36975.37 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3697.54 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218132.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33277.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221732.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29677.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225332.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26077.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228932.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22477.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232532.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18877.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236132.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15277.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239732.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11677.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243332.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8077.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246932.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4477.83 sec) 07:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250532.325868 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 877.83 sec) 07:39:00 [GPUGRID] [rr_sim] 2251410.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990122652.48G/439.78G) 07:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851991.07 07:39:00 [---] pick_jobs_to_run() start 07:39:00 [---] pick_jobs_to_run() end 07:39:00 [---] [rr_sim] end 07:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:39:00 [---] [cpu_sched_debug] preliminary job list: 07:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:39:00 [---] [cpu_sched_debug] final job list: 07:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:39:00 [---] [cpu_sched_debug] enforce_run_list: end 07:39:00 [---] choose_project(): 27540.000000 07:39:00 [iThena.Measurements] doing RPC to get work 07:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 408059.09 cpu 2153192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:39:00 [---] Got 0 tasks 07:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:40:00 [---] [rr_sim] doing sim: CPU sched 07:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990095062.12G/439.78G) 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251347.42 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 225134.74 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225134.741895 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026212.68 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 202621.27 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427756.009601 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823591.41 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 182359.14 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610115.150536 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641232.27 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 164123.23 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774238.377378 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477109.04 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 147710.90 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921949.281535 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329398.14 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 132939.81 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054889.095277 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196458.32 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 119645.83 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174534.927644 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076812.49 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 107681.25 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282216.176775 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969131.24 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 96913.12 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379129.300993 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872218.12 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 87221.81 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466351.112789 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784996.31 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 78499.63 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544850.743405 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706496.68 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 70649.67 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615500.410960 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635847.01 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 63584.70 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679085.111759 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572262.31 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 57226.23 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736311.342478 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515036.08 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 51503.61 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787814.950126 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463532.47 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 46353.25 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834168.197008 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417179.22 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 41717.92 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875886.119203 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375461.30 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 37546.13 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913432.249178 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337915.17 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 33791.52 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947223.766155 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304123.65 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 30412.37 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977636.131435 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273711.29 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 27371.13 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005007.260186 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246340.16 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 24634.02 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029641.276063 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221706.14 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 22170.61 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051811.890352 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199535.53 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 19953.55 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071765.443212 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179581.98 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 17958.20 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089723.640786 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161623.78 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 16162.38 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105886.018603 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145461.40 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 14546.14 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120432.158637 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130915.26 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 13091.53 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133523.684669 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117823.73 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 11782.37 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145306.058097 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106041.36 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 10604.14 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155910.194183 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95437.22 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 9543.72 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165453.916660 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85893.50 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 8589.35 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174043.266889 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77304.15 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 7730.42 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181773.682095 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69573.74 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 6957.37 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188731.055781 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62616.36 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 6261.64 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194992.692098 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56354.73 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 5635.47 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200628.164783 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50719.25 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 5071.93 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205700.090200 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45647.33 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 4564.73 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210264.823075 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41082.60 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 4108.26 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214373.082663 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36974.34 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3697.43 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218070.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33276.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221670.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29676.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225270.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26076.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228870.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22476.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232470.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18876.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236070.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15276.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239670.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11676.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243270.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8076.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246870.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4476.90 sec) 07:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250470.516292 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 876.90 sec) 07:40:00 [GPUGRID] [rr_sim] 2251347.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990095062.12G/439.78G) 07:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851988.33 07:40:00 [---] pick_jobs_to_run() start 07:40:00 [---] pick_jobs_to_run() end 07:40:00 [---] [rr_sim] end 07:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:40:00 [---] [cpu_sched_debug] preliminary job list: 07:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:40:00 [---] [cpu_sched_debug] final job list: 07:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:40:00 [---] [cpu_sched_debug] enforce_run_list: end 07:40:00 [---] choose_project(): 27600.000000 07:40:00 [iThena.Measurements] doing RPC to get work 07:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407999.09 cpu 2153132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:40:00 [---] Got 0 tasks 07:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:41:00 [---] [rr_sim] doing sim: CPU sched 07:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990067471.76G/439.78G) 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251284.68 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 225128.47 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225128.468207 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026156.21 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 202615.62 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427744.089593 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823540.59 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 182354.06 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610098.148841 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641186.53 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 164118.65 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774216.802164 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477067.88 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 147706.79 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921923.590154 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329361.09 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 132936.11 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054859.699346 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196424.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 119642.50 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174502.197618 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076782.48 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 107678.25 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282180.446064 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969104.24 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 96910.42 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379090.869664 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872193.81 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 87219.38 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466310.250905 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784974.43 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 78497.44 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544807.694021 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706476.99 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 70647.70 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615455.392826 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635829.29 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 63582.93 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1679038.321750 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572246.36 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 57224.64 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736262.957782 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515021.72 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 51502.17 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787765.130211 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463519.55 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 46351.96 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834117.085397 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417167.60 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 41716.76 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875833.845064 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375450.84 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 37545.08 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913378.928765 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337905.75 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 33790.58 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947169.504095 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304115.18 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 30411.52 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977581.021893 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273703.66 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 27370.37 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004951.387911 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246333.29 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 24633.33 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029584.717326 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221699.96 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 22170.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051754.713801 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199529.97 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 19953.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071707.710628 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179576.97 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 17957.70 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089665.407772 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161619.27 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 16161.93 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105827.335202 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145457.35 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 14545.73 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120373.069889 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130911.61 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 13091.16 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133464.231107 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117820.45 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 11782.05 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145246.276203 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106038.41 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 10603.84 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155850.116790 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95434.57 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 9543.46 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165393.573318 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85891.11 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 8589.11 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173982.684193 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77302.00 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 7730.20 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181712.883981 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69571.80 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 6957.18 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188670.063790 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62614.62 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 6261.46 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194931.525618 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56353.16 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 5635.32 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200566.841263 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50717.84 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 5071.78 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205638.625343 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45646.06 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 4564.61 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210203.231016 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41081.45 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 4108.15 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214311.376122 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36973.31 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3697.33 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218008.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33275.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221608.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29675.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225208.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26075.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228808.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22475.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232408.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18875.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236008.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15275.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239608.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11675.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243208.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8075.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246808.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4475.98 sec) 07:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250408.706716 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875.98 sec) 07:41:00 [GPUGRID] [rr_sim] 2251284.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990067471.76G/439.78G) 07:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851985.59 07:41:00 [---] pick_jobs_to_run() start 07:41:00 [---] pick_jobs_to_run() end 07:41:00 [---] [rr_sim] end 07:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:41:00 [---] [cpu_sched_debug] preliminary job list: 07:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:41:00 [---] [cpu_sched_debug] final job list: 07:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:41:00 [---] [cpu_sched_debug] enforce_run_list: end 07:41:00 [---] choose_project(): 27660.000000 07:41:00 [iThena.Measurements] doing RPC to get work 07:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407939.09 cpu 2153072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:41:00 [---] Got 0 tasks 07:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:42:00 [---] [rr_sim] doing sim: CPU sched 07:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990039881.41G/439.78G) 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251221.95 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 225122.19 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225122.194519 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026099.75 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 202609.98 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427732.169586 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823489.78 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 182348.98 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610081.147146 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641140.80 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 164114.08 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774195.226950 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1477026.72 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 147702.67 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921897.898774 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329324.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 132932.40 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054830.303415 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196391.64 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 119639.16 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174469.467592 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076752.48 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 107675.25 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282144.715352 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969077.23 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 96907.72 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379052.438336 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872169.51 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 87216.95 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466269.389021 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784952.56 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 78495.26 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544764.644637 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706457.30 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 70645.73 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615410.374692 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635811.57 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 63581.16 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678991.531742 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572230.41 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 57223.04 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736214.573087 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515007.37 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 51500.74 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787715.310297 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463506.63 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 46350.66 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834065.973786 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417155.97 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 41715.60 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875781.570926 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375440.37 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 37544.04 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913325.608352 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337896.34 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 33789.63 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947115.242036 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304106.70 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 30410.67 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977525.912351 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273696.03 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 27369.60 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004895.515635 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246326.43 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 24632.64 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029528.158590 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221693.79 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 22169.38 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051697.537250 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199524.41 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 19952.44 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071649.978043 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179571.97 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 17957.20 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089607.174758 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161614.77 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 16161.48 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105768.651801 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145453.29 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 14545.33 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120313.981140 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130907.96 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 13090.80 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133404.777544 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117817.17 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 11781.72 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145186.494309 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106035.45 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 10603.55 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155790.039397 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95431.91 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 9543.19 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165333.229976 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85888.72 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 8588.87 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173922.101497 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77299.84 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 7729.98 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181652.085866 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69569.86 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 6956.99 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188609.071798 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62612.87 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 6261.29 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194870.359137 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56351.59 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 5635.16 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200505.517742 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50716.43 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 5071.64 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205577.160487 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45644.78 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 4564.48 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210141.638957 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41080.31 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 4108.03 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214249.669580 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36972.28 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3697.23 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217946.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33275.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221546.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29675.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225146.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26075.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228746.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22475.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232346.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18875.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235946.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15275.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239546.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11675.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243146.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8075.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246746.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4475.05 sec) 07:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250346.897141 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 875.05 sec) 07:42:00 [GPUGRID] [rr_sim] 2251221.95: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990039881.41G/439.78G) 07:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851982.86 07:42:00 [---] pick_jobs_to_run() start 07:42:00 [---] pick_jobs_to_run() end 07:42:00 [---] [rr_sim] end 07:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:42:00 [---] [cpu_sched_debug] preliminary job list: 07:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:42:00 [---] [cpu_sched_debug] final job list: 07:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:42:00 [---] [cpu_sched_debug] enforce_run_list: end 07:42:00 [---] choose_project(): 27720.000000 07:42:00 [iThena.Measurements] doing RPC to get work 07:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407879.09 cpu 2153012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2153012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2153012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:42:00 [---] Got 0 tasks 07:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:43:00 [---] [rr_sim] doing sim: CPU sched 07:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (990012291.05G/439.78G) 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251159.21 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 225115.92 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225115.920831 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2026043.29 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 202604.33 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427720.249578 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823438.96 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 182343.90 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610064.145451 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641095.06 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 164109.51 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774173.651736 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476985.56 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 147698.56 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921872.207393 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329287.00 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 132928.70 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054800.907484 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196358.30 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 119635.83 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174436.737567 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076722.47 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 107672.25 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282108.984640 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969050.22 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 96905.02 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1379014.007007 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872145.20 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 87214.52 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466228.527137 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784930.68 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 78493.07 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544721.595254 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706437.61 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 70643.76 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615365.356559 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635793.85 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 63579.39 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678944.741734 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572214.47 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 57221.45 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736166.188391 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514993.02 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 51499.30 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787665.490382 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463493.72 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 46349.37 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834014.862175 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417144.35 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 41714.43 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875729.296788 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375429.91 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 37542.99 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913272.287939 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337886.92 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 33788.69 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947060.979976 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304098.23 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 30409.82 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977470.802809 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273688.41 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 27368.84 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004839.643359 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246319.56 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 24631.96 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029471.599853 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221687.61 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 22168.76 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051640.360699 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199518.85 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 19951.88 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071592.245459 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179566.96 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 17956.70 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089548.941744 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161610.27 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 16161.03 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105709.968400 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145449.24 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 14544.92 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120254.892391 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130904.32 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 13090.43 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133345.323982 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117813.88 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 11781.39 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145126.712414 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106032.50 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 10603.25 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155729.962004 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95429.25 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 9542.92 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165272.886634 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85886.32 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 8588.63 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173861.518801 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77297.69 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 7729.77 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181591.287751 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69567.92 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 6956.79 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188548.079807 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62611.13 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 6261.11 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194809.192657 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56350.02 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 5635.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200444.194222 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50715.01 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 5071.50 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205515.695630 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45643.51 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 4564.35 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210080.046898 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41079.16 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 4107.92 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214187.963038 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36971.25 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3697.12 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217885.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33274.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221485.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29674.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225085.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26074.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228685.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22474.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232285.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18874.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235885.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15274.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239485.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11674.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243085.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8074.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246685.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4474.12 sec) 07:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250285.087565 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 874.12 sec) 07:43:00 [GPUGRID] [rr_sim] 2251159.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (990012291.05G/439.78G) 07:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851980.12 07:43:00 [---] pick_jobs_to_run() start 07:43:00 [---] pick_jobs_to_run() end 07:43:00 [---] [rr_sim] end 07:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:43:00 [---] [cpu_sched_debug] preliminary job list: 07:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:43:00 [---] [cpu_sched_debug] final job list: 07:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:43:00 [---] [cpu_sched_debug] enforce_run_list: end 07:43:00 [---] choose_project(): 27780.000000 07:43:00 [iThena.Measurements] doing RPC to get work 07:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407819.09 cpu 2152952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:43:00 [---] Got 0 tasks 07:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:44:00 [---] [rr_sim] doing sim: CPU sched 07:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989984700.70G/439.78G) 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251096.47 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 225109.65 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225109.647142 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025986.82 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 202598.68 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427708.329570 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823388.14 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 182338.81 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610047.143756 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641049.33 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 164104.93 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774152.076522 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476944.39 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 147694.44 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921846.516013 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329249.96 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 132925.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054771.511554 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196324.96 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 119632.50 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174404.007541 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076692.46 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 107669.25 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282073.253929 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 969023.22 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 96902.32 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378975.575678 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872120.90 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 87212.09 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466187.665253 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784908.81 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 78490.88 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544678.545870 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706417.93 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 70641.79 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615320.338425 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635776.13 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 63577.61 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678897.951725 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572198.52 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 57219.85 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736117.803695 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514978.67 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 51497.87 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787615.670468 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463480.80 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 46348.08 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833963.750563 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417132.72 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 41713.27 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875677.022649 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375419.45 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 37541.94 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913218.967527 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337877.50 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 33787.75 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947006.717916 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304089.75 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 30408.98 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977415.693267 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273680.78 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 27368.08 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004783.771083 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246312.70 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 24631.27 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029415.041117 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221681.43 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 22168.14 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051583.184147 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199513.29 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 19951.33 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071534.512875 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179561.96 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 17956.20 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089490.708730 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161605.76 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 16160.58 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105651.284999 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145445.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 14544.52 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120195.803642 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130900.67 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 13090.07 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133285.870420 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117810.60 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 11781.06 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145066.930520 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106029.54 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 10602.95 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155669.884610 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95426.59 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 9542.66 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165212.543292 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85883.93 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 8588.39 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173800.936105 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77295.54 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 7729.55 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181530.489637 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69565.98 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 6956.60 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188487.087815 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62609.38 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 6260.94 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194748.026176 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56348.45 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 5634.84 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200382.870701 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50713.60 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 5071.36 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205454.230773 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45642.24 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 4564.22 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210018.454838 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41078.02 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 4107.80 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214126.256497 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36970.21 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3697.02 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217823.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33273.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221423.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29673.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225023.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26073.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228623.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22473.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232223.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18873.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235823.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15273.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239423.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11673.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243023.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8073.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246623.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4473.19 sec) 07:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250223.277989 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 873.19 sec) 07:44:00 [GPUGRID] [rr_sim] 2251096.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989984700.70G/439.78G) 07:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851977.38 07:44:00 [---] pick_jobs_to_run() start 07:44:00 [---] pick_jobs_to_run() end 07:44:00 [---] [rr_sim] end 07:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:44:00 [---] [cpu_sched_debug] preliminary job list: 07:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:44:00 [---] [cpu_sched_debug] final job list: 07:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:44:00 [---] [cpu_sched_debug] enforce_run_list: end 07:44:00 [---] choose_project(): 27840.000000 07:44:00 [iThena.Measurements] doing RPC to get work 07:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407759.09 cpu 2152892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:44:00 [---] Got 0 tasks 07:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:45:00 [---] [rr_sim] doing sim: CPU sched 07:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989957110.34G/439.78G) 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2251033.73 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 225103.37 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225103.373454 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025930.36 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 202593.04 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427696.409563 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823337.32 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 182333.73 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610030.142061 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1641003.59 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 164100.36 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774130.501309 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476903.23 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 147690.32 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921820.824632 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329212.91 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 132921.29 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054742.115623 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196291.62 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 119629.16 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174371.277515 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076662.46 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 107666.25 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282037.523217 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968996.21 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 96899.62 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378937.144350 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872096.59 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 87209.66 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466146.803369 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784886.93 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 78488.69 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544635.496486 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706398.24 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 70639.82 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615275.320292 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635758.41 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 63575.84 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678851.161717 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572182.57 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 57218.26 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736069.418999 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514964.32 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 51496.43 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787565.850553 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463467.88 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 46346.79 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833912.638952 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417121.10 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 41712.11 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875624.748511 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375408.99 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 37540.90 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913165.647114 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337868.09 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 33786.81 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946952.455857 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304081.28 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 30408.13 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977360.583725 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273673.15 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 27367.32 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004727.898807 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246305.84 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 24630.58 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029358.482380 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221675.25 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 22167.53 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051526.007596 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199507.73 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 19950.77 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071476.780291 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179556.95 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 17955.70 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089432.475716 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161601.26 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 16160.13 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105592.601598 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145441.13 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 14544.11 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120136.714893 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130897.02 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 13089.70 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133226.416858 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117807.32 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 11780.73 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145007.148626 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106026.59 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 10602.66 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155609.807217 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95423.93 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 9542.39 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165152.199950 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85881.53 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 8588.15 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173740.353409 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77293.38 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 7729.34 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181469.691522 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69564.04 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 6956.40 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188426.095824 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62607.64 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 6260.76 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194686.859696 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56346.87 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 5634.69 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200321.547180 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50712.19 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 5071.22 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205392.765916 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45640.97 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 4564.10 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209956.862779 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41076.87 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 4107.69 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214064.549955 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36969.18 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3696.92 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217761.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33272.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221361.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29672.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224961.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26072.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228561.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22472.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232161.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18872.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235761.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15272.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239361.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11672.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242961.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8072.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246561.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4472.27 sec) 07:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250161.468414 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872.27 sec) 07:45:00 [GPUGRID] [rr_sim] 2251033.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989957110.34G/439.78G) 07:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851974.65 07:45:00 [---] pick_jobs_to_run() start 07:45:00 [---] pick_jobs_to_run() end 07:45:00 [---] [rr_sim] end 07:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:45:00 [---] [cpu_sched_debug] preliminary job list: 07:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:45:00 [---] [cpu_sched_debug] final job list: 07:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:45:00 [---] [cpu_sched_debug] enforce_run_list: end 07:45:00 [---] choose_project(): 27900.000000 07:45:00 [iThena.Measurements] doing RPC to get work 07:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407699.09 cpu 2152832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:45:00 [---] Got 0 tasks 07:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:46:00 [---] [rr_sim] doing sim: CPU sched 07:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989929519.98G/439.78G) 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250971.00 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 225097.10 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225097.099766 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025873.90 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 202587.39 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427684.489555 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823286.51 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 182328.65 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 610013.140366 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640957.86 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 164095.79 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774108.926095 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476862.07 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 147686.21 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921795.133251 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329175.86 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 132917.59 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054712.719692 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196258.28 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 119625.83 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174338.547489 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076632.45 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 107663.25 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1282001.792506 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968969.21 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 96896.92 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378898.713021 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872072.28 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 87207.23 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466105.941485 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784865.06 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 78486.51 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544592.447102 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706378.55 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 70637.86 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615230.302158 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635740.70 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 63574.07 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678804.371708 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572166.63 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 57216.66 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1736021.034303 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514949.96 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 51495.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787516.030639 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463454.97 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 46345.50 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833861.527341 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417109.47 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 41710.95 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875572.474373 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375398.52 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 37539.85 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913112.326701 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337858.67 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 33785.87 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946898.193797 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304072.80 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 30407.28 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977305.474183 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273665.52 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 27366.55 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004672.026531 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246298.97 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 24629.90 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029301.923644 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221669.07 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 22166.91 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051468.831045 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199502.17 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 19950.22 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071419.047707 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179551.95 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 17955.19 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089374.242702 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161596.75 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 16159.68 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105533.918198 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145437.08 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 14543.71 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120077.626144 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130893.37 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 13089.34 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133166.963295 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117804.03 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 11780.40 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144947.366732 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106023.63 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 10602.36 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155549.729824 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95421.27 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 9542.13 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165091.856608 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85879.14 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 8587.91 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173679.770713 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77291.23 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 7729.12 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181408.893408 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69562.10 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 6956.21 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188365.103833 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62605.89 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 6260.59 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194625.693215 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56345.30 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 5634.53 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200260.223660 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50710.77 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 5071.08 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205331.301060 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45639.70 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 4563.97 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209895.270720 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41075.73 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 4107.57 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214002.843414 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36968.15 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3696.82 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217699.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33271.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221299.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29671.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224899.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26071.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228499.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22471.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232099.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18871.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235699.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15271.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239299.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11671.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242899.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8071.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246499.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4471.34 sec) 07:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250099.658838 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871.34 sec) 07:46:00 [GPUGRID] [rr_sim] 2250971.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989929519.98G/439.78G) 07:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851971.91 07:46:00 [---] pick_jobs_to_run() start 07:46:00 [---] pick_jobs_to_run() end 07:46:00 [---] [rr_sim] end 07:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:46:00 [---] [cpu_sched_debug] preliminary job list: 07:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:46:00 [---] [cpu_sched_debug] final job list: 07:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:46:00 [---] [cpu_sched_debug] enforce_run_list: end 07:46:00 [---] choose_project(): 27960.000000 07:46:00 [iThena.Measurements] doing RPC to get work 07:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407639.09 cpu 2152772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:46:00 [---] Got 0 tasks 07:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:47:00 [---] [rr_sim] doing sim: CPU sched 07:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989901929.63G/439.78G) 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250908.26 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 225090.83 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225090.826078 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025817.43 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 202581.74 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427672.569548 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823235.69 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 182323.57 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609996.138671 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640912.12 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 164091.21 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774087.350881 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476820.91 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 147682.09 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921769.441871 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329138.82 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 132913.88 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054683.323761 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196224.94 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 119622.49 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174305.817463 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076602.44 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 107660.24 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281966.061794 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968942.20 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 96894.22 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378860.281693 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872047.98 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 87204.80 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466065.079601 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784843.18 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 78484.32 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544549.397719 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706358.86 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 70635.89 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615185.284024 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635722.98 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 63572.30 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678757.581700 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572150.68 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 57215.07 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735972.649607 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514935.61 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 51493.56 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787466.210724 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463442.05 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 46344.21 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833810.415730 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417097.85 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 41709.78 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875520.200234 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375388.06 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 37538.81 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913059.006289 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337849.25 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 33784.93 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946843.931737 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304064.33 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 30406.43 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977250.364641 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273657.90 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 27365.79 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004616.154255 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246292.11 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 24629.21 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029245.364907 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221662.90 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 22166.29 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051411.654494 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199496.61 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 19949.66 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071361.315122 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179546.95 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 17954.69 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089316.009688 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161592.25 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 16159.23 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105475.234797 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145433.03 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 14543.30 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120018.537395 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130889.72 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 13088.97 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133107.509733 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117800.75 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 11780.08 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144887.584837 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106020.68 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 10602.07 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155489.652431 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95418.61 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 9541.86 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165031.513266 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85876.75 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 8587.67 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173619.188017 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77289.07 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 7728.91 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181348.095293 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69560.17 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 6956.02 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188304.111841 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62604.15 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 6260.41 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194564.526735 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56343.73 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 5634.37 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200198.900139 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50709.36 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 5070.94 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205269.836203 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45638.42 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 4563.84 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209833.678660 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41074.58 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 4107.46 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213941.136872 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36967.12 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3696.71 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217637.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33270.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221237.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29670.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224837.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26070.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228437.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22470.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232037.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18870.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235637.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15270.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239237.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11670.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242837.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8070.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246437.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4470.41 sec) 07:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2250037.849262 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870.41 sec) 07:47:00 [GPUGRID] [rr_sim] 2250908.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989901929.63G/439.78G) 07:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851969.17 07:47:00 [---] pick_jobs_to_run() start 07:47:00 [---] pick_jobs_to_run() end 07:47:00 [---] [rr_sim] end 07:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:47:00 [---] [cpu_sched_debug] preliminary job list: 07:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:47:00 [---] [cpu_sched_debug] final job list: 07:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:47:00 [---] [cpu_sched_debug] enforce_run_list: end 07:47:00 [---] choose_project(): 28020.000000 07:47:00 [iThena.Measurements] doing RPC to get work 07:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407579.09 cpu 2152712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:47:00 [---] Got 0 tasks 07:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:48:00 [---] [rr_sim] doing sim: CPU sched 07:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989874339.27G/439.78G) 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250845.52 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 225084.55 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225084.552389 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025760.97 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 202576.10 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427660.649540 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823184.87 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 182318.49 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609979.136975 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640866.39 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 164086.64 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774065.775667 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476779.75 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 147677.97 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921743.750490 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329101.77 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 132910.18 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054653.927831 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196191.60 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 119619.16 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174273.087437 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076572.44 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 107657.24 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281930.331083 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968915.19 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 96891.52 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378821.850364 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 872023.67 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 87202.37 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1466024.217717 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784821.31 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 78482.13 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544506.348335 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706339.18 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 70633.92 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615140.265891 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635705.26 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 63570.53 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678710.791691 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572134.73 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 57213.47 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735924.264912 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514921.26 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 51492.13 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787416.390810 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463429.13 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 46342.91 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833759.304118 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417086.22 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 41708.62 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875467.926096 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375377.60 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 37537.76 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913005.685876 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337839.84 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 33783.98 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946789.669678 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304055.85 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 30405.59 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977195.255099 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273650.27 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 27365.03 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004560.281979 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246285.24 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 24628.52 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029188.806171 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221656.72 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 22165.67 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051354.477943 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199491.05 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 19949.10 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071303.582538 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179541.94 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 17954.19 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089257.776674 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161587.75 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 16158.77 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105416.551396 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145428.97 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 14542.90 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119959.448646 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130886.08 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 13088.61 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133048.056171 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117797.47 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 11779.75 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144827.802943 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106017.72 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 10601.77 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155429.575038 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95415.95 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 9541.59 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164971.169924 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85874.35 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 8587.44 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173558.605321 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77286.92 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 7728.69 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181287.297178 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69558.23 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 6955.82 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188243.119850 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62602.40 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 6260.24 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194503.360254 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56342.16 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 5634.22 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200137.576618 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50707.95 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 5070.79 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205208.371346 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45637.15 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 4563.72 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209772.086601 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41073.44 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 4107.34 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213879.430330 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36966.09 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3696.61 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217576.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33269.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221176.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29669.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224776.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26069.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228376.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22469.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231976.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18869.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235576.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15269.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239176.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11669.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242776.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8069.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246376.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4469.48 sec) 07:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249976.039687 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869.48 sec) 07:48:00 [GPUGRID] [rr_sim] 2250845.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989874339.27G/439.78G) 07:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851966.44 07:48:00 [---] pick_jobs_to_run() start 07:48:00 [---] pick_jobs_to_run() end 07:48:00 [---] [rr_sim] end 07:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:48:00 [---] [cpu_sched_debug] preliminary job list: 07:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:48:00 [---] [cpu_sched_debug] final job list: 07:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:48:00 [---] [cpu_sched_debug] enforce_run_list: end 07:48:00 [---] choose_project(): 28080.000000 07:48:00 [iThena.Measurements] doing RPC to get work 07:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407519.09 cpu 2152652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:48:00 [---] Got 0 tasks 07:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:49:00 [---] [rr_sim] doing sim: CPU sched 07:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989846748.91G/439.78G) 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250782.79 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 225078.28 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225078.278701 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025704.51 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 202570.45 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427648.729532 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823134.06 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 182313.41 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609962.135280 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640820.65 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 164082.07 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774044.200454 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476738.59 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 147673.86 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921718.059110 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329064.73 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 132906.47 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054624.531900 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196158.26 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 119615.83 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174240.357411 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076542.43 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 107654.24 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281894.600371 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968888.19 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 96888.82 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378783.419035 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871999.37 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 87199.94 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465983.355833 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784799.43 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 78479.94 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544463.298951 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706319.49 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 70631.95 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615095.247757 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635687.54 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 63568.75 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678664.001683 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572118.79 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 57211.88 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735875.880216 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514906.91 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 51490.69 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787366.570895 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463416.22 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 46341.62 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833708.192507 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417074.59 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 41707.46 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875415.651958 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375367.14 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 37536.71 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912952.365463 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337830.42 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 33783.04 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946735.407618 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304047.38 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 30404.74 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977140.145558 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273642.64 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 27364.26 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004504.409703 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246278.38 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 24627.84 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029132.247434 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221650.54 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 22165.05 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051297.301392 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199485.49 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 19948.55 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071245.849954 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179536.94 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 17953.69 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089199.543660 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161583.24 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 16158.32 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105357.867995 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145424.92 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 14542.49 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119900.359897 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130882.43 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 13088.24 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132988.602608 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117794.18 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 11779.42 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144768.021049 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106014.77 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 10601.48 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155369.497645 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95413.29 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 9541.33 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164910.826582 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85871.96 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 8587.20 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173498.022625 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77284.76 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 7728.48 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181226.499064 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69556.29 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 6955.63 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188182.127859 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62600.66 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 6260.07 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194442.193774 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56340.59 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 5634.06 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200076.253098 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50706.53 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 5070.65 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205146.906489 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45635.88 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 4563.59 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209710.494542 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41072.29 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 4107.23 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213817.723789 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36965.06 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3696.51 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217514.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33268.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221114.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29668.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224714.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26068.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228314.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22468.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231914.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18868.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235514.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15268.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239114.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11668.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242714.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8068.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246314.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4468.56 sec) 07:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249914.230111 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868.56 sec) 07:49:00 [GPUGRID] [rr_sim] 2250782.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989846748.91G/439.78G) 07:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851963.70 07:49:00 [---] pick_jobs_to_run() start 07:49:00 [---] pick_jobs_to_run() end 07:49:00 [---] [rr_sim] end 07:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:49:00 [---] [cpu_sched_debug] preliminary job list: 07:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:49:00 [---] [cpu_sched_debug] final job list: 07:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:49:00 [---] [cpu_sched_debug] enforce_run_list: end 07:49:00 [---] choose_project(): 28140.000000 07:49:00 [iThena.Measurements] doing RPC to get work 07:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407459.09 cpu 2152592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:49:00 [---] Got 0 tasks 07:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:50:00 [---] [rr_sim] doing sim: CPU sched 07:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989819158.56G/439.78G) 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250720.05 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 225072.01 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225072.005013 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025648.05 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 202564.80 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427636.809525 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823083.24 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 182308.32 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609945.133585 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640774.92 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 164077.49 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774022.625240 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476697.42 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 147669.74 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921692.367729 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1329027.68 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 132902.77 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054595.135969 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196124.91 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 119612.49 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174207.627385 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076512.42 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 107651.24 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281858.869660 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968861.18 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 96886.12 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378744.987707 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871975.06 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 87197.51 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465942.493949 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784777.56 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 78477.76 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544420.249567 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706299.80 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 70629.98 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615050.229624 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635669.82 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 63566.98 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678617.211674 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572102.84 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 57210.28 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735827.495520 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514892.55 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 51489.26 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787316.750981 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463403.30 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 46340.33 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833657.080896 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417062.97 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 41706.30 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875363.377819 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375356.67 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 37535.67 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912899.045050 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337821.01 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 33782.10 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946681.145558 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304038.90 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 30403.89 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977085.036016 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273635.01 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 27363.50 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004448.537427 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246271.51 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 24627.15 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029075.688697 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221644.36 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 22164.44 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051240.124841 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199479.93 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 19947.99 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071188.117370 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179531.93 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 17953.19 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089141.310646 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161578.74 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 16157.87 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105299.184594 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145420.87 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 14542.09 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119841.271148 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130878.78 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 13087.88 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132929.149046 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117790.90 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 11779.09 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144708.239155 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106011.81 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 10601.18 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155309.420252 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95410.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 9541.06 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164850.483240 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85869.57 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 8586.96 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173437.439929 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77282.61 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 7728.26 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181165.700949 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69554.35 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 6955.43 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188121.135867 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62598.91 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 6259.89 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194381.027294 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56339.02 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 5633.90 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200014.929577 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50705.12 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 5070.51 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205085.441633 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45634.61 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 4563.46 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209648.902482 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41071.15 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 4107.11 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213756.017247 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36964.03 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3696.40 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217452.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33267.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221052.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29667.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224652.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26067.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228252.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22467.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231852.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18867.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235452.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15267.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239052.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11667.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242652.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8067.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246252.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4467.63 sec) 07:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249852.420535 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867.63 sec) 07:50:00 [GPUGRID] [rr_sim] 2250720.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989819158.56G/439.78G) 07:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851960.96 07:50:00 [---] pick_jobs_to_run() start 07:50:00 [---] pick_jobs_to_run() end 07:50:00 [---] [rr_sim] end 07:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:50:00 [---] [cpu_sched_debug] preliminary job list: 07:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:50:00 [---] [cpu_sched_debug] final job list: 07:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:50:00 [---] [cpu_sched_debug] enforce_run_list: end 07:50:00 [---] choose_project(): 28200.000000 07:50:00 [iThena.Measurements] doing RPC to get work 07:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407399.09 cpu 2152532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:50:00 [---] Got 0 tasks 07:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:51:00 [---] [rr_sim] doing sim: CPU sched 07:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989791568.20G/439.78G) 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250657.31 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 225065.73 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225065.731325 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025591.58 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 202559.16 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427624.889517 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1823032.42 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 182303.24 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609928.131890 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640729.18 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 164072.92 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 774001.050026 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476656.26 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 147665.63 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921666.676348 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328990.64 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 132899.06 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054565.740038 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196091.57 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 119609.16 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174174.897359 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076482.42 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 107648.24 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281823.138948 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968834.17 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 96883.42 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378706.556378 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871950.76 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 87195.08 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465901.632065 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784755.68 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 78475.57 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544377.200183 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706280.11 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 70628.01 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1615005.211490 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635652.10 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 63565.21 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678570.421666 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572086.89 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 57208.69 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735779.110824 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514878.20 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 51487.82 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787266.931066 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463390.38 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 46339.04 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833605.969285 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417051.34 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 41705.13 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875311.103681 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375346.21 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 37534.62 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912845.724638 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337811.59 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 33781.16 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946626.883499 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304030.43 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 30403.04 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977029.926474 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273627.39 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 27362.74 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004392.665151 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246264.65 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 24626.46 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029019.129961 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221638.18 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 22163.82 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051182.948290 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199474.36 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 19947.44 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071130.384785 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179526.93 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 17952.69 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089083.077632 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161574.24 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 16157.42 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105240.501193 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145416.81 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 14541.68 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119782.182399 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130875.13 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 13087.51 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132869.695484 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117787.62 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 11778.76 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144648.457260 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106008.86 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 10600.89 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155249.342859 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95407.97 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 9540.80 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164790.139898 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85867.17 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 8586.72 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173376.857233 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77280.46 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 7728.05 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181104.902835 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69552.41 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 6955.24 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188060.143876 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62597.17 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 6259.72 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194319.860813 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56337.45 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 5633.75 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199953.606057 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50703.71 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 5070.37 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205023.976776 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45633.34 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 4563.33 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209587.310423 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41070.00 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 4107.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213694.310706 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36963.00 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3696.30 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217390.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33266.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220990.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29666.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224590.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26066.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228190.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22466.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231790.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18866.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235390.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15266.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238990.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11666.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242590.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8066.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246190.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4466.70 sec) 07:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249790.610960 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866.70 sec) 07:51:00 [GPUGRID] [rr_sim] 2250657.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989791568.20G/439.78G) 07:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851958.23 07:51:00 [---] pick_jobs_to_run() start 07:51:00 [---] pick_jobs_to_run() end 07:51:00 [---] [rr_sim] end 07:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:51:00 [---] [cpu_sched_debug] preliminary job list: 07:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:51:00 [---] [cpu_sched_debug] final job list: 07:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:51:00 [---] [cpu_sched_debug] enforce_run_list: end 07:51:00 [---] choose_project(): 28260.000000 07:51:00 [iThena.Measurements] doing RPC to get work 07:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407339.09 cpu 2152472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:51:00 [---] Got 0 tasks 07:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:52:00 [---] [rr_sim] doing sim: CPU sched 07:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989763977.85G/439.78G) 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250594.58 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 225059.46 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225059.457637 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025535.12 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 202553.51 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427612.969510 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822981.61 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 182298.16 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609911.130195 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640683.45 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 164068.34 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773979.474812 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476615.10 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 147661.51 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921640.984968 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328953.59 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 132895.36 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054536.344108 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196058.23 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 119605.82 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174142.167333 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076452.41 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 107645.24 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281787.408237 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968807.17 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 96880.72 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378668.125050 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871926.45 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 87192.65 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465860.770181 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784733.81 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 78473.38 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544334.150800 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706260.43 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 70626.04 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614960.193356 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635634.38 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 63563.44 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678523.631657 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572070.94 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 57207.09 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735730.726128 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514863.85 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 51486.39 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787217.111152 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463377.47 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 46337.75 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833554.857673 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417039.72 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 41703.97 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875258.829543 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375335.75 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 37533.57 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912792.404225 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337802.17 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 33780.22 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946572.621439 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304021.95 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 30402.20 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976974.816932 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273619.76 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 27361.98 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004336.792875 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246257.78 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 24625.78 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028962.571224 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221632.01 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 22163.20 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051125.771738 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199468.80 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 19946.88 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071072.652201 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179521.92 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 17952.19 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089024.844618 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161569.73 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 16156.97 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105181.817793 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145412.76 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 14541.28 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119723.093650 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130871.48 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 13087.15 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132810.241922 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117784.33 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 11778.43 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144588.675366 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106005.90 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 10600.59 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155189.265466 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95405.31 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 9540.53 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164729.796556 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85864.78 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 8586.48 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173316.274537 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77278.30 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 7727.83 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181044.104720 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69550.47 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 6955.05 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187999.151885 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62595.42 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 6259.54 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194258.694333 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56335.88 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 5633.59 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199892.282536 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50702.29 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 5070.23 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204962.511919 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45632.06 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 4563.21 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209525.718364 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41068.86 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 4106.89 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213632.604164 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36961.97 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3696.20 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217328.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33265.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220928.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29665.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224528.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26065.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228128.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22465.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231728.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18865.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235328.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15265.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238928.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11665.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242528.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8065.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246128.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4465.77 sec) 07:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249728.801384 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865.77 sec) 07:52:00 [GPUGRID] [rr_sim] 2250594.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989763977.85G/439.78G) 07:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851955.49 07:52:00 [---] pick_jobs_to_run() start 07:52:00 [---] pick_jobs_to_run() end 07:52:00 [---] [rr_sim] end 07:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:52:00 [---] [cpu_sched_debug] preliminary job list: 07:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:52:00 [---] [cpu_sched_debug] final job list: 07:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:52:00 [---] [cpu_sched_debug] enforce_run_list: end 07:52:00 [---] choose_project(): 28320.000000 07:52:00 [iThena.Measurements] doing RPC to get work 07:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407279.09 cpu 2152412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:52:00 [---] Got 0 tasks 07:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:53:00 [---] [rr_sim] doing sim: CPU sched 07:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989736387.49G/439.78G) 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250531.84 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 225053.18 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225053.183948 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025478.66 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 202547.87 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427601.049502 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822930.79 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 182293.08 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609894.128500 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640637.71 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 164063.77 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773957.899599 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476573.94 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 147657.39 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921615.293587 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328916.55 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 132891.65 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054506.948177 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1196024.89 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 119602.49 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174109.437307 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076422.40 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 107642.24 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281751.677525 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968780.16 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 96878.02 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378629.693721 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871902.15 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 87190.21 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465819.908297 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784711.93 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 78471.19 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544291.101416 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706240.74 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 70624.07 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614915.175223 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635616.66 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 63561.67 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678476.841649 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572055.00 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 57205.50 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735682.341432 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514849.50 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 51484.95 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787167.291237 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463364.55 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 46336.45 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833503.746062 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417028.09 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 41702.81 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875206.555404 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375325.28 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 37532.53 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912739.083812 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337792.76 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 33779.28 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946518.359379 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304013.48 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 30401.35 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976919.707390 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273612.13 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 27361.21 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004280.920599 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246250.92 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 24625.09 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028906.012488 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221625.83 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 22162.58 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051068.595187 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199463.24 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 19946.32 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071014.919617 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179516.92 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 17951.69 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088966.611604 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161565.23 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 16156.52 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105123.134392 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145408.71 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 14540.87 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119664.004901 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130867.83 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 13086.78 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132750.788359 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117781.05 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 11778.11 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144528.893472 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106002.95 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 10600.29 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155129.188073 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95402.65 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 9540.27 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164669.453214 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85862.39 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 8586.24 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173255.691841 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77276.15 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 7727.61 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180983.306605 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69548.53 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 6954.85 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187938.159893 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62593.68 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 6259.37 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194197.527852 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56334.31 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 5633.43 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199830.959015 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50700.88 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 5070.09 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204901.047062 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45630.79 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 4563.08 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209464.126304 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41067.71 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 4106.77 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213570.897622 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36960.94 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3696.09 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217266.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33264.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220866.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29664.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224466.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26064.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228066.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22464.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231666.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18864.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235266.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15264.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238866.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11664.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242466.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8064.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246066.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4464.85 sec) 07:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249666.991809 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864.85 sec) 07:53:00 [GPUGRID] [rr_sim] 2250531.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989736387.49G/439.78G) 07:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851952.75 07:53:00 [---] pick_jobs_to_run() start 07:53:00 [---] pick_jobs_to_run() end 07:53:00 [---] [rr_sim] end 07:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:53:00 [---] [cpu_sched_debug] preliminary job list: 07:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:53:00 [---] [cpu_sched_debug] final job list: 07:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:53:00 [---] [cpu_sched_debug] enforce_run_list: end 07:53:00 [---] choose_project(): 28380.000000 07:53:00 [iThena.Measurements] doing RPC to get work 07:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 580019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407219.09 cpu 2152352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:53:00 [---] Got 0 tasks 07:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:54:00 [---] [rr_sim] doing sim: CPU sched 07:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989708797.13G/439.78G) 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250469.10 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 225046.91 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225046.910260 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025422.19 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 202542.22 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427589.129494 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822879.97 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 182288.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609877.126805 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640591.98 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 164059.20 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773936.324385 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476532.78 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 147653.28 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921589.602206 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328879.50 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 132887.95 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054477.552246 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195991.55 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 119599.16 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174076.707282 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076392.40 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 107639.24 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281715.946814 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968753.16 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 96875.32 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378591.262392 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871877.84 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 87187.78 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465779.046413 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784690.06 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 78469.01 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544248.052032 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706221.05 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 70622.11 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614870.157089 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635598.95 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 63559.89 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678430.051640 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572039.05 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 57203.91 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735633.956737 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514835.15 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 51483.51 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787117.471323 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463351.63 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 46335.16 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833452.634451 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417016.47 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 41701.65 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875154.281266 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375314.82 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 37531.48 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912685.763400 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337783.34 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 33778.33 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946464.097320 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304005.01 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 30400.50 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976864.597848 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273604.50 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 27360.45 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004225.048323 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246244.05 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 24624.41 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028849.453751 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221619.65 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 22161.96 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051011.418636 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199457.68 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 19945.77 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070957.187033 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179511.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 17951.19 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088908.378590 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161560.72 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 16156.07 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105064.450991 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145404.65 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 14540.47 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119604.916152 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130864.19 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 13086.42 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132691.334797 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117777.77 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 11777.78 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144469.111577 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105999.99 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 10600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155069.110680 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95399.99 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 9540.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164609.109872 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85859.99 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 8586.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173195.109145 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77273.99 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 7727.40 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180922.508491 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69546.59 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 6954.66 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187877.167902 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62591.93 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 6259.19 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194136.361372 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56332.74 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 5633.27 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199769.635495 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50699.47 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 5069.95 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204839.582205 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45629.52 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 4562.95 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209402.534245 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41066.57 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 4106.66 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213509.191081 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36959.91 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3695.99 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217205.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33263.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220805.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29663.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224405.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26063.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228005.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22463.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231605.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18863.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235205.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15263.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238805.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11663.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242405.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8063.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246005.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4463.92 sec) 07:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249605.182233 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863.92 sec) 07:54:00 [GPUGRID] [rr_sim] 2250469.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989708797.13G/439.78G) 07:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851950.02 07:54:00 [---] pick_jobs_to_run() start 07:54:00 [---] pick_jobs_to_run() end 07:54:00 [---] [rr_sim] end 07:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:54:00 [---] [cpu_sched_debug] preliminary job list: 07:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:54:00 [---] [cpu_sched_debug] final job list: 07:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:54:00 [---] [cpu_sched_debug] enforce_run_list: end 07:54:00 [---] choose_project(): 28440.000000 07:54:00 [iThena.Measurements] doing RPC to get work 07:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407159.09 cpu 2152292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:54:00 [---] Got 0 tasks 07:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:55:00 [---] [rr_sim] doing sim: CPU sched 07:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989681206.78G/439.78G) 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250406.37 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 225040.64 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225040.636572 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025365.73 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 202536.57 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427577.209487 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822829.16 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 182282.92 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609860.125110 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640546.24 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 164054.62 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773914.749171 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476491.62 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 147649.16 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921563.910826 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328842.45 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 132884.25 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054448.156315 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195958.21 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 119595.82 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174043.977256 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076362.39 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 107636.24 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281680.216102 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968726.15 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 96872.61 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378552.831064 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871853.53 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 87185.35 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465738.184529 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784668.18 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 78466.82 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544205.002648 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706201.36 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 70620.14 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614825.138955 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635581.23 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 63558.12 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678383.261632 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572023.10 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 57202.31 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735585.572041 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514820.79 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 51482.08 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787067.651409 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463338.71 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 46333.87 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833401.522840 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417004.84 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 41700.48 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875102.007128 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375304.36 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 37530.44 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912632.442987 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337773.92 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 33777.39 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946409.835260 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303996.53 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 30399.65 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976809.488306 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273596.88 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 27359.69 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004169.176047 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246237.19 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 24623.72 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028792.895015 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221613.47 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 22161.35 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050954.242085 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199452.12 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 19945.21 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070899.454449 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179506.91 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 17950.69 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088850.145576 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161556.22 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 16155.62 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105005.767590 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145400.60 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 14540.06 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119545.827403 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130860.54 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 13086.05 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132631.881235 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117774.48 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 11777.45 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144409.329683 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105997.04 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 10599.70 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155009.033287 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95397.33 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 9539.73 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164548.766530 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85857.60 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 8585.76 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173134.526449 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77271.84 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 7727.18 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180861.710376 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69544.66 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 6954.47 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187816.175910 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62590.19 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 6259.02 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194075.194891 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56331.17 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 5633.12 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199708.311974 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50698.05 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 5069.81 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204778.117349 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45628.25 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 4562.82 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209340.942186 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41065.42 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 4106.54 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213447.484539 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36958.88 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3695.89 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217143.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33262.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220743.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29662.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224343.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26062.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227943.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22462.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231543.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18862.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235143.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15262.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238743.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11662.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242343.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8062.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245943.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4462.99 sec) 07:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249543.372657 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862.99 sec) 07:55:00 [GPUGRID] [rr_sim] 2250406.37: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989681206.78G/439.78G) 07:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851947.28 07:55:00 [---] pick_jobs_to_run() start 07:55:00 [---] pick_jobs_to_run() end 07:55:00 [---] [rr_sim] end 07:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:55:00 [---] [cpu_sched_debug] preliminary job list: 07:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:55:00 [---] [cpu_sched_debug] final job list: 07:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:55:00 [---] [cpu_sched_debug] enforce_run_list: end 07:55:00 [---] choose_project(): 28500.000000 07:55:00 [iThena.Measurements] doing RPC to get work 07:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407099.09 cpu 2152232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:55:00 [---] Got 0 tasks 07:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:56:00 [---] [rr_sim] doing sim: CPU sched 07:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989653616.42G/439.78G) 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250343.63 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 225034.36 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225034.362884 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025309.27 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 202530.93 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427565.289479 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822778.34 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 182277.83 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609843.123415 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640500.51 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 164050.05 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773893.173957 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476450.45 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 147645.05 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921538.219445 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328805.41 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 132880.54 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054418.760384 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195924.87 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 119592.49 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1174011.247230 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076332.38 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 107633.24 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281644.485390 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968699.14 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 96869.91 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378514.399735 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871829.23 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 87182.92 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465697.322645 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784646.31 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 78464.63 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544161.953265 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706181.68 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 70618.17 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614780.120822 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635563.51 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 63556.35 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678336.471623 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572007.16 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 57200.72 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735537.187345 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514806.44 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 51480.64 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787017.831494 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463325.80 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 46332.58 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833350.411228 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416993.22 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 41699.32 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875049.732989 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375293.90 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 37529.39 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912579.122574 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337764.51 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 33776.45 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946355.573200 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303988.06 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 30398.81 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976754.378764 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273589.25 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 27358.93 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004113.303771 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246230.33 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 24623.03 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028736.336278 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221607.29 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 22160.73 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050897.065534 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199446.56 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 19944.66 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070841.721864 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179501.91 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 17950.19 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088791.912562 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161551.72 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 16155.17 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104947.084189 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145396.54 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 14539.65 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119486.738654 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130856.89 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 13085.69 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132572.427672 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117771.20 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 11777.12 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144349.547789 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105994.08 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 10599.41 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154948.955894 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95394.67 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 9539.47 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164488.423188 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85855.21 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 8585.52 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173073.943753 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77269.69 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 7726.97 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180800.912261 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69542.72 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 6954.27 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187755.183919 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62588.44 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 6258.84 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194014.028411 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56329.60 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 5632.96 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199646.988454 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50696.64 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 5069.66 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204716.652492 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45626.98 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 4562.70 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209279.350126 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41064.28 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 4106.43 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213385.777998 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36957.85 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3695.79 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217081.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33262.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220681.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29662.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224281.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26062.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227881.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22462.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231481.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18862.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235081.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15262.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238681.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11662.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242281.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8062.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245881.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4462.07 sec) 07:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249481.563082 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862.07 sec) 07:56:00 [GPUGRID] [rr_sim] 2250343.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989653616.42G/439.78G) 07:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851944.54 07:56:00 [---] pick_jobs_to_run() start 07:56:00 [---] pick_jobs_to_run() end 07:56:00 [---] [rr_sim] end 07:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:56:00 [---] [cpu_sched_debug] preliminary job list: 07:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:56:00 [---] [cpu_sched_debug] final job list: 07:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:56:00 [---] [cpu_sched_debug] enforce_run_list: end 07:56:00 [---] choose_project(): 28560.000000 07:56:00 [iThena.Measurements] doing RPC to get work 07:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 407039.09 cpu 2152172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:56:00 [---] Got 0 tasks 07:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:57:00 [---] [rr_sim] doing sim: CPU sched 07:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989626026.06G/439.78G) 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250280.89 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 225028.09 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225028.089196 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025252.80 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 202525.28 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427553.369471 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822727.52 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 182272.75 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609826.121720 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640454.77 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 164045.48 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773871.598743 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476409.29 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 147640.93 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921512.528065 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328768.36 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 132876.84 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054389.364454 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195891.53 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 119589.15 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173978.517204 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076302.37 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 107630.24 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281608.754679 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968672.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 96867.21 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378475.968407 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871804.92 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 87180.49 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465656.460761 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784624.43 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 78462.44 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544118.903881 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706161.99 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 70616.20 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614735.102688 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635545.79 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 63554.58 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678289.681615 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571991.21 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 57199.12 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735488.802649 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514792.09 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 51479.21 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786968.011580 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463312.88 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 46331.29 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833299.299617 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416981.59 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 41698.16 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874997.458851 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375283.43 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 37528.34 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912525.802161 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337755.09 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 33775.51 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946301.311141 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303979.58 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 30397.96 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976699.269222 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273581.62 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 27358.16 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004057.431496 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246223.46 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 24622.35 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028679.777541 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221601.11 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 22160.11 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050839.888983 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199441.00 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 19944.10 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070783.989280 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179496.90 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 17949.69 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088733.679548 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161547.21 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 16154.72 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104888.400788 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145392.49 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 14539.25 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119427.649905 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130853.24 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 13085.32 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132512.974110 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117767.92 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 11776.79 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144289.765895 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105991.13 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 10599.11 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154888.878501 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95392.01 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 9539.20 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164428.079846 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85852.81 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 8585.28 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173013.361057 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77267.53 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 7726.75 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180740.114147 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69540.78 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 6954.08 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187694.191928 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62586.70 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 6258.67 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193952.861930 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56328.03 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 5632.80 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199585.664933 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50695.23 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 5069.52 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204655.187635 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45625.70 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 4562.57 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209217.758067 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41063.13 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 4106.31 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213324.071456 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36956.82 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3695.68 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217019.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33261.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220619.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29661.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224219.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26061.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227819.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22461.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231419.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18861.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235019.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15261.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238619.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11661.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242219.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8061.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245819.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4461.14 sec) 07:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249419.753506 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861.14 sec) 07:57:00 [GPUGRID] [rr_sim] 2250280.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989626026.06G/439.78G) 07:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851941.80 07:57:00 [---] pick_jobs_to_run() start 07:57:00 [---] pick_jobs_to_run() end 07:57:00 [---] [rr_sim] end 07:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:57:00 [---] [cpu_sched_debug] preliminary job list: 07:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:57:00 [---] [cpu_sched_debug] final job list: 07:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:57:00 [---] [cpu_sched_debug] enforce_run_list: end 07:57:00 [---] choose_project(): 28620.000000 07:57:00 [iThena.Measurements] doing RPC to get work 07:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406979.09 cpu 2152112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:57:00 [---] Got 0 tasks 07:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:58:00 [---] [rr_sim] doing sim: CPU sched 07:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989598435.71G/439.78G) 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250218.16 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 225021.82 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225021.815507 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025196.34 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 202519.63 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427541.449464 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822676.71 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 182267.67 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609809.120025 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640409.04 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 164040.90 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773850.023530 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476368.13 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 147636.81 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921486.836684 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328731.32 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 132873.13 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054359.968523 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195858.19 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 119585.82 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173945.787178 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076272.37 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 107627.24 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281573.023967 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968645.13 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 96864.51 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378437.537078 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871780.62 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 87178.06 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465615.598877 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784602.56 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 78460.26 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544075.854497 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706142.30 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 70614.23 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614690.084555 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635528.07 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 63552.81 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678242.891606 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571975.26 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 57197.53 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735440.417953 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514777.74 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 51477.77 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786918.191665 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463299.96 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 46330.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833248.188006 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416969.97 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 41697.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874945.184713 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375272.97 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 37527.30 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912472.481749 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337745.67 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 33774.57 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946247.049081 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303971.11 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 30397.11 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976644.159680 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273574.00 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 27357.40 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004001.559220 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246216.60 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 24621.66 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028623.218805 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221594.94 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 22159.49 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050782.712432 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199435.44 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 19943.54 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070726.256696 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179491.90 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 17949.19 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088675.446534 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161542.71 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 16154.27 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104829.717388 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145388.44 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 14538.84 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119368.561156 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130849.59 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 13084.96 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132453.520548 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117764.63 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 11776.46 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144229.984000 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105988.17 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 10598.82 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154828.801108 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95389.35 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 9538.94 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164367.736504 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85850.42 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 8585.04 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172952.778361 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77265.38 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 7726.54 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180679.316032 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69538.84 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 6953.88 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187633.199936 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62584.96 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 6258.50 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193891.695450 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56326.46 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 5632.65 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199524.341412 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50693.81 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 5069.38 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204593.722778 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45624.43 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 4562.44 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209156.166008 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41061.99 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 4106.20 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213262.364914 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36955.79 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3695.58 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216957.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33260.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220557.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29660.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224157.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26060.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227757.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22460.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231357.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18860.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234957.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15260.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238557.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11660.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242157.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8060.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245757.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4460.21 sec) 07:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249357.943930 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860.21 sec) 07:58:00 [GPUGRID] [rr_sim] 2250218.16: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989598435.71G/439.78G) 07:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851939.07 07:58:00 [---] pick_jobs_to_run() start 07:58:00 [---] pick_jobs_to_run() end 07:58:00 [---] [rr_sim] end 07:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:58:00 [---] [cpu_sched_debug] preliminary job list: 07:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:58:00 [---] [cpu_sched_debug] final job list: 07:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:58:00 [---] [cpu_sched_debug] enforce_run_list: end 07:58:00 [---] choose_project(): 28680.000000 07:58:00 [iThena.Measurements] doing RPC to get work 07:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406919.09 cpu 2152052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2152052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2152052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:58:00 [---] Got 0 tasks 07:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 07:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 07:59:00 [---] [rr_sim] doing sim: CPU sched 07:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989570845.35G/439.78G) 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250155.42 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 225015.54 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225015.541819 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025139.88 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 202513.99 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427529.529456 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822625.89 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 182262.59 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609792.118330 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640363.30 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 164036.33 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773828.448316 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476326.97 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 147632.70 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921461.145303 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328694.27 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 132869.43 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054330.572592 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195824.85 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 119582.48 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173913.057152 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076242.36 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 107624.24 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281537.293256 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968618.12 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 96861.81 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378399.105749 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871756.31 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 87175.63 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465574.736994 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784580.68 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 78458.07 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1544032.805113 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706122.61 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 70612.26 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614645.066421 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635510.35 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 63551.04 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678196.101598 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571959.32 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 57195.93 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735392.033257 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514763.38 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 51476.34 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786868.371751 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463287.05 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 46328.70 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833197.076395 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416958.34 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 41695.83 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874892.910574 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375262.51 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 37526.25 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912419.161336 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337736.26 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 33773.63 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946192.787021 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303962.63 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 30396.26 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976589.050138 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273566.37 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 27356.64 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003945.686944 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246209.73 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 24620.97 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028566.660068 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221588.76 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 22158.88 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050725.535881 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199429.88 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 19942.99 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070668.524112 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179486.89 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 17948.69 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088617.213520 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161538.20 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 16153.82 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104771.033987 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145384.38 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 14538.44 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119309.472407 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130845.95 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 13084.59 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132394.066986 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117761.35 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 11776.14 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144170.202106 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105985.22 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 10598.52 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154768.723715 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95386.69 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 9538.67 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164307.393162 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85848.03 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 8584.80 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172892.195665 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77263.22 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 7726.32 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180618.517918 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69536.90 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 6953.69 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187572.207945 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62583.21 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 6258.32 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193830.528970 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56324.89 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 5632.49 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199463.017892 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50692.40 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 5069.24 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204532.257922 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45623.16 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 4562.32 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209094.573949 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41060.84 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 4106.08 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213200.658373 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36954.76 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3695.48 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216896.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33259.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220496.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29659.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224096.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26059.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227696.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22459.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231296.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18859.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234896.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15259.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238496.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11659.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242096.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8059.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245696.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4459.28 sec) 07:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249296.134355 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859.28 sec) 07:59:00 [GPUGRID] [rr_sim] 2250155.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989570845.35G/439.78G) 07:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851936.33 07:59:00 [---] pick_jobs_to_run() start 07:59:00 [---] pick_jobs_to_run() end 07:59:00 [---] [rr_sim] end 07:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 07:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 07:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 07:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 07:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 07:59:00 [---] [cpu_sched_debug] preliminary job list: 07:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:59:00 [---] [cpu_sched_debug] final job list: 07:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 07:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 07:59:00 [---] [cpu_sched_debug] enforce_run_list: end 07:59:00 [---] choose_project(): 28740.000000 07:59:00 [iThena.Measurements] doing RPC to get work 07:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406859.09 cpu 2151992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 07:59:00 [---] Got 0 tasks 08:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:00:00 [---] [rr_sim] doing sim: CPU sched 08:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989543255.00G/439.78G) 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250092.68 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 225009.27 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225009.268131 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025083.41 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 202508.34 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427517.609449 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822575.07 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 182257.51 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609775.116635 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640317.56 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 164031.76 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773806.873102 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476285.81 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 147628.58 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921435.453923 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328657.23 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 132865.72 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054301.176661 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195791.50 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 119579.15 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173880.327126 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076212.35 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 107621.24 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281501.562544 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968591.12 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 96859.11 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378360.674421 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871732.01 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 87173.20 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465533.875110 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784558.81 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 78455.88 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543989.755729 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706102.93 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 70610.29 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614600.048287 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635492.63 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 63549.26 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678149.311590 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571943.37 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 57194.34 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735343.648561 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514749.03 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 51474.90 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786818.551836 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463274.13 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 46327.41 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833145.964783 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416946.72 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 41694.67 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874840.636436 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375252.04 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 37525.20 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912365.840923 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337726.84 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 33772.68 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946138.524962 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303954.16 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 30395.42 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976533.940596 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273558.74 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 27355.87 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003889.814668 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246202.87 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 24620.29 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028510.101332 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221582.58 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 22158.26 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050668.359330 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199424.32 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 19942.43 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070610.791527 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179481.89 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 17948.19 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088558.980506 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161533.70 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 16153.37 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104712.350586 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145380.33 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 14538.03 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119250.383658 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130842.30 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 13084.23 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132334.613423 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117758.07 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 11775.81 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144110.420212 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105982.26 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 10598.23 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154708.646321 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95384.03 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 9538.40 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164247.049820 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85845.63 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 8584.56 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172831.612969 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77261.07 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 7726.11 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180557.719803 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69534.96 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 6953.50 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187511.215954 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62581.47 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 6258.15 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193769.362489 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56323.32 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 5632.33 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199401.694371 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50690.99 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 5069.10 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204470.793065 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45621.89 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 4562.19 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209032.981889 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41059.70 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 4105.97 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213138.951831 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36953.73 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3695.37 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216834.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33258.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220434.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29658.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224034.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26058.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227634.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22458.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231234.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18858.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234834.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15258.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238434.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11658.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242034.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8058.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245634.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4458.36 sec) 08:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249234.324779 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858.36 sec) 08:00:00 [GPUGRID] [rr_sim] 2250092.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989543255.00G/439.78G) 08:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851933.59 08:00:00 [---] pick_jobs_to_run() start 08:00:00 [---] pick_jobs_to_run() end 08:00:00 [---] [rr_sim] end 08:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:00:00 [---] [cpu_sched_debug] preliminary job list: 08:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:00:00 [---] [cpu_sched_debug] final job list: 08:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:00:00 [---] [cpu_sched_debug] enforce_run_list: end 08:00:00 [---] choose_project(): 28800.000000 08:00:00 [iThena.Measurements] doing RPC to get work 08:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406799.09 cpu 2151932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:00:00 [---] Got 0 tasks 08:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:01:00 [---] [rr_sim] doing sim: CPU sched 08:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989515664.64G/439.78G) 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2250029.94 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 225002.99 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 225002.994443 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2025026.95 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 202502.69 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427505.689441 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822524.25 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 182252.43 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609758.114940 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640271.83 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 164027.18 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773785.297888 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476244.65 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 147624.46 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921409.762542 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328620.18 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 132862.02 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054271.780731 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195758.16 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 119575.82 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173847.597100 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076182.35 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 107618.23 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281465.831833 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968564.11 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 96856.41 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378322.243092 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871707.70 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 87170.77 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465493.013226 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784536.93 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 78453.69 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543946.706346 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706083.24 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 70608.32 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614555.030154 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635474.91 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 63547.49 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678102.521581 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571927.42 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 57192.74 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735295.263866 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514734.68 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 51473.47 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786768.731922 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463261.21 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 46326.12 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833094.853172 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416935.09 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 41693.51 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874788.362298 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375241.58 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 37524.16 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912312.520511 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337717.42 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 33771.74 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946084.262902 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303945.68 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 30394.57 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976478.831055 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273551.11 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 27355.11 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003833.942392 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246196.00 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 24619.60 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028453.542595 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221576.40 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 22157.64 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050611.182778 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199418.76 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 19941.88 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070553.058943 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179476.89 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 17947.69 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088500.747492 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161529.20 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 16152.92 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104653.667185 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145376.28 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 14537.63 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119191.294909 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130838.65 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 13083.86 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132275.159861 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117754.78 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 11775.48 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144050.638317 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105979.31 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 10597.93 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154648.568928 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95381.38 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 9538.14 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164186.706478 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85843.24 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 8584.32 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172771.030273 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77258.91 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 7725.89 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180496.921688 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69533.02 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 6953.30 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187450.223962 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62579.72 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 6257.97 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193708.196009 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56321.75 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 5632.17 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199340.370850 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50689.57 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 5068.96 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204409.328208 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45620.62 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 4562.06 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208971.389830 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41058.55 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 4105.86 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213077.245290 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36952.70 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3695.27 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216772.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33257.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220372.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29657.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223972.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26057.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227572.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22457.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231172.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18857.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234772.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15257.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238372.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11657.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241972.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8057.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245572.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4457.43 sec) 08:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249172.515203 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857.43 sec) 08:01:00 [GPUGRID] [rr_sim] 2250029.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989515664.64G/439.78G) 08:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851930.86 08:01:00 [---] pick_jobs_to_run() start 08:01:00 [---] pick_jobs_to_run() end 08:01:00 [---] [rr_sim] end 08:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:01:00 [---] [cpu_sched_debug] preliminary job list: 08:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:01:00 [---] [cpu_sched_debug] final job list: 08:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:01:00 [---] [cpu_sched_debug] enforce_run_list: end 08:01:00 [---] choose_project(): 28860.000000 08:01:00 [iThena.Measurements] doing RPC to get work 08:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406739.09 cpu 2151872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:01:00 [---] Got 0 tasks 08:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:02:00 [---] [rr_sim] doing sim: CPU sched 08:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989488074.28G/439.78G) 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249967.21 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 224996.72 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224996.720754 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024970.49 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 202497.05 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427493.769433 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822473.44 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 182247.34 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609741.113245 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640226.09 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 164022.61 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773763.722675 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476203.48 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 147620.35 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921384.071162 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328583.14 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 132858.31 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054242.384800 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195724.82 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 119572.48 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173814.867074 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076152.34 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 107615.23 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281430.101121 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968537.11 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 96853.71 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378283.811764 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871683.40 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 87168.34 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465452.151342 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784515.06 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 78451.51 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543903.656962 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706063.55 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 70606.36 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614510.012020 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635457.20 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 63545.72 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678055.731573 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571911.48 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 57191.15 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735246.879170 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514720.33 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 51472.03 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786718.912007 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463248.30 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 46324.83 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833043.741561 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416923.47 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 41692.35 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874736.088159 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375231.12 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 37523.11 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912259.200098 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337708.01 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 33770.80 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946030.000842 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303937.21 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 30393.72 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976423.721513 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273543.49 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 27354.35 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003778.070116 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246189.14 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 24618.91 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028396.983859 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221570.22 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 22157.02 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050554.006227 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199413.20 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 19941.32 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070495.326359 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179471.88 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 17947.19 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088442.514478 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161524.69 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 16152.47 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104594.983784 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145372.22 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 14537.22 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119132.206160 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130835.00 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 13083.50 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132215.706299 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117751.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 11775.15 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143990.856423 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105976.35 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 10597.64 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154588.491535 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95378.72 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 9537.87 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164126.363136 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85840.84 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 8584.08 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172710.447577 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77256.76 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 7725.68 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180436.123574 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69531.08 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 6953.11 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187389.231971 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62577.98 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 6257.80 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193647.029528 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56320.18 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 5632.02 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199279.047330 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50688.16 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 5068.82 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204347.863351 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45619.34 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 4561.93 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208909.797771 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41057.41 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 4105.74 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213015.538748 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36951.67 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3695.17 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216710.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33256.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220310.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29656.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223910.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26056.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227510.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22456.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231110.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18856.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234710.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15256.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238310.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11656.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241910.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8056.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245510.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4456.50 sec) 08:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249110.705628 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856.50 sec) 08:02:00 [GPUGRID] [rr_sim] 2249967.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989488074.28G/439.78G) 08:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851928.12 08:02:00 [---] pick_jobs_to_run() start 08:02:00 [---] pick_jobs_to_run() end 08:02:00 [---] [rr_sim] end 08:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:02:00 [---] [cpu_sched_debug] preliminary job list: 08:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:02:00 [---] [cpu_sched_debug] final job list: 08:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:02:00 [---] [cpu_sched_debug] enforce_run_list: end 08:02:00 [---] choose_project(): 28920.000000 08:02:00 [iThena.Measurements] doing RPC to get work 08:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406679.09 cpu 2151812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:02:00 [---] Got 0 tasks 08:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:03:00 [---] [rr_sim] doing sim: CPU sched 08:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989460483.93G/439.78G) 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249904.47 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 224990.45 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224990.447066 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024914.02 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 202491.40 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427481.849426 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822422.62 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 182242.26 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609724.111549 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640180.36 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 164018.04 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773742.147461 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476162.32 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 147616.23 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921358.379781 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328546.09 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 132854.61 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054212.988869 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195691.48 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 119569.15 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173782.137048 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076122.33 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 107612.23 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281394.370410 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968510.10 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 96851.01 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378245.380435 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871659.09 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 87165.91 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465411.289458 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784493.18 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 78449.32 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543860.607578 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706043.86 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 70604.39 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614464.993887 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635439.48 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 63543.95 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1678008.941564 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571895.53 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 57189.55 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735198.494474 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514705.98 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 51470.60 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786669.092093 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463235.38 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 46323.54 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832992.629950 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416911.84 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 41691.18 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874683.814021 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375220.66 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 37522.07 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912205.879685 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337698.59 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 33769.86 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945975.738783 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303928.73 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 30392.87 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976368.611971 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273535.86 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 27353.59 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003722.197840 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246182.27 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 24618.23 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028340.425122 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221564.05 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 22156.40 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050496.829676 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199407.64 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 19940.76 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070437.593775 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179466.88 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 17946.69 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088384.281464 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161520.19 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 16152.02 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104536.300383 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145368.17 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 14536.82 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119073.117411 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130831.35 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 13083.14 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132156.252736 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117748.22 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 11774.82 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143931.074529 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105973.40 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 10597.34 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154528.414142 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95376.06 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 9537.61 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164066.019794 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85838.45 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 8583.85 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172649.864881 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77254.61 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 7725.46 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180375.325459 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69529.15 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 6952.91 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187328.239979 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62576.23 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 6257.62 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193585.863048 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56318.61 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 5631.86 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199217.723809 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50686.75 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 5068.67 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204286.398495 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45618.07 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 4561.81 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208848.205711 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41056.26 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 4105.63 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212953.832206 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36950.64 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3695.06 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216648.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33255.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220248.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29655.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223848.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26055.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227448.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22455.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231048.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18855.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234648.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15255.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238248.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11655.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241848.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8055.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245448.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4455.57 sec) 08:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2249048.896052 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855.57 sec) 08:03:00 [GPUGRID] [rr_sim] 2249904.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989460483.93G/439.78G) 08:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851925.38 08:03:00 [---] pick_jobs_to_run() start 08:03:00 [---] pick_jobs_to_run() end 08:03:00 [---] [rr_sim] end 08:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:03:00 [---] [cpu_sched_debug] preliminary job list: 08:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:03:00 [---] [cpu_sched_debug] final job list: 08:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:03:00 [---] [cpu_sched_debug] enforce_run_list: end 08:03:00 [---] choose_project(): 28980.000000 08:03:00 [iThena.Measurements] doing RPC to get work 08:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406619.09 cpu 2151752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:03:00 [---] Got 0 tasks 08:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:04:00 [---] [rr_sim] doing sim: CPU sched 08:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989432893.57G/439.78G) 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249841.73 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 224984.17 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224984.173378 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024857.56 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 202485.76 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427469.929418 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822371.80 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 182237.18 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609707.109854 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640134.62 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 164013.46 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773720.572247 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476121.16 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 147612.12 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921332.688400 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328509.05 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 132850.90 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054183.592938 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195658.14 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 119565.81 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173749.407022 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076092.33 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 107609.23 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281358.639698 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968483.09 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 96848.31 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378206.949106 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871634.78 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 87163.48 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465370.427574 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784471.31 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 78447.13 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543817.558194 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706024.18 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 70602.42 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614419.975753 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635421.76 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 63542.18 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677962.151556 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571879.58 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 57187.96 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735150.109778 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514691.62 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 51469.16 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786619.272178 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463222.46 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 46322.25 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832941.518338 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416900.22 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 41690.02 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874631.539883 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375210.19 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 37521.02 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912152.559272 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337689.17 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 33768.92 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945921.476723 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303920.26 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 30392.03 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976313.502429 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273528.23 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 27352.82 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003666.325564 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246175.41 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 24617.54 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028283.866386 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221557.87 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 22155.79 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050439.653125 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199402.08 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 19940.21 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070379.861191 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179461.87 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 17946.19 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088326.048449 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161515.69 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 16151.57 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104477.616983 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145364.12 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 14536.41 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119014.028662 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130827.71 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 13082.77 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132096.799174 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117744.93 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 11774.49 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143871.292635 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105970.44 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 10597.04 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154468.336749 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95373.40 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 9537.34 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164005.676452 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85836.06 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 8583.61 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172589.282185 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77252.45 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 7725.25 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180314.527345 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69527.21 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 6952.72 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187267.247988 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62574.49 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 6257.45 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193524.696567 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56317.04 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 5631.70 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199156.400289 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50685.33 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 5068.53 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204224.933638 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45616.80 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 4561.68 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208786.613652 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41055.12 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 4105.51 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212892.125665 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36949.61 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3694.96 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216587.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33254.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220187.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29654.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223787.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26054.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227387.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22454.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230987.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18854.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234587.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15254.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238187.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11654.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241787.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8054.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245387.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4454.65 sec) 08:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248987.086476 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854.65 sec) 08:04:00 [GPUGRID] [rr_sim] 2249841.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989432893.57G/439.78G) 08:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851922.65 08:04:00 [---] pick_jobs_to_run() start 08:04:00 [---] pick_jobs_to_run() end 08:04:00 [---] [rr_sim] end 08:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:04:00 [---] [cpu_sched_debug] preliminary job list: 08:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:04:00 [---] [cpu_sched_debug] final job list: 08:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:04:00 [---] [cpu_sched_debug] enforce_run_list: end 08:04:00 [---] choose_project(): 29040.000000 08:04:00 [iThena.Measurements] doing RPC to get work 08:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406559.09 cpu 2151692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:04:00 [---] Got 0 tasks 08:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:05:00 [---] [rr_sim] doing sim: CPU sched 08:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989405303.21G/439.78G) 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249779.00 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 224977.90 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224977.899690 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024801.10 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 202480.11 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427458.009411 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822320.99 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 182232.10 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609690.108159 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640088.89 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 164008.89 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773698.997033 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476080.00 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 147608.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921306.997020 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328472.00 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 132847.20 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054154.197007 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195624.80 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 119562.48 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173716.676997 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076062.32 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 107606.23 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281322.908987 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968456.09 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 96845.61 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378168.517778 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871610.48 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 87161.05 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465329.565690 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784449.43 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 78444.94 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543774.508811 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706004.49 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 70600.45 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614374.957619 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635404.04 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 63540.40 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677915.361547 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571863.64 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 57186.36 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735101.725082 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514677.27 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 51467.73 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786569.452264 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463209.54 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 46320.95 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832890.406727 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416888.59 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 41688.86 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874579.265744 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375199.73 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 37519.97 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912099.238860 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337679.76 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 33767.98 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945867.214663 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303911.78 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 30391.18 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976258.392887 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273520.60 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 27352.06 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003610.453288 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246168.54 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 24616.85 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028227.307649 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221551.69 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 22155.17 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050382.476574 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199396.52 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 19939.65 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070322.128606 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179456.87 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 17945.69 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088267.815435 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161511.18 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 16151.12 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104418.933582 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145360.06 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 14536.01 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118954.939913 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130824.06 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 13082.41 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132037.345612 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117741.65 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 11774.17 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143811.510740 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105967.49 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 10596.75 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154408.259356 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95370.74 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 9537.07 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163945.333110 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85833.66 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 8583.37 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172528.699489 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77250.30 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 7725.03 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180253.729230 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69525.27 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 6952.53 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187206.255997 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62572.74 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 6257.27 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193463.530087 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56315.47 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 5631.55 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199095.076768 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50683.92 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 5068.39 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204163.468781 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45615.53 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 4561.55 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208725.021593 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41053.98 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 4105.40 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212830.419123 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36948.58 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3694.86 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216525.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33253.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220125.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29653.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223725.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26053.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227325.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22453.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230925.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18853.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234525.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15253.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238125.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11653.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241725.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8053.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245325.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4453.72 sec) 08:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248925.276901 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853.72 sec) 08:05:00 [GPUGRID] [rr_sim] 2249779.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989405303.21G/439.78G) 08:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851919.91 08:05:00 [---] pick_jobs_to_run() start 08:05:00 [---] pick_jobs_to_run() end 08:05:00 [---] [rr_sim] end 08:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:05:00 [---] [cpu_sched_debug] preliminary job list: 08:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:05:00 [---] [cpu_sched_debug] final job list: 08:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:05:00 [---] [cpu_sched_debug] enforce_run_list: end 08:05:00 [---] choose_project(): 29100.000000 08:05:00 [iThena.Measurements] doing RPC to get work 08:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406499.09 cpu 2151632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:05:00 [---] Got 0 tasks 08:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:06:00 [---] [rr_sim] doing sim: CPU sched 08:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989377712.86G/439.78G) 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249716.26 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 224971.63 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224971.626002 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024744.63 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 202474.46 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427446.089403 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822270.17 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 182227.02 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609673.106464 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1640043.15 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 164004.32 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773677.421819 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1476038.84 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 147603.88 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921281.305639 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328434.95 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 132843.50 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054124.801077 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195591.46 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 119559.15 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173683.946971 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076032.31 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 107603.23 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281287.178275 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968429.08 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 96842.91 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378130.086449 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871586.17 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 87158.62 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465288.703806 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784427.56 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 78442.76 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543731.459427 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705984.80 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 70598.48 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614329.939486 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635386.32 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 63538.63 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677868.571539 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571847.69 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 57184.77 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735053.340386 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514662.92 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 51466.29 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786519.632349 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463196.63 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 46319.66 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832839.295116 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416876.96 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 41687.70 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874526.991606 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375189.27 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 37518.93 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912045.918447 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337670.34 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 33767.03 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945812.952604 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303903.31 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 30390.33 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976203.283345 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273512.98 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 27351.30 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003554.581012 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246161.68 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 24616.17 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028170.748912 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221545.51 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 22154.55 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050325.300023 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199390.96 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 19939.10 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070264.396022 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179451.86 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 17945.19 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088209.582421 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161506.68 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 16150.67 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104360.250181 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145356.01 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 14535.60 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118895.851164 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130820.41 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 13082.04 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131977.892050 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117738.37 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 11773.84 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143751.728846 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105964.53 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 10596.45 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154348.181963 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95368.08 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 9536.81 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163884.989768 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85831.27 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 8583.13 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172468.116793 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77248.14 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 7724.81 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180192.931115 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69523.33 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 6952.33 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187145.264005 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62571.00 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 6257.10 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193402.363606 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56313.90 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 5631.39 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199033.753247 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50682.51 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 5068.25 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204102.003924 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45614.26 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 4561.43 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208663.429533 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41052.83 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 4105.28 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212768.712582 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36947.55 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3694.75 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216463.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33252.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220063.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29652.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223663.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26052.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227263.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22452.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230863.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18852.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234463.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15252.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238063.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11652.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241663.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8052.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245263.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4452.79 sec) 08:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248863.467325 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852.79 sec) 08:06:00 [GPUGRID] [rr_sim] 2249716.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989377712.86G/439.78G) 08:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851917.17 08:06:00 [---] pick_jobs_to_run() start 08:06:00 [---] pick_jobs_to_run() end 08:06:00 [---] [rr_sim] end 08:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:06:00 [---] [cpu_sched_debug] preliminary job list: 08:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:06:00 [---] [cpu_sched_debug] final job list: 08:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:06:00 [---] [cpu_sched_debug] enforce_run_list: end 08:06:00 [---] choose_project(): 29160.000000 08:06:00 [iThena.Measurements] doing RPC to get work 08:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406439.09 cpu 2151572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:06:00 [---] Got 0 tasks 08:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:07:00 [---] [rr_sim] doing sim: CPU sched 08:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989350122.50G/439.78G) 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249653.52 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 224965.35 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224965.352313 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024688.17 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 202468.82 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427434.169395 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822219.35 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 182221.94 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609656.104769 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639997.42 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 163999.74 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773655.846606 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475997.68 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 147599.77 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921255.614258 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328397.91 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 132839.79 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054095.405146 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195558.12 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 119555.81 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173651.216945 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1076002.31 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 107600.23 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281251.447564 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968402.08 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 96840.21 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378091.655121 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871561.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 87156.19 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465247.841922 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784405.68 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 78440.57 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543688.410043 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705965.11 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 70596.51 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614284.921352 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635368.60 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 63536.86 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677821.781530 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571831.74 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 57183.17 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1735004.955691 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514648.57 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 51464.86 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786469.812435 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463183.71 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 46318.37 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832788.183505 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416865.34 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 41686.53 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874474.717468 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375178.81 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 37517.88 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911992.598034 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337660.93 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 33766.09 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945758.690544 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303894.83 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 30389.48 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976148.173803 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273505.35 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 27350.53 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003498.708736 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246154.81 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 24615.48 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028114.190176 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221539.33 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 22153.93 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050268.123472 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199385.40 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 19938.54 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070206.663438 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179446.86 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 17944.69 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088151.349407 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161502.17 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 16150.22 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104301.566780 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145351.96 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 14535.20 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118836.762415 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130816.76 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 13081.68 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131918.438487 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117735.08 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 11773.51 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143691.946952 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105961.58 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 10596.16 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154288.104570 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95365.42 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 9536.54 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163824.646426 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85828.88 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 8582.89 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172407.534097 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77245.99 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 7724.60 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180132.133001 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69521.39 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 6952.14 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187084.272014 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62569.25 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 6256.93 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193341.197126 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56312.33 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 5631.23 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198972.429727 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50681.09 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 5068.11 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204040.539067 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45612.98 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 4561.30 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208601.837474 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41051.69 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 4105.17 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212707.006040 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36946.52 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3694.65 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216401.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33251.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220001.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29651.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223601.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26051.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227201.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22451.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230801.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18851.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234401.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15251.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238001.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11651.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241601.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8051.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245201.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4451.87 sec) 08:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248801.657749 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851.87 sec) 08:07:00 [GPUGRID] [rr_sim] 2249653.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989350122.50G/439.78G) 08:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851914.44 08:07:00 [---] pick_jobs_to_run() start 08:07:00 [---] pick_jobs_to_run() end 08:07:00 [---] [rr_sim] end 08:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:07:00 [---] [cpu_sched_debug] preliminary job list: 08:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:07:00 [---] [cpu_sched_debug] final job list: 08:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:07:00 [---] [cpu_sched_debug] enforce_run_list: end 08:07:00 [---] choose_project(): 29220.000000 08:07:00 [iThena.Measurements] doing RPC to get work 08:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406379.09 cpu 2151512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:07:00 [---] Got 0 tasks 08:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:08:00 [---] [rr_sim] doing sim: CPU sched 08:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989322532.15G/439.78G) 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249590.79 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 224959.08 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224959.078625 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024631.71 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 202463.17 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427422.249388 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822168.54 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 182216.85 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609639.103074 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639951.68 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 163995.17 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773634.271392 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475956.51 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 147595.65 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921229.922878 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328360.86 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 132836.09 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054066.009215 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195524.78 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 119552.48 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173618.486919 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075972.30 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 107597.23 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281215.716852 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968375.07 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 96837.51 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378053.223792 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871537.56 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 87153.76 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465206.980038 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784383.81 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 78438.38 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543645.360659 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705945.43 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 70594.54 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614239.903218 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635350.88 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 63535.09 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677774.991522 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571815.79 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 57181.58 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734956.570995 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514634.22 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 51463.42 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786419.992520 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463170.79 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 46317.08 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832737.071894 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416853.71 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 41685.37 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874422.443329 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375168.34 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 37516.83 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911939.277622 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337651.51 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 33765.15 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945704.428485 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303886.36 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 30388.64 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976093.064261 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273497.72 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 27349.77 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003442.836460 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246147.95 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 24614.79 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028057.631439 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221533.15 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 22153.32 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050210.946921 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199379.84 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 19937.98 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070148.930854 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179441.86 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 17944.19 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088093.116393 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161497.67 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 16149.77 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104242.883379 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145347.90 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 14534.79 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118777.673666 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130813.11 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 13081.31 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131858.984925 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117731.80 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 11773.18 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143632.165058 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105958.62 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 10595.86 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154228.027177 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95362.76 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 9536.28 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163764.303084 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85826.48 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 8582.65 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172346.951401 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77243.83 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 7724.38 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180071.334886 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69519.45 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 6951.95 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187023.280023 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62567.51 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 6256.75 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193280.030646 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56310.76 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 5631.08 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198911.106206 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50679.68 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 5067.97 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203979.074211 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45611.71 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 4561.17 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208540.245415 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41050.54 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 4105.05 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212645.299498 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36945.49 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3694.55 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216339.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33250.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219939.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29650.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223539.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26050.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227139.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22450.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230739.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18850.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234339.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15250.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237939.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11650.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241539.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8050.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245139.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4450.94 sec) 08:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248739.848174 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850.94 sec) 08:08:00 [GPUGRID] [rr_sim] 2249590.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989322532.15G/439.78G) 08:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851911.70 08:08:00 [---] pick_jobs_to_run() start 08:08:00 [---] pick_jobs_to_run() end 08:08:00 [---] [rr_sim] end 08:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:08:00 [---] [cpu_sched_debug] preliminary job list: 08:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:08:00 [---] [cpu_sched_debug] final job list: 08:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:08:00 [---] [cpu_sched_debug] enforce_run_list: end 08:08:00 [---] choose_project(): 29280.000000 08:08:00 [iThena.Measurements] doing RPC to get work 08:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406319.09 cpu 2151452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:08:00 [---] Got 0 tasks 08:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:09:00 [---] [rr_sim] doing sim: CPU sched 08:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989294941.79G/439.78G) 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249528.05 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 224952.80 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224952.804937 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024575.24 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 202457.52 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427410.329380 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822117.72 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 182211.77 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609622.101379 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639905.95 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 163990.59 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773612.696178 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475915.35 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 147591.54 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921204.231497 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328323.82 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 132832.38 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054036.613284 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195491.44 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 119549.14 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173585.756893 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075942.29 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 107594.23 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281179.986141 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968348.06 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 96834.81 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1378014.792463 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871513.26 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 87151.33 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465166.118154 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784361.93 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 78436.19 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543602.311276 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705925.74 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 70592.57 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614194.885085 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635333.16 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 63533.32 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677728.201513 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571799.85 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 57179.98 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734908.186299 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514619.86 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 51461.99 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786370.172606 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463157.88 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 46315.79 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832685.960282 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416842.09 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 41684.21 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874370.169191 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375157.88 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 37515.79 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911885.957209 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337642.09 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 33764.21 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945650.166425 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303877.88 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 30387.79 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976037.954719 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273490.09 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 27349.01 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003386.964184 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246141.09 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 24614.11 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028001.072703 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221526.98 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 22152.70 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050153.770369 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199374.28 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 19937.43 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070091.198269 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179436.85 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 17943.69 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088034.883379 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161493.17 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 16149.32 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104184.199978 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145343.85 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 14534.38 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118718.584917 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130809.46 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 13080.95 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131799.531363 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117728.52 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 11772.85 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143572.383163 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105955.67 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 10595.57 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154167.949784 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95360.10 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 9536.01 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163703.959742 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85824.09 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 8582.41 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172286.368705 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77241.68 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 7724.17 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180010.536772 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69517.51 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 6951.75 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186962.288031 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62565.76 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 6256.58 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193218.864165 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56309.19 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 5630.92 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198849.782686 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50678.27 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 5067.83 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203917.609354 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45610.44 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 4561.04 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208478.653355 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41049.40 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 4104.94 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212583.592957 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36944.46 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3694.45 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216278.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33250.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219878.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29650.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223478.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26050.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227078.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22450.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230678.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18850.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234278.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15250.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237878.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11650.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241478.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8050.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245078.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4450.01 sec) 08:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248678.038598 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850.01 sec) 08:09:00 [GPUGRID] [rr_sim] 2249528.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989294941.79G/439.78G) 08:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851908.96 08:09:00 [---] pick_jobs_to_run() start 08:09:00 [---] pick_jobs_to_run() end 08:09:00 [---] [rr_sim] end 08:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:09:00 [---] [cpu_sched_debug] preliminary job list: 08:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:09:00 [---] [cpu_sched_debug] final job list: 08:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:09:00 [---] [cpu_sched_debug] enforce_run_list: end 08:09:00 [---] choose_project(): 29340.000000 08:09:00 [iThena.Measurements] doing RPC to get work 08:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 579059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406259.09 cpu 2151392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:09:00 [---] Got 0 tasks 08:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:10:00 [---] [rr_sim] doing sim: CPU sched 08:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989267351.43G/439.78G) 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249465.31 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 224946.53 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224946.531249 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024518.78 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 202451.88 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427398.409373 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822066.90 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 182206.69 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609605.099684 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639860.21 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 163986.02 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773591.120964 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475874.19 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 147587.42 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921178.540117 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328286.77 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 132828.68 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1054007.217354 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195458.10 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 119545.81 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173553.026867 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075912.29 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 107591.23 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281144.255429 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968321.06 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 96832.11 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377976.361135 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871488.95 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 87148.90 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465125.256270 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784340.06 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 78434.01 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543559.261892 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705906.05 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 70590.61 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614149.866951 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635315.45 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 63531.54 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677681.411505 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571783.90 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 57178.39 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734859.801603 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514605.51 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 51460.55 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786320.352691 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463144.96 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 46314.50 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832634.848671 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416830.46 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 41683.05 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874317.895053 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375147.42 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 37514.74 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911832.636796 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337632.68 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 33763.27 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945595.904365 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303869.41 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 30386.94 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975982.845177 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273482.47 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 27348.25 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003331.091908 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246134.22 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 24613.42 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027944.513966 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221520.80 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 22152.08 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050096.593818 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199368.72 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 19936.87 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070033.465685 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179431.85 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 17943.18 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087976.650365 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161488.66 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 16148.87 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104125.516578 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145339.80 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 14533.98 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118659.496169 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130805.82 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 13080.58 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131740.077800 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117725.23 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 11772.52 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143512.601269 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105952.71 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 10595.27 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154107.872391 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95357.44 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 9535.74 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163643.616400 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85821.70 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 8582.17 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172225.786009 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77239.53 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 7723.95 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179949.738657 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69515.57 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 6951.56 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186901.296040 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62564.02 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 6256.40 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193157.697685 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56307.61 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 5630.76 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198788.459165 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50676.85 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 5067.69 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203856.144497 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45609.17 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 4560.92 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208417.061296 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41048.25 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 4104.83 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212521.886415 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36943.43 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3694.34 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216216.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33249.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219816.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29649.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223416.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26049.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227016.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22449.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230616.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18849.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234216.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15249.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237816.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11649.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241416.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8049.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245016.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4449.08 sec) 08:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248616.229022 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849.08 sec) 08:10:00 [GPUGRID] [rr_sim] 2249465.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989267351.43G/439.78G) 08:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851906.23 08:10:00 [---] pick_jobs_to_run() start 08:10:00 [---] pick_jobs_to_run() end 08:10:00 [---] [rr_sim] end 08:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:10:00 [---] [cpu_sched_debug] preliminary job list: 08:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:10:00 [---] [cpu_sched_debug] final job list: 08:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:10:00 [---] [cpu_sched_debug] enforce_run_list: end 08:10:00 [---] choose_project(): 29400.000000 08:10:00 [iThena.Measurements] doing RPC to get work 08:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406199.09 cpu 2151332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:10:00 [---] Got 0 tasks 08:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:11:00 [---] [rr_sim] doing sim: CPU sched 08:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989239761.08G/439.78G) 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249402.58 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 224940.26 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224940.257560 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024462.32 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 202446.23 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427386.489365 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1822016.09 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 182201.61 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609588.097989 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639814.48 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 163981.45 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773569.545751 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475833.03 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 147583.30 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921152.848736 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328249.73 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 132824.97 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053977.821423 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195424.75 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 119542.48 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173520.296841 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075882.28 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 107588.23 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281108.524717 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968294.05 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 96829.41 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377937.929806 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871464.65 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 87146.46 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465084.394386 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784318.18 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 78431.82 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543516.212508 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705886.36 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 70588.64 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614104.848818 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635297.73 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 63529.77 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677634.621496 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571767.95 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 57176.80 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734811.416907 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514591.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 51459.12 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786270.532777 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463132.04 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 46313.20 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832583.737060 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416818.84 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 41681.88 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874265.620914 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375136.95 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 37513.70 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911779.316383 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337623.26 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 33762.33 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945541.642306 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303860.93 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 30386.09 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975927.735635 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273474.84 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 27347.48 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003275.219632 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246127.36 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 24612.74 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027887.955230 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221514.62 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 22151.46 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050039.417267 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199363.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 19936.32 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069975.733101 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179426.84 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 17942.68 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087918.417351 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161484.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 16148.42 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104066.833177 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145335.74 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 14533.57 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118600.407420 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130802.17 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 13080.22 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131680.624238 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117721.95 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 11772.20 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143452.819375 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105949.76 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 10594.98 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154047.794998 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95354.78 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 9535.48 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163583.273058 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85819.30 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 8581.93 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172165.203313 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77237.37 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 7723.74 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179888.940542 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69513.64 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 6951.36 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186840.304049 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62562.27 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 6256.23 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193096.531204 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56306.04 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 5630.60 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198727.135644 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50675.44 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 5067.54 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203794.679640 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45607.90 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 4560.79 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208355.469237 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41047.11 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 4104.71 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212460.179874 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36942.40 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3694.24 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216154.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33248.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219754.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29648.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223354.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26048.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226954.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22448.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230554.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18848.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234154.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15248.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237754.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11648.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241354.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8048.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244954.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4448.16 sec) 08:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248554.419447 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 848.16 sec) 08:11:00 [GPUGRID] [rr_sim] 2249402.58: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989239761.08G/439.78G) 08:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851903.49 08:11:00 [---] pick_jobs_to_run() start 08:11:00 [---] pick_jobs_to_run() end 08:11:00 [---] [rr_sim] end 08:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:11:00 [---] [cpu_sched_debug] preliminary job list: 08:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:11:00 [---] [cpu_sched_debug] final job list: 08:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:11:00 [---] [cpu_sched_debug] enforce_run_list: end 08:11:00 [---] choose_project(): 29460.000000 08:11:00 [iThena.Measurements] doing RPC to get work 08:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406139.09 cpu 2151272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:11:00 [---] Got 0 tasks 08:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:12:00 [---] [rr_sim] doing sim: CPU sched 08:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989212170.72G/439.78G) 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249339.84 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 224933.98 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224933.983872 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024405.85 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 202440.59 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427374.569357 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821965.27 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 182196.53 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609571.096294 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639768.74 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 163976.87 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773547.970537 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475791.87 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 147579.19 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921127.157355 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328212.68 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 132821.27 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053948.425492 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195391.41 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 119539.14 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173487.566815 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075852.27 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 107585.23 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281072.794006 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968267.04 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 96826.70 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377899.498478 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871440.34 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 87144.03 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465043.532502 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784296.31 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 78429.63 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543473.163124 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705866.68 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 70586.67 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614059.830684 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635280.01 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 63528.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677587.831488 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571752.01 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 57175.20 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734763.032211 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514576.81 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 51457.68 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786220.712863 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463119.13 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 46311.91 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832532.625449 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416807.21 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 41680.72 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874213.346776 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375126.49 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 37512.65 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911725.995971 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337613.84 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 33761.38 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945487.380246 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303852.46 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 30385.25 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975872.626094 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273467.21 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 27346.72 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003219.347356 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246120.49 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 24612.05 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027831.396493 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221508.44 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 22150.84 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049982.240716 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199357.60 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 19935.76 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069918.000517 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179421.84 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 17942.18 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087860.184337 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161479.65 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 16147.97 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104008.149776 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145331.69 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 14533.17 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118541.318671 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130798.52 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 13079.85 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131621.170676 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117718.67 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 11771.87 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143393.037480 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105946.80 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 10594.68 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153987.717605 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95352.12 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 9535.21 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163522.929717 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85816.91 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 8581.69 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172104.620617 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77235.22 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 7723.52 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179828.142428 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69511.70 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 6951.17 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186779.312057 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62560.53 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 6256.05 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193035.364724 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56304.47 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 5630.45 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198665.812124 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50674.03 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 5067.40 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203733.214784 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45606.62 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 4560.66 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208293.877177 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41045.96 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 4104.60 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212398.473332 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36941.37 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3694.14 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216092.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33247.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219692.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29647.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223292.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26047.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226892.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22447.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230492.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18847.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234092.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15247.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237692.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11647.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241292.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8047.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244892.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4447.23 sec) 08:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248492.609871 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 847.23 sec) 08:12:00 [GPUGRID] [rr_sim] 2249339.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989212170.72G/439.78G) 08:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851900.75 08:12:00 [---] pick_jobs_to_run() start 08:12:00 [---] pick_jobs_to_run() end 08:12:00 [---] [rr_sim] end 08:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:12:00 [---] [cpu_sched_debug] preliminary job list: 08:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:12:00 [---] [cpu_sched_debug] final job list: 08:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:12:00 [---] [cpu_sched_debug] enforce_run_list: end 08:12:00 [---] choose_project(): 29520.000000 08:12:00 [iThena.Measurements] doing RPC to get work 08:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406079.09 cpu 2151212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:12:00 [---] Got 0 tasks 08:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:13:00 [---] [rr_sim] doing sim: CPU sched 08:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989184580.37G/439.78G) 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249277.10 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 224927.71 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224927.710184 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024349.39 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 202434.94 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427362.649350 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821914.45 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 182191.45 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609554.094599 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639723.01 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 163972.30 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773526.395323 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475750.71 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 147575.07 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921101.465975 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328175.64 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 132817.56 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053919.029561 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195358.07 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 119535.81 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173454.836789 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075822.27 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 107582.23 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281037.063294 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968240.04 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 96824.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377861.067149 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871416.03 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 87141.60 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1465002.670618 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784274.43 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 78427.44 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543430.113740 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705846.99 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 70584.70 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1614014.812550 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635262.29 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 63526.23 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677541.041479 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571736.06 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 57173.61 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734714.647516 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514562.45 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 51456.25 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786170.892948 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463106.21 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 46310.62 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832481.513837 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416795.59 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 41679.56 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874161.072638 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375116.03 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 37511.60 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911672.675558 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337604.43 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 33760.44 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945433.118186 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303843.98 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 30384.40 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975817.516552 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273459.59 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 27345.96 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003163.475081 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246113.63 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 24611.36 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027774.837757 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221502.26 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 22150.23 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049925.064165 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199352.04 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 19935.20 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069860.267933 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179416.83 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 17941.68 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087801.951323 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161475.15 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 16147.52 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103949.466375 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145327.64 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 14532.76 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118482.229922 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130794.87 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 13079.49 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131561.717113 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117715.38 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 11771.54 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143333.255586 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105943.85 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 10594.38 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153927.640212 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95349.46 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 9534.95 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163462.586375 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85814.52 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 8581.45 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172044.037921 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77233.06 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 7723.31 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179767.344313 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69509.76 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 6950.98 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186718.320066 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62558.78 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 6255.88 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192974.198243 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56302.90 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 5630.29 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198604.488603 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50672.61 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 5067.26 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203671.749927 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45605.35 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 4560.54 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208232.285118 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41044.82 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 4104.48 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212336.766790 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36940.34 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3694.03 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216030.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33246.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219630.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29646.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223230.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26046.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226830.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22446.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230430.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18846.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234030.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15246.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237630.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11646.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241230.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8046.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244830.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4446.30 sec) 08:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248430.800295 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 846.30 sec) 08:13:00 [GPUGRID] [rr_sim] 2249277.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989184580.37G/439.78G) 08:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851898.01 08:13:00 [---] pick_jobs_to_run() start 08:13:00 [---] pick_jobs_to_run() end 08:13:00 [---] [rr_sim] end 08:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:13:00 [---] [cpu_sched_debug] preliminary job list: 08:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:13:00 [---] [cpu_sched_debug] final job list: 08:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:13:00 [---] [cpu_sched_debug] enforce_run_list: end 08:13:00 [---] choose_project(): 29580.000000 08:13:00 [iThena.Measurements] doing RPC to get work 08:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 406019.09 cpu 2151152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:13:00 [---] Got 0 tasks 08:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:14:00 [---] [rr_sim] doing sim: CPU sched 08:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989156990.01G/439.78G) 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249214.36 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 224921.44 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224921.436496 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024292.93 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 202429.29 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427350.729342 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821863.64 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 182186.36 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609537.092904 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639677.27 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 163967.73 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773504.820109 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475709.54 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 147570.95 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921075.774594 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328138.59 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 132813.86 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053889.633631 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195324.73 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 119532.47 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173422.106763 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075792.26 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 107579.23 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1281001.332583 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968213.03 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 96821.30 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377822.635820 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871391.73 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 87139.17 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464961.808734 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784252.56 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 78425.26 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543387.064357 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705827.30 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 70582.73 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613969.794417 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635244.57 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 63524.46 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677494.251471 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571720.11 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 57172.01 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734666.262820 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514548.10 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 51454.81 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786121.073034 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463093.29 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 46309.33 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832430.402226 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416783.96 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 41678.40 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874108.798499 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375105.57 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 37510.56 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911619.355145 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337595.01 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 33759.50 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945378.856127 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303835.51 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 30383.55 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975762.407010 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273451.96 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 27345.20 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003107.602805 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246106.76 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 24610.68 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027718.279020 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221496.09 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 22149.61 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049867.887614 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199346.48 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 19934.65 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069802.535348 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179411.83 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 17941.18 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087743.718309 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161470.65 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 16147.06 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103890.782974 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145323.58 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 14532.36 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118423.141173 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130791.22 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 13079.12 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131502.263551 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117712.10 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 11771.21 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143273.473692 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105940.89 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 10594.09 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153867.562819 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95346.80 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 9534.68 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163402.243033 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85812.12 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 8581.21 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171983.455225 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77230.91 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 7723.09 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179706.546198 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69507.82 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 6950.78 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186657.328074 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62557.04 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 6255.70 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192913.031763 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56301.33 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 5630.13 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198543.165082 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50671.20 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 5067.12 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203610.285070 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45604.08 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 4560.41 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208170.693059 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41043.67 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 4104.37 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212275.060249 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36939.30 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3693.93 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215968.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33245.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219568.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29645.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223168.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26045.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226768.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22445.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230368.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18845.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233968.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15245.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237568.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11645.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241168.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8045.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244768.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4445.37 sec) 08:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248368.990720 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 845.37 sec) 08:14:00 [GPUGRID] [rr_sim] 2249214.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989156990.01G/439.78G) 08:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851895.28 08:14:00 [---] pick_jobs_to_run() start 08:14:00 [---] pick_jobs_to_run() end 08:14:00 [---] [rr_sim] end 08:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:14:00 [---] [cpu_sched_debug] preliminary job list: 08:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:14:00 [---] [cpu_sched_debug] final job list: 08:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:14:00 [---] [cpu_sched_debug] enforce_run_list: end 08:14:00 [---] choose_project(): 29640.000000 08:14:00 [iThena.Measurements] doing RPC to get work 08:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405959.09 cpu 2151092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:14:00 [---] Got 0 tasks 08:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:15:00 [---] [rr_sim] doing sim: CPU sched 08:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989129399.65G/439.78G) 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249151.63 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 224915.16 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224915.162808 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024236.47 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 202423.65 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427338.809334 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821812.82 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 182181.28 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609520.091209 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639631.54 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 163963.15 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773483.244895 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475668.38 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 147566.84 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921050.083214 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328101.54 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 132810.15 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053860.237700 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195291.39 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 119529.14 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173389.376737 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075762.25 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 107576.23 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280965.601871 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968186.03 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 96818.60 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377784.204492 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871367.42 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 87136.74 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464920.946850 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784230.68 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 78423.07 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543344.014973 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705807.61 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 70580.76 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613924.776283 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635226.85 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 63522.69 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677447.461463 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571704.17 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 57170.42 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734617.878124 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514533.75 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 51453.37 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786071.253119 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463080.37 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 46308.04 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832379.290615 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416772.34 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 41677.23 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874056.524361 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375095.10 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 37509.51 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911566.034733 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337585.59 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 33758.56 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945324.594067 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303827.03 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 30382.70 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975707.297468 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273444.33 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 27344.43 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003051.730529 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246099.90 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 24609.99 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027661.720283 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221489.91 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 22148.99 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049810.711063 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199340.92 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 19934.09 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069744.802764 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179406.83 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 17940.68 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087685.485295 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161466.14 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 16146.61 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103832.099573 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145319.53 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 14531.95 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118364.052424 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130787.58 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 13078.76 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131442.809989 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117708.82 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 11770.88 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143213.691798 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105937.94 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 10593.79 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153807.485426 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95344.14 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 9534.41 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163341.899691 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85809.73 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 8580.97 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171922.872529 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77228.76 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 7722.88 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179645.748084 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69505.88 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 6950.59 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186596.336083 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62555.29 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 6255.53 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192851.865282 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56299.76 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 5629.98 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198481.841562 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50669.79 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 5066.98 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203548.820213 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45602.81 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 4560.28 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208109.101000 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41042.53 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 4104.25 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212213.353707 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36938.27 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3693.83 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215907.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33244.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219507.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29644.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223107.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26044.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226707.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22444.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230307.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18844.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233907.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15244.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237507.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11644.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241107.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8044.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244707.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4444.45 sec) 08:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248307.181144 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 844.45 sec) 08:15:00 [GPUGRID] [rr_sim] 2249151.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989129399.65G/439.78G) 08:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851892.54 08:15:00 [---] pick_jobs_to_run() start 08:15:00 [---] pick_jobs_to_run() end 08:15:00 [---] [rr_sim] end 08:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:15:00 [---] [cpu_sched_debug] preliminary job list: 08:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:15:00 [---] [cpu_sched_debug] final job list: 08:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:15:00 [---] [cpu_sched_debug] enforce_run_list: end 08:15:00 [---] choose_project(): 29700.000000 08:15:00 [iThena.Measurements] doing RPC to get work 08:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405899.09 cpu 2151032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2151032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2151032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:15:00 [---] Got 0 tasks 08:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:16:00 [---] [rr_sim] doing sim: CPU sched 08:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989101809.30G/439.78G) 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249088.89 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 224908.89 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224908.889119 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024180.00 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 202418.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427326.889327 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821762.00 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 182176.20 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609503.089514 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639585.80 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 163958.58 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773461.669682 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475627.22 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 147562.72 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 921024.391833 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328064.50 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 132806.45 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053830.841769 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195258.05 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 119525.80 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173356.646712 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075732.24 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 107573.22 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280929.871160 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968159.02 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 96815.90 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377745.773163 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871343.12 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 87134.31 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464880.084966 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784208.81 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 78420.88 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543300.965589 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705787.93 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 70578.79 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613879.758150 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635209.13 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 63520.91 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677400.671454 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571688.22 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 57168.82 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734569.493428 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514519.40 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 51451.94 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786021.433205 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463067.46 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 46306.75 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832328.179004 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416760.71 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 41676.07 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874004.250223 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375084.64 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 37508.46 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911512.714320 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337576.18 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 33757.62 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945270.332007 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303818.56 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 30381.86 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975652.187926 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273436.70 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 27343.67 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002995.858253 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246093.03 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 24609.30 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027605.161547 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221483.73 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 22148.37 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049753.534512 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199335.36 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 19933.54 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069687.070180 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179401.82 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 17940.18 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087627.252281 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161461.64 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 16146.16 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103773.416173 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145315.48 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 14531.55 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118304.963675 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130783.93 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 13078.39 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131383.356427 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117705.53 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 11770.55 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143153.909903 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105934.98 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 10593.50 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153747.408032 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95341.48 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 9534.15 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163281.556349 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85807.33 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 8580.73 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171862.289833 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77226.60 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 7722.66 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179584.949969 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69503.94 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 6950.39 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186535.344092 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62553.55 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 6255.35 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192790.698802 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56298.19 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 5629.82 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198420.518041 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50668.37 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 5066.84 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203487.355356 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45601.54 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 4560.15 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208047.508940 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41041.38 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 4104.14 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212151.647166 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36937.24 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3693.72 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215845.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33243.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219445.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29643.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223045.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26043.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226645.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22443.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230245.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18843.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233845.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15243.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237445.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11643.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241045.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8043.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244645.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4443.52 sec) 08:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248245.371568 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 843.52 sec) 08:16:00 [GPUGRID] [rr_sim] 2249088.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989101809.30G/439.78G) 08:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851889.80 08:16:00 [---] pick_jobs_to_run() start 08:16:00 [---] pick_jobs_to_run() end 08:16:00 [---] [rr_sim] end 08:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:16:00 [---] [cpu_sched_debug] preliminary job list: 08:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:16:00 [---] [cpu_sched_debug] final job list: 08:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:16:00 [---] [cpu_sched_debug] enforce_run_list: end 08:16:00 [---] choose_project(): 29760.000000 08:16:00 [iThena.Measurements] doing RPC to get work 08:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405839.09 cpu 2150972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:16:00 [---] Got 0 tasks 08:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:17:00 [---] [rr_sim] doing sim: CPU sched 08:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989074218.94G/439.78G) 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2249026.15 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 224902.62 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224902.615431 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024123.54 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 202412.35 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427314.969319 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821711.18 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 182171.12 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609486.087819 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639540.07 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 163954.01 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773440.094468 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475586.06 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 147558.61 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920998.700452 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1328027.45 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 132802.75 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053801.445838 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195224.71 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 119522.47 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173323.916686 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075702.24 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 107570.22 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280894.140448 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968132.01 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 96813.20 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377707.341835 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871318.81 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 87131.88 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464839.223082 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784186.93 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 78418.69 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543257.916205 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705768.24 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 70576.82 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613834.740016 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635191.41 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 63519.14 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677353.881446 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571672.27 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 57167.23 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734521.108732 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514505.05 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 51450.50 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785971.613290 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463054.54 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 46305.45 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832277.067392 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416749.09 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 41674.91 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873951.976084 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375074.18 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 37507.42 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911459.393907 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337566.76 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 33756.68 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945216.069948 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303810.08 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 30381.01 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975597.078384 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273429.08 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 27342.91 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002939.985977 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246086.17 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 24608.62 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027548.602810 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221477.55 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 22147.76 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049696.357960 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199329.80 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 19932.98 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069629.337596 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179396.82 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 17939.68 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087569.019267 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161457.14 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 16145.71 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103714.732772 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145311.42 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 14531.14 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118245.874926 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130780.28 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 13078.03 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131323.902864 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117702.25 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 11770.23 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143094.128009 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105932.03 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 10593.20 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153687.330639 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95338.82 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 9533.88 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163221.213007 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85804.94 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 8580.49 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171801.707137 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77224.45 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 7722.44 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179524.151855 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69502.00 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 6950.20 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186474.352100 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62551.80 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 6255.18 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192729.532322 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56296.62 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 5629.66 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198359.194521 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50666.96 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 5066.70 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203425.890500 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45600.26 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 4560.03 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207985.916881 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41040.24 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 4104.02 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212089.940624 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36936.21 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3693.62 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215783.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33242.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219383.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29642.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222983.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26042.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226583.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22442.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230183.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18842.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233783.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15242.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237383.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11642.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240983.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8042.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244583.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4442.59 sec) 08:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248183.561993 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 842.59 sec) 08:17:00 [GPUGRID] [rr_sim] 2249026.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989074218.94G/439.78G) 08:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851887.07 08:17:00 [---] pick_jobs_to_run() start 08:17:00 [---] pick_jobs_to_run() end 08:17:00 [---] [rr_sim] end 08:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:17:00 [---] [cpu_sched_debug] preliminary job list: 08:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:17:00 [---] [cpu_sched_debug] final job list: 08:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:17:00 [---] [cpu_sched_debug] enforce_run_list: end 08:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:18:00 [---] [rr_sim] doing sim: CPU sched 08:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989046628.58G/439.78G) 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248963.42 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 224896.34 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224896.341743 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024067.08 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 202406.71 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427303.049312 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821660.37 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 182166.04 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609469.086123 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639494.33 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 163949.43 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773418.519254 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475544.90 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 147554.49 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920973.009072 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327990.41 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 132799.04 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053772.049907 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195191.37 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 119519.14 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173291.186660 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075672.23 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 107567.22 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280858.409737 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968105.01 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 96810.50 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377668.910506 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871294.51 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 87129.45 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464798.361198 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784165.06 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 78416.51 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543214.866822 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705748.55 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 70574.86 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613789.721882 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635173.70 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 63517.37 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677307.091437 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571656.33 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 57165.63 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734472.724036 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514490.69 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 51449.07 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785921.793376 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463041.62 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 46304.16 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832225.955781 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416737.46 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 41673.75 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873899.701946 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375063.72 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 37506.37 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911406.073494 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337557.34 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 33755.73 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945161.807888 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303801.61 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 30380.16 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975541.968842 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273421.45 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 27342.14 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002884.113701 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246079.30 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 24607.93 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027492.044074 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221471.37 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 22147.14 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049639.181409 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199324.24 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 19932.42 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069571.605011 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179391.81 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 17939.18 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087510.786253 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161452.63 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 16145.26 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103656.049371 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145307.37 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 14530.74 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118186.786177 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130776.63 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 13077.66 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131264.449302 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117698.97 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 11769.90 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143034.346115 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105929.07 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 10592.91 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153627.253246 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95336.16 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 9533.62 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163160.869665 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85802.55 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 8580.25 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171741.124441 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77222.29 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 7722.23 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179463.353740 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69500.06 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 6950.01 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186413.360109 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62550.06 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 6255.01 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192668.365841 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56295.05 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 5629.51 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198297.871000 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50665.55 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 5066.55 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203364.425643 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45598.99 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 4559.90 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207924.324822 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41039.09 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 4103.91 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212028.234082 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36935.18 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3693.52 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215721.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33241.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219321.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29641.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222921.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26041.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226521.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22441.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230121.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18841.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233721.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15241.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237321.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11641.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240921.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8041.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244521.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4441.67 sec) 08:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248121.752417 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 841.67 sec) 08:18:00 [GPUGRID] [rr_sim] 2248963.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989046628.58G/439.78G) 08:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851884.33 08:18:00 [---] pick_jobs_to_run() start 08:18:00 [---] pick_jobs_to_run() end 08:18:00 [---] [rr_sim] end 08:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:18:00 [---] [cpu_sched_debug] preliminary job list: 08:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:18:00 [---] [cpu_sched_debug] final job list: 08:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:18:00 [---] [cpu_sched_debug] enforce_run_list: end 08:18:00 [---] choose_project(): 29880.000000 08:18:00 [iThena.Measurements] doing RPC to get work 08:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405719.09 cpu 2150852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:18:00 [---] Got 0 tasks 08:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:19:00 [---] [rr_sim] doing sim: CPU sched 08:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (989019038.23G/439.78G) 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248900.68 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 224890.07 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224890.068055 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2024010.61 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 202401.06 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427291.129304 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821609.55 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 182160.96 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609452.084428 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639448.60 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 163944.86 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773396.944040 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475503.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 147550.37 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920947.317691 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327953.36 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 132795.34 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053742.653977 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195158.03 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 119515.80 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173258.456634 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075642.22 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 107564.22 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280822.679025 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968078.00 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 96807.80 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377630.479177 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871270.20 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 87127.02 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464757.499314 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784143.18 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 78414.32 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543171.817438 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705728.86 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 70572.89 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613744.703749 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635155.98 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 63515.60 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677260.301429 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571640.38 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 57164.04 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734424.339341 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514476.34 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 51447.63 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785871.973461 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463028.71 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 46302.87 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832174.844170 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416725.84 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 41672.58 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873847.427808 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375053.25 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 37505.33 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911352.753082 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337547.93 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 33754.79 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945107.545828 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303793.13 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 30379.31 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975486.859300 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273413.82 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 27341.38 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002828.241425 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246072.44 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 24607.24 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027435.485337 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221465.20 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 22146.52 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049582.004858 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199318.68 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 19931.87 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069513.872427 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179386.81 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 17938.68 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087452.553239 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161448.13 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 16144.81 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103597.365970 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145303.31 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 14530.33 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118127.697428 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130772.98 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 13077.30 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131204.995740 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117695.68 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 11769.57 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142974.564221 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105926.12 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 10592.61 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153567.175853 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95333.50 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 9533.35 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163100.526323 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85800.15 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 8580.02 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171680.541745 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77220.14 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 7722.01 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179402.555625 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69498.12 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 6949.81 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186352.368118 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62548.31 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 6254.83 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192607.199361 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56293.48 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 5629.35 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198236.547479 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50664.13 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 5066.41 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203302.960786 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45597.72 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 4559.77 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207862.732762 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41037.95 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 4103.79 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211966.527541 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36934.15 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3693.42 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215659.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33240.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219259.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29640.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222859.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26040.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226459.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22440.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230059.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18840.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233659.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15240.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237259.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11640.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240859.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8040.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244459.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4440.74 sec) 08:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2248059.942842 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 840.74 sec) 08:19:00 [GPUGRID] [rr_sim] 2248900.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (989019038.23G/439.78G) 08:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851881.59 08:19:00 [---] pick_jobs_to_run() start 08:19:00 [---] pick_jobs_to_run() end 08:19:00 [---] [rr_sim] end 08:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:19:00 [---] [cpu_sched_debug] preliminary job list: 08:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:19:00 [---] [cpu_sched_debug] final job list: 08:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:19:00 [---] [cpu_sched_debug] enforce_run_list: end 08:19:00 [---] choose_project(): 29940.000000 08:19:00 [iThena.Measurements] doing RPC to get work 08:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405659.09 cpu 2150792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:19:00 [---] Got 0 tasks 08:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:20:00 [---] [rr_sim] doing sim: CPU sched 08:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988991447.87G/439.78G) 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248837.94 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 224883.79 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224883.794367 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023954.15 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 202395.41 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427279.209296 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821558.73 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 182155.87 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609435.082733 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639402.86 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 163940.29 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773375.368827 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475462.57 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 147546.26 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920921.626310 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327916.32 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 132791.63 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053713.258046 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195124.69 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 119512.47 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173225.726608 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075612.22 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 107561.22 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280786.948314 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968051.00 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 96805.10 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377592.047849 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871245.90 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 87124.59 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464716.637430 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784121.31 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 78412.13 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543128.768054 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705709.18 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 70570.92 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613699.685615 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635138.26 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 63513.83 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677213.511420 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571624.43 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 57162.44 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734375.954645 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514461.99 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 51446.20 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785822.153547 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463015.79 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 46301.58 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832123.732559 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416714.21 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 41671.42 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873795.153669 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375042.79 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 37504.28 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911299.432669 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337538.51 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 33753.85 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945053.283769 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303784.66 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 30378.47 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975431.749758 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273406.19 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 27340.62 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002772.369149 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246065.57 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 24606.56 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027378.926601 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221459.02 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 22145.90 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049524.828307 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199313.12 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 19931.31 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069456.139843 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179381.80 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 17938.18 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087394.320225 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161443.62 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 16144.36 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103538.682569 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145299.26 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 14529.93 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118068.608679 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130769.33 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 13076.93 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131145.542177 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117692.40 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 11769.24 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142914.782326 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105923.16 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 10592.32 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153507.098460 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95330.85 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 9533.08 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163040.182981 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85797.76 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 8579.78 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171619.959049 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77217.98 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 7721.80 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179341.757511 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69496.19 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 6949.62 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186291.376126 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62546.57 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 6254.66 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192546.032880 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56291.91 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 5629.19 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198175.223959 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50662.72 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 5066.27 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203241.495929 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45596.45 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 4559.64 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207801.140703 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41036.80 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 4103.68 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211904.820999 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36933.12 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3693.31 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215598.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33239.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219198.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29639.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222798.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26039.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226398.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22439.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229998.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18839.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233598.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15239.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237198.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11639.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240798.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8039.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244398.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4439.81 sec) 08:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247998.133266 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 839.81 sec) 08:20:00 [GPUGRID] [rr_sim] 2248837.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988991447.87G/439.78G) 08:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851878.86 08:20:00 [---] pick_jobs_to_run() start 08:20:00 [---] pick_jobs_to_run() end 08:20:00 [---] [rr_sim] end 08:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:20:00 [---] [cpu_sched_debug] preliminary job list: 08:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:20:00 [---] [cpu_sched_debug] final job list: 08:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:20:00 [---] [cpu_sched_debug] enforce_run_list: end 08:20:00 [---] choose_project(): 30000.000000 08:20:00 [iThena.Measurements] doing RPC to get work 08:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405599.09 cpu 2150732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:20:00 [---] Got 0 tasks 08:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:21:00 [---] [rr_sim] doing sim: CPU sched 08:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988963857.52G/439.78G) 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248775.21 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 224877.52 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224877.520678 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023897.69 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 202389.77 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427267.289289 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821507.92 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 182150.79 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609418.081038 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639357.13 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 163935.71 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773353.793613 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475421.41 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 147542.14 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920895.934930 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327879.27 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 132787.93 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053683.862115 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195091.34 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 119509.13 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173192.996582 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075582.21 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 107558.22 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280751.217602 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 968023.99 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 96802.40 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377553.616520 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871221.59 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 87122.16 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464675.775547 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784099.43 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 78409.94 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543085.718670 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705689.49 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 70568.95 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613654.667482 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635120.54 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 63512.05 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677166.721412 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571608.49 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 57160.85 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734327.569949 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514447.64 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 51444.76 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785772.333632 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463002.87 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 46300.29 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832072.620947 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416702.59 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 41670.26 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873742.879531 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375032.33 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 37503.23 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911246.112256 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337529.09 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 33752.91 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944999.021709 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303776.19 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 30377.62 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975376.640216 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273398.57 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 27339.86 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002716.496873 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246058.71 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 24605.87 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027322.367864 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221452.84 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 22145.28 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049467.651756 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199307.56 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 19930.76 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069398.407259 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179376.80 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 17937.68 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087336.087211 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161439.12 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 16143.91 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103479.999168 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145295.21 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 14529.52 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118009.519930 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130765.69 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 13076.57 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131086.088615 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117689.12 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 11768.91 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142855.000432 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105920.21 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 10592.02 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153447.021067 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95328.19 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 9532.82 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162979.839639 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85795.37 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 8579.54 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171559.376353 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77215.83 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 7721.58 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179280.959396 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69494.25 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 6949.42 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186230.384135 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62544.82 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 6254.48 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192484.866400 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56290.34 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 5629.03 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198113.900438 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50661.31 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 5066.13 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203180.031073 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45595.18 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 4559.52 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207739.548644 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41035.66 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 4103.57 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211843.114458 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36932.09 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3693.21 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215536.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33238.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219136.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29638.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222736.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26038.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226336.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22438.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229936.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18838.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233536.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15238.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237136.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11638.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240736.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8038.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244336.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4438.88 sec) 08:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247936.323690 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 838.88 sec) 08:21:00 [GPUGRID] [rr_sim] 2248775.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988963857.52G/439.78G) 08:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851876.12 08:21:00 [---] pick_jobs_to_run() start 08:21:00 [---] pick_jobs_to_run() end 08:21:00 [---] [rr_sim] end 08:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:21:00 [---] [cpu_sched_debug] preliminary job list: 08:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:21:00 [---] [cpu_sched_debug] final job list: 08:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:21:00 [---] [cpu_sched_debug] enforce_run_list: end 08:21:00 [---] choose_project(): 30060.000000 08:21:00 [iThena.Measurements] doing RPC to get work 08:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405539.09 cpu 2150672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:21:00 [---] Got 0 tasks 08:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:22:00 [---] [rr_sim] doing sim: CPU sched 08:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988936267.16G/439.78G) 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248712.47 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 224871.25 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224871.246990 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023841.22 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 202384.12 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427255.369281 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821457.10 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 182145.71 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609401.079343 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639311.39 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 163931.14 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773332.218399 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475380.25 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 147538.03 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920870.243549 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327842.23 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 132784.22 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053654.466184 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195058.00 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 119505.80 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173160.266556 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075552.20 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 107555.22 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280715.486891 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967996.98 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 96799.70 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377515.185192 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871197.28 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 87119.73 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464634.913663 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784077.56 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 78407.76 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1543042.669286 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705669.80 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 70566.98 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613609.649348 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635102.82 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 63510.28 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677119.931403 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571592.54 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 57159.25 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734279.185253 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514433.28 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 51443.33 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785722.513718 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462989.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 46299.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832021.509336 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416690.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 41669.10 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873690.605393 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375021.86 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 37502.19 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911192.791843 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337519.68 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 33751.97 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944944.759649 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303767.71 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 30376.77 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975321.530674 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273390.94 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 27339.09 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002660.624597 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246051.85 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 24605.18 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027265.809128 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221446.66 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 22144.67 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049410.475205 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199301.99 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 19930.20 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069340.674674 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179371.80 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 17937.18 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087277.854197 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161434.62 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 16143.46 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103421.315768 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145291.15 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 14529.12 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117950.431181 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130762.04 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 13076.20 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131026.635053 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117685.83 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 11768.58 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142795.218538 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105917.25 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 10591.73 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153386.943674 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95325.53 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 9532.55 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162919.496297 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85792.97 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 8579.30 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171498.793657 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77213.68 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 7721.37 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179220.161282 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69492.31 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 6949.23 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186169.392144 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62543.08 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 6254.31 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192423.699919 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56288.77 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 5628.88 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198052.576917 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50659.89 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 5065.99 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203118.566216 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45593.90 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 4559.39 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207677.956584 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41034.51 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 4103.45 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211781.407916 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36931.06 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3693.11 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215474.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33237.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219074.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29637.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222674.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26037.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226274.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22437.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229874.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18837.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233474.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15237.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237074.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11637.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240674.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8037.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244274.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4437.96 sec) 08:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247874.514115 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 837.96 sec) 08:22:00 [GPUGRID] [rr_sim] 2248712.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988936267.16G/439.78G) 08:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851873.38 08:22:00 [---] pick_jobs_to_run() start 08:22:00 [---] pick_jobs_to_run() end 08:22:00 [---] [rr_sim] end 08:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:22:00 [---] [cpu_sched_debug] preliminary job list: 08:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:22:00 [---] [cpu_sched_debug] final job list: 08:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:22:00 [---] [cpu_sched_debug] enforce_run_list: end 08:22:00 [---] choose_project(): 30120.000000 08:22:00 [iThena.Measurements] doing RPC to get work 08:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405479.09 cpu 2150612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:22:00 [---] Got 0 tasks 08:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:23:00 [---] [rr_sim] doing sim: CPU sched 08:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988908676.80G/439.78G) 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248649.73 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 224864.97 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224864.973302 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023784.76 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 202378.48 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427243.449274 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821406.28 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 182140.63 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609384.077648 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639265.66 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 163926.57 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773310.643185 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475339.09 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 147533.91 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920844.552169 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327805.18 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 132780.52 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053625.070254 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1195024.66 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 119502.47 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173127.536530 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075522.20 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 107552.22 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280679.756179 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967969.98 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 96797.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377476.753863 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871172.98 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 87117.30 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464594.051779 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784055.68 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 78405.57 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542999.619903 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705650.11 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 70565.01 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613564.631214 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635085.10 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 63508.51 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677073.141395 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571576.59 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 57157.66 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734230.800557 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514418.93 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 51441.89 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785672.693803 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462977.04 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 46297.70 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831970.397725 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416679.34 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 41667.93 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873638.331254 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375011.40 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 37501.14 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911139.471431 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337510.26 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 33751.03 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944890.497590 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303759.24 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 30375.92 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975266.421133 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273383.31 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 27338.33 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002604.752321 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246044.98 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 24604.50 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027209.250391 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221440.48 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 22144.05 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049353.298654 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199296.43 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 19929.64 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069282.942090 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179366.79 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 17936.68 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087219.621183 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161430.11 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 16143.01 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103362.632367 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145287.10 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 14528.71 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117891.342432 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130758.39 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 13075.84 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130967.181491 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117682.55 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 11768.26 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142735.436643 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105914.30 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 10591.43 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153326.866281 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95322.87 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 9532.29 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162859.152955 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85790.58 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 8579.06 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171438.210961 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77211.52 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 7721.15 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179159.363167 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69490.37 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 6949.04 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186108.400152 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62541.33 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 6254.13 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192362.533439 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56287.20 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 5628.72 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197991.253397 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50658.48 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 5065.85 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203057.101359 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45592.63 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 4559.26 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207616.364525 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41033.37 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 4103.34 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211719.701374 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36930.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3693.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215412.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33237.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219012.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29637.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222612.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26037.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226212.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22437.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229812.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18837.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233412.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15237.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237012.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11637.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240612.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8037.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244212.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4437.03 sec) 08:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247812.704539 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 837.03 sec) 08:23:00 [GPUGRID] [rr_sim] 2248649.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988908676.80G/439.78G) 08:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851870.65 08:23:00 [---] pick_jobs_to_run() start 08:23:00 [---] pick_jobs_to_run() end 08:23:00 [---] [rr_sim] end 08:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:23:00 [---] [cpu_sched_debug] preliminary job list: 08:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:23:00 [---] [cpu_sched_debug] final job list: 08:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:23:00 [---] [cpu_sched_debug] enforce_run_list: end 08:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:24:00 [---] [rr_sim] doing sim: CPU sched 08:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988881086.45G/439.78G) 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248587.00 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 224858.70 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224858.699614 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023728.30 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 202372.83 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427231.529266 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821355.47 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 182135.55 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609367.075953 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639219.92 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 163921.99 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773289.067971 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475297.93 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 147529.79 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920818.860788 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327768.14 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 132776.81 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053595.674323 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194991.32 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 119499.13 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173094.806504 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075492.19 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 107549.22 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280644.025468 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967942.97 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 96794.30 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377438.322534 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871148.67 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 87114.87 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464553.189895 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784033.81 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 78403.38 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542956.570519 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705630.43 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 70563.04 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613519.613081 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635067.38 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 63506.74 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1677026.351386 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571560.64 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 57156.06 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734182.415861 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514404.58 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 51440.46 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785622.873889 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462964.12 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 46296.41 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831919.286114 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416667.71 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 41666.77 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873586.057116 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375000.94 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 37500.09 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911086.151018 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337500.85 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 33750.08 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944836.235530 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303750.76 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 30375.08 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975211.311591 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273375.68 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 27337.57 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002548.880045 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246038.12 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 24603.81 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027152.691654 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221434.30 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 22143.43 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049296.122103 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199290.87 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 19929.09 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069225.209506 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179361.79 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 17936.18 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087161.388169 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161425.61 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 16142.56 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103303.948966 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145283.05 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 14528.30 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117832.253683 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130754.74 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 13075.47 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130907.727928 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117679.27 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 11767.93 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142675.654749 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105911.34 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 10591.13 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153266.788888 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95320.21 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 9532.02 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162798.809613 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85788.19 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 8578.82 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171377.628265 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77209.37 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 7720.94 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179098.565052 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69488.43 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 6948.84 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186047.408161 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62539.59 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 6253.96 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192301.366958 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56285.63 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 5628.56 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197929.929876 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50657.07 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 5065.71 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202995.636502 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45591.36 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 4559.14 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207554.772466 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41032.22 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 4103.22 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211657.994833 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36929.00 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3692.90 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215350.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33236.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218950.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29636.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222550.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26036.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226150.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22436.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229750.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18836.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233350.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15236.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236950.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11636.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240550.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8036.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244150.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4436.10 sec) 08:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247750.894963 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 836.10 sec) 08:24:00 [GPUGRID] [rr_sim] 2248587.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988881086.45G/439.78G) 08:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851867.91 08:24:00 [---] pick_jobs_to_run() start 08:24:00 [---] pick_jobs_to_run() end 08:24:00 [---] [rr_sim] end 08:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:24:00 [---] [cpu_sched_debug] preliminary job list: 08:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:24:00 [---] [cpu_sched_debug] final job list: 08:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:24:00 [---] [cpu_sched_debug] enforce_run_list: end 08:24:00 [---] choose_project(): 30240.000000 08:24:00 [iThena.Measurements] doing RPC to get work 08:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405359.09 cpu 2150492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:24:00 [---] Got 0 tasks 08:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:25:00 [---] [rr_sim] doing sim: CPU sched 08:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988853496.09G/439.78G) 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248524.26 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 224852.43 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224852.425925 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023671.83 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 202367.18 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427219.609258 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821304.65 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 182130.46 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609350.074258 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639174.18 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 163917.42 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773267.492758 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475256.77 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 147525.68 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920793.169407 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327731.09 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 132773.11 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053566.278392 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194957.98 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 119495.80 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173062.076478 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075462.18 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 107546.22 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280608.294756 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967915.96 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 96791.60 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377399.891206 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871124.37 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 87112.44 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464512.328011 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784011.93 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 78401.19 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542913.521135 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705610.74 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 70561.07 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613474.594947 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635049.66 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 63504.97 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676979.561378 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571544.70 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 57154.47 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734134.031165 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514390.23 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 51439.02 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785573.053974 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462951.21 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 46295.12 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831868.174502 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416656.08 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 41665.61 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873533.782978 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374990.48 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 37499.05 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911032.830605 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337491.43 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 33749.14 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944781.973470 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303742.29 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 30374.23 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975156.202049 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273368.06 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 27336.81 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002493.007769 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246031.25 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 24603.13 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027096.132918 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221428.13 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 22142.81 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049238.945552 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199285.31 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 19928.53 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069167.476922 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179356.78 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 17935.68 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087103.155155 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161421.10 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 16142.11 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103245.265565 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145278.99 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 14527.90 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117773.164934 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130751.09 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 13075.11 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130848.274366 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117675.98 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 11767.60 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142615.872855 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105908.39 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 10590.84 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153206.711495 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95317.55 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 9531.75 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162738.466271 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85785.79 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 8578.58 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171317.045569 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77207.21 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 7720.72 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179037.766938 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69486.49 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 6948.65 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185986.416169 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62537.84 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 6253.78 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192240.200478 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56284.06 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 5628.41 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197868.606356 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50655.65 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 5065.57 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202934.171646 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45590.09 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 4559.01 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207493.180406 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41031.08 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 4103.11 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211596.288291 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36927.97 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3692.80 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215289.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33235.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218889.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29635.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222489.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26035.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226089.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22435.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229689.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18835.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233289.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15235.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236889.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11635.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240489.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8035.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244089.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4435.17 sec) 08:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247689.085388 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 835.17 sec) 08:25:00 [GPUGRID] [rr_sim] 2248524.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988853496.09G/439.78G) 08:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851865.17 08:25:00 [---] pick_jobs_to_run() start 08:25:00 [---] pick_jobs_to_run() end 08:25:00 [---] [rr_sim] end 08:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:25:00 [---] [cpu_sched_debug] preliminary job list: 08:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:25:00 [---] [cpu_sched_debug] final job list: 08:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:25:00 [---] [cpu_sched_debug] enforce_run_list: end 08:25:00 [---] choose_project(): 30300.000000 08:25:00 [iThena.Measurements] doing RPC to get work 08:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405299.09 cpu 2150432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:25:00 [---] Got 0 tasks 08:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:26:00 [---] [rr_sim] doing sim: CPU sched 08:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988825905.73G/439.78G) 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248461.52 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 224846.15 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224846.152237 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023615.37 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 202361.54 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427207.689251 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821253.83 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 182125.38 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609333.072563 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639128.45 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 163912.84 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773245.917544 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475215.60 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 147521.56 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920767.478027 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327694.04 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 132769.40 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053536.882461 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194924.64 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 119492.46 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1173029.346452 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075432.18 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 107543.22 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280572.564044 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967888.96 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 96788.90 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377361.459877 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871100.06 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 87110.01 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464471.466127 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783990.06 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 78399.01 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542870.471751 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705591.05 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 70559.11 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613429.576813 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635031.95 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 63503.19 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676932.771369 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571528.75 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 57152.88 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734085.646470 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514375.88 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 51437.59 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785523.234060 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462938.29 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 46293.83 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831817.062891 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416644.46 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 41664.45 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873481.508839 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374980.01 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 37498.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910979.510193 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337482.01 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 33748.20 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944727.711411 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303733.81 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 30373.38 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975101.092507 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273360.43 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 27336.04 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002437.135493 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246024.39 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 24602.44 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027039.574181 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221421.95 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 22142.19 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049181.769000 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199279.75 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 19927.98 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069109.744338 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179351.78 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 17935.18 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087044.922141 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161416.60 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 16141.66 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103186.582164 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145274.94 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 14527.49 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117714.076185 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130747.45 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 13074.74 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130788.820804 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117672.70 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 11767.27 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142556.090961 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105905.43 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 10590.54 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153146.634102 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95314.89 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 9531.49 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162678.122929 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85783.40 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 8578.34 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171256.462873 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77205.06 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 7720.51 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178976.968823 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69484.55 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 6948.46 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185925.424178 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62536.10 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 6253.61 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192179.033998 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56282.49 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 5628.25 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197807.282835 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50654.24 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 5065.42 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202872.706789 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45588.82 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 4558.88 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207431.588347 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41029.93 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 4102.99 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211534.581750 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36926.94 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3692.69 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215227.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33234.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218827.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29634.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222427.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26034.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226027.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22434.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229627.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18834.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233227.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15234.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236827.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11634.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240427.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8034.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244027.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4434.25 sec) 08:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247627.275812 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 834.25 sec) 08:26:00 [GPUGRID] [rr_sim] 2248461.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988825905.73G/439.78G) 08:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851862.44 08:26:00 [---] pick_jobs_to_run() start 08:26:00 [---] pick_jobs_to_run() end 08:26:00 [---] [rr_sim] end 08:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:26:00 [---] [cpu_sched_debug] preliminary job list: 08:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:26:00 [---] [cpu_sched_debug] final job list: 08:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:26:00 [---] [cpu_sched_debug] enforce_run_list: end 08:26:00 [---] choose_project(): 30360.000000 08:26:00 [iThena.Measurements] doing RPC to get work 08:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 578039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405239.09 cpu 2150372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:26:00 [---] Got 0 tasks 08:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:27:00 [---] [rr_sim] doing sim: CPU sched 08:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988798315.38G/439.78G) 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248398.79 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 224839.88 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224839.878549 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023558.91 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 202355.89 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427195.769243 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821203.02 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 182120.30 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609316.070868 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639082.71 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 163908.27 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773224.342330 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475174.44 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 147517.44 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920741.786646 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327657.00 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 132765.70 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053507.486531 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194891.30 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 119489.13 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172996.616427 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075402.17 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 107540.22 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280536.833333 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967861.95 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 96786.20 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377323.028549 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871075.76 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 87107.58 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464430.604243 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783968.18 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 78396.82 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542827.422368 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705571.36 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 70557.14 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613384.558680 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635014.23 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 63501.42 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676885.981361 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571512.80 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 57151.28 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734037.261774 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514361.52 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 51436.15 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785473.414145 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462925.37 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 46292.54 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831765.951280 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416632.83 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 41663.28 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873429.234701 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374969.55 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 37496.96 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910926.189780 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337472.60 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 33747.26 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944673.449351 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303725.34 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 30372.53 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975045.982965 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273352.80 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 27335.28 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002381.263217 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246017.52 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 24601.75 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026983.015445 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221415.77 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 22141.58 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049124.592449 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199274.19 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 19927.42 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069052.011753 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179346.77 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 17934.68 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086986.689127 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161412.10 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 16141.21 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103127.898763 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145270.89 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 14527.09 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117654.987436 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130743.80 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 13074.38 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130729.367241 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117669.42 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 11766.94 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142496.309066 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105902.48 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 10590.25 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153086.556709 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95312.23 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 9531.22 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162617.779587 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85781.01 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 8578.10 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171195.880177 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77202.91 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 7720.29 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178916.170709 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69482.61 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 6948.26 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185864.432187 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62534.35 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 6253.44 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192117.867517 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56280.92 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 5628.09 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197745.959314 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50652.83 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 5065.28 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202811.241932 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45587.54 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 4558.75 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207369.996288 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41028.79 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 4102.88 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211472.875208 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36925.91 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3692.59 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215165.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33233.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218765.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29633.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222365.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26033.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225965.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22433.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229565.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18833.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233165.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15233.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236765.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11633.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240365.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8033.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243965.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4433.32 sec) 08:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247565.466236 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 833.32 sec) 08:27:00 [GPUGRID] [rr_sim] 2248398.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988798315.38G/439.78G) 08:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851859.70 08:27:00 [---] pick_jobs_to_run() start 08:27:00 [---] pick_jobs_to_run() end 08:27:00 [---] [rr_sim] end 08:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:27:00 [---] [cpu_sched_debug] preliminary job list: 08:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:27:00 [---] [cpu_sched_debug] final job list: 08:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:27:00 [---] [cpu_sched_debug] enforce_run_list: end 08:27:00 [---] choose_project(): 30420.000000 08:27:00 [iThena.Measurements] doing RPC to get work 08:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405179.09 cpu 2150312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:27:00 [---] Got 0 tasks 08:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:28:00 [---] [rr_sim] doing sim: CPU sched 08:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988798315.38G/439.78G) 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248398.79 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 224839.88 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224839.878549 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023558.91 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 202355.89 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427195.769243 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821203.02 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 182120.30 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609316.070868 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639082.71 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 163908.27 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773224.342330 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475174.44 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 147517.44 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920741.786646 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327657.00 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 132765.70 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053507.486531 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194891.30 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 119489.13 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172996.616427 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075402.17 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 107540.22 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280536.833333 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967861.95 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 96786.20 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377323.028549 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871075.76 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 87107.58 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464430.604243 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783968.18 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 78396.82 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542827.422368 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705571.36 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 70557.14 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613384.558680 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635014.23 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 63501.42 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676885.981361 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571512.80 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 57151.28 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1734037.261774 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514361.52 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 51436.15 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785473.414145 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462925.37 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 46292.54 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831765.951280 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416632.83 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 41663.28 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873429.234701 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374969.55 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 37496.96 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910926.189780 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337472.60 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 33747.26 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944673.449351 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303725.34 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 30372.53 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975045.982965 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273352.80 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 27335.28 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002381.263217 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246017.52 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 24601.75 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026983.015445 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221415.77 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 22141.58 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049124.592449 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199274.19 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 19927.42 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069052.011753 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179346.77 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 17934.68 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086986.689127 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161412.10 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 16141.21 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103127.898763 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145270.89 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 14527.09 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117654.987436 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130743.80 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 13074.38 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130729.367241 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117669.42 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 11766.94 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142496.309066 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105902.48 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 10590.25 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153086.556709 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95312.23 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 9531.22 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162617.779587 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85781.01 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 8578.10 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171195.880177 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77202.91 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 7720.29 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178916.170709 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69482.61 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 6948.26 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185864.432187 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62534.35 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 6253.44 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192117.867517 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56280.92 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 5628.09 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197745.959314 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50652.83 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 5065.28 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202811.241932 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45587.54 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 4558.75 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207369.996288 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41028.79 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 4102.88 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211472.875208 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36925.91 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3692.59 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215165.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33233.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218765.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29633.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222365.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26033.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225965.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22433.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229565.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18833.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233165.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15233.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236765.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11633.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240365.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8033.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243965.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4433.32 sec) 08:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247565.466236 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 833.32 sec) 08:28:00 [GPUGRID] [rr_sim] 2248398.79: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988798315.38G/439.78G) 08:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851919.70 08:28:00 [---] pick_jobs_to_run() start 08:28:00 [---] pick_jobs_to_run() end 08:28:00 [---] [rr_sim] end 08:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:28:00 [---] [cpu_sched_debug] preliminary job list: 08:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:28:00 [---] [cpu_sched_debug] final job list: 08:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:28:00 [---] [cpu_sched_debug] enforce_run_list: end 08:28:00 [---] choose_project(): 30480.000000 08:28:00 [iThena.Measurements] doing RPC to get work 08:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405119.09 cpu 2150312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:28:00 [---] Got 0 tasks 08:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:29:00 [---] [rr_sim] doing sim: CPU sched 08:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988770725.02G/439.78G) 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248336.05 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 224833.60 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224833.604861 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023502.44 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 202350.24 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427183.849236 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821152.20 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 182115.22 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609299.069173 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639036.98 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 163903.70 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773202.767116 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475133.28 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 147513.33 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920716.095266 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327619.95 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 132762.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053478.090600 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194857.96 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 119485.80 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172963.886401 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075372.16 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 107537.22 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280501.102621 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967834.95 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 96783.49 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377284.597220 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871051.45 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 87105.15 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464389.742359 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783946.31 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 78394.63 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542784.372984 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705551.68 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 70555.17 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613339.540546 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634996.51 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 63499.65 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676839.191352 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571496.86 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 57149.69 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733988.877078 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514347.17 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 51434.72 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785423.594231 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462912.45 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 46291.25 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831714.839669 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416621.21 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 41662.12 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873376.960563 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374959.09 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 37495.91 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910872.869367 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337463.18 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 33746.32 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944619.187291 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303716.86 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 30371.69 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974990.873423 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273345.18 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 27334.52 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002325.390941 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246010.66 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 24601.07 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026926.456708 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221409.59 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 22140.96 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049067.415898 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199268.63 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 19926.86 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068994.279169 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179341.77 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 17934.18 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086928.456113 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161407.59 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 16140.76 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103069.215363 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145266.83 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 14526.68 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117595.898687 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130740.15 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 13074.01 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130669.913679 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117666.13 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 11766.61 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142436.527172 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105899.52 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 10589.95 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153026.479316 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95309.57 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 9530.96 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162557.436245 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85778.61 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 8577.86 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171135.297481 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77200.75 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 7720.08 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178855.372594 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69480.68 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 6948.07 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185803.440195 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62532.61 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 6253.26 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192056.701037 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56279.35 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 5627.93 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197684.635794 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50651.41 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 5065.14 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202749.777075 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45586.27 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 4558.63 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207308.404229 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41027.64 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 4102.76 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211411.168666 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36924.88 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3692.49 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215103.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33232.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218703.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29632.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222303.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26032.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225903.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22432.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229503.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18832.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233103.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15232.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236703.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11632.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240303.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8032.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243903.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4432.39 sec) 08:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247503.656661 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 832.39 sec) 08:29:00 [GPUGRID] [rr_sim] 2248336.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988770725.02G/439.78G) 08:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851916.96 08:29:00 [---] pick_jobs_to_run() start 08:29:00 [---] pick_jobs_to_run() end 08:29:00 [---] [rr_sim] end 08:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:29:00 [---] [cpu_sched_debug] preliminary job list: 08:29:00 [---] [cpu_sched_debug] final job list: 08:29:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 08:29:00 [---] [work_fetch] Request work fetch: CPUs idle 08:29:00 [---] [cpu_sched_debug] enforce_run_list: end 08:29:00 [---] choose_project(): 30540.000000 08:29:00 [iThena.Measurements] doing RPC to get work 08:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 405059.09 cpu 2150252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:29:00 [---] Got 0 tasks 08:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:30:00 [---] [rr_sim] doing sim: CPU sched 08:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988770725.02G/439.78G) 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248336.05 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 224833.60 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224833.604861 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023502.44 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 202350.24 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427183.849236 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821152.20 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 182115.22 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609299.069173 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1639036.98 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 163903.70 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773202.767116 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475133.28 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 147513.33 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920716.095266 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327619.95 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 132762.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053478.090600 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194857.96 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 119485.80 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172963.886401 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075372.16 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 107537.22 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280501.102621 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967834.95 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 96783.49 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377284.597220 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871051.45 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 87105.15 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464389.742359 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783946.31 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 78394.63 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542784.372984 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705551.68 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 70555.17 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613339.540546 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634996.51 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 63499.65 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676839.191352 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571496.86 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 57149.69 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733988.877078 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514347.17 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 51434.72 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785423.594231 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462912.45 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 46291.25 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831714.839669 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416621.21 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 41662.12 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873376.960563 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374959.09 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 37495.91 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910872.869367 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337463.18 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 33746.32 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944619.187291 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303716.86 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 30371.69 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974990.873423 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273345.18 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 27334.52 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002325.390941 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246010.66 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 24601.07 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026926.456708 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221409.59 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 22140.96 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049067.415898 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199268.63 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 19926.86 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068994.279169 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179341.77 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 17934.18 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086928.456113 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161407.59 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 16140.76 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103069.215363 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145266.83 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 14526.68 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117595.898687 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130740.15 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 13074.01 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130669.913679 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117666.13 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 11766.61 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142436.527172 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105899.52 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 10589.95 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153026.479316 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95309.57 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 9530.96 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162557.436245 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85778.61 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 8577.86 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171135.297481 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77200.75 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 7720.08 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178855.372594 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69480.68 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 6948.07 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185803.440195 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62532.61 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 6253.26 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192056.701037 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56279.35 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 5627.93 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197684.635794 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50651.41 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 5065.14 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202749.777075 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45586.27 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 4558.63 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207308.404229 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41027.64 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 4102.76 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211411.168666 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36924.88 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3692.49 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215103.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33232.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218703.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29632.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222303.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26032.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225903.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22432.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229503.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18832.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233103.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15232.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236703.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11632.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240303.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8032.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243903.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4432.39 sec) 08:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247503.656661 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 832.39 sec) 08:30:00 [GPUGRID] [rr_sim] 2248336.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988770725.02G/439.78G) 08:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851976.96 08:30:00 [---] pick_jobs_to_run() start 08:30:00 [---] pick_jobs_to_run() end 08:30:00 [---] [rr_sim] end 08:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:30:00 [---] [cpu_sched_debug] preliminary job list: 08:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:30:00 [---] [cpu_sched_debug] final job list: 08:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:30:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 988770725.02G 08:30:00 [---] [cpu_sched_debug] enforce_run_list: end 08:30:00 [---] choose_project(): 30600.000000 08:30:00 [iThena.Measurements] doing RPC to get work 08:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404999.09 cpu 2150252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:30:00 [---] Got 0 tasks 08:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:31:00 [---] [rr_sim] doing sim: CPU sched 08:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988743134.67G/439.78G) 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248273.31 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 224827.33 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224827.331173 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023445.98 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 202344.60 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427171.929228 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821101.38 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 182110.14 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609282.067478 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638991.24 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 163899.12 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773181.191903 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475092.12 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 147509.21 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920690.403885 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327582.91 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 132758.29 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053448.694669 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194824.62 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 119482.46 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172931.156375 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075342.16 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 107534.22 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280465.371910 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967807.94 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 96780.79 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377246.165891 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871027.15 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 87102.71 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464348.880475 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783924.43 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 78392.44 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542741.323600 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705531.99 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 70553.20 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613294.522413 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634978.79 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 63497.88 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676792.401344 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571480.91 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 57148.09 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733940.492382 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514332.82 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 51433.28 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785373.774317 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462899.54 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 46289.95 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831663.728057 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416609.58 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 41660.96 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873324.686424 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374948.63 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 37494.86 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910819.548954 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337453.76 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 33745.38 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944564.925232 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303708.39 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 30370.84 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974935.763881 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273337.55 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 27333.75 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002269.518666 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246003.79 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 24600.38 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026869.897972 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221403.41 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 22140.34 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049010.239347 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199263.07 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 19926.31 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068936.546585 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179336.77 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 17933.68 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086870.223099 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161403.09 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 16140.31 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103010.531962 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145262.78 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 14526.28 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117536.809938 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130736.50 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 13073.65 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130610.460117 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117662.85 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 11766.29 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142376.745278 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105896.57 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 10589.66 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152966.401923 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95306.91 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 9530.69 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162497.092903 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85776.22 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 8577.62 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171074.714785 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77198.60 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 7719.86 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178794.574479 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69478.74 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 6947.87 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185742.448204 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62530.86 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 6253.09 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191995.534556 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56277.78 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 5627.78 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197623.312273 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50650.00 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 5065.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202688.312218 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45585.00 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 4558.50 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207246.812169 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41026.50 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 4102.65 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211349.462125 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36923.85 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3692.38 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215041.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33231.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218641.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29631.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222241.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26031.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225841.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22431.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229441.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18831.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233041.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15231.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236641.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11631.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240241.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8031.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243841.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4431.46 sec) 08:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247441.847085 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 831.46 sec) 08:31:00 [GPUGRID] [rr_sim] 2248273.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988743134.67G/439.78G) 08:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851974.22 08:31:00 [---] pick_jobs_to_run() start 08:31:00 [---] pick_jobs_to_run() end 08:31:00 [---] [rr_sim] end 08:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:31:00 [---] [cpu_sched_debug] preliminary job list: 08:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:31:00 [---] [cpu_sched_debug] final job list: 08:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:31:00 [---] [cpu_sched_debug] enforce_run_list: end 08:31:00 [---] choose_project(): 30660.000000 08:31:00 [iThena.Measurements] doing RPC to get work 08:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404939.09 cpu 2150192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:31:00 [---] Got 0 tasks 08:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:32:00 [---] [rr_sim] doing sim: CPU sched 08:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988715544.31G/439.78G) 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248210.57 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 224821.06 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224821.057484 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023389.52 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 202338.95 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427160.009220 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1821050.57 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 182105.06 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609265.065783 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638945.51 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 163894.55 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773159.616689 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475050.96 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 147505.10 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920664.712504 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327545.86 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 132754.59 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053419.298738 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194791.28 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 119479.13 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172898.426349 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075312.15 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 107531.21 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280429.641198 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967780.93 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 96778.09 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377207.734563 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 871002.84 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 87100.28 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464308.018591 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783902.56 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 78390.26 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542698.274216 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705512.30 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 70551.23 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613249.504279 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634961.07 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 63496.11 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676745.611335 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571464.96 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 57146.50 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733892.107686 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514318.47 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 51431.85 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785323.954402 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462886.62 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 46288.66 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831612.616446 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416597.96 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 41659.80 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873272.412286 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374938.16 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 37493.82 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910766.228542 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337444.35 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 33744.43 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944510.663172 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303699.91 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 30369.99 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974880.654339 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273329.92 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 27332.99 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002213.646390 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245996.93 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 24599.69 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026813.339235 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221397.24 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 22139.72 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048953.062796 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199257.51 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 19925.75 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068878.814001 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179331.76 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 17933.18 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086811.990085 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161398.58 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 16139.86 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102951.848561 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145258.73 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 14525.87 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117477.721189 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130732.85 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 13073.29 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130551.006555 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117659.57 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 11765.96 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142316.963384 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105893.61 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 10589.36 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152906.324530 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95304.25 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 9530.43 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162436.749561 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85773.83 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 8577.38 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171014.132089 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77196.44 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 7719.64 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178733.776365 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69476.80 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 6947.68 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185681.456213 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62529.12 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 6252.91 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191934.368076 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56276.21 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 5627.62 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197561.988753 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50648.59 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 5064.86 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202626.847362 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45583.73 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 4558.37 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207185.220110 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41025.35 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 4102.54 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211287.755583 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36922.82 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3692.28 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214980.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33230.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218580.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29630.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222180.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26030.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225780.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22430.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229380.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18830.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232980.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15230.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236580.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11630.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240180.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8030.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243780.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4430.54 sec) 08:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247380.037509 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 830.54 sec) 08:32:00 [GPUGRID] [rr_sim] 2248210.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988715544.31G/439.78G) 08:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851971.49 08:32:00 [---] pick_jobs_to_run() start 08:32:00 [---] pick_jobs_to_run() end 08:32:00 [---] [rr_sim] end 08:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:32:00 [---] [cpu_sched_debug] preliminary job list: 08:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:32:00 [---] [cpu_sched_debug] final job list: 08:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:32:00 [---] [cpu_sched_debug] enforce_run_list: end 08:32:00 [---] choose_project(): 30720.000000 08:32:00 [iThena.Measurements] doing RPC to get work 08:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404879.09 cpu 2150132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:32:00 [---] Got 0 tasks 08:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:33:00 [---] [rr_sim] doing sim: CPU sched 08:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988687953.95G/439.78G) 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248147.84 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 224814.78 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224814.783796 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023333.05 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 202333.31 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427148.089213 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820999.75 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 182099.97 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609248.064088 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638899.77 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 163889.98 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773138.041475 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1475009.80 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 147500.98 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920639.021124 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327508.82 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 132750.88 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053389.902807 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194757.94 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 119475.79 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172865.696323 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075282.14 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 107528.21 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280393.910487 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967753.93 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 96775.39 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377169.303234 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870978.53 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 87097.85 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464267.156707 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783880.68 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 78388.07 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542655.224832 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705492.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 70549.26 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613204.486145 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634943.35 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 63494.34 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676698.821327 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571449.02 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 57144.90 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733843.722990 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514304.11 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 51430.41 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785274.134488 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462873.70 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 46287.37 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831561.504835 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416586.33 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 41658.63 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873220.138148 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374927.70 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 37492.77 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910712.908129 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337434.93 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 33743.49 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944456.401112 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303691.44 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 30369.14 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974825.544797 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273322.29 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 27332.23 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002157.774114 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245990.06 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 24599.01 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026756.780498 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221391.06 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 22139.11 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048895.886245 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199251.95 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 19925.20 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068821.081416 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179326.76 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 17932.68 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086753.757071 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161394.08 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 16139.41 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102893.165160 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145254.67 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 14525.47 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117418.632440 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130729.21 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 13072.92 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130491.552992 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117656.28 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 11765.63 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142257.181489 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105890.66 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 10589.07 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152846.247137 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95301.59 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 9530.16 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162376.406219 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85771.43 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 8577.14 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170953.549393 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77194.29 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 7719.43 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178672.978250 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69474.86 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 6947.49 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185620.464221 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62527.37 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 6252.74 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191873.201595 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56274.64 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 5627.46 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197500.665232 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50647.17 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 5064.72 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202565.382505 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45582.46 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 4558.25 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207123.628051 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41024.21 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 4102.42 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211226.049042 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36921.79 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3692.18 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214918.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33229.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218518.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29629.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222118.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26029.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225718.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22429.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229318.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18829.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232918.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15229.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236518.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11629.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240118.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8029.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243718.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4429.61 sec) 08:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247318.227934 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 829.61 sec) 08:33:00 [GPUGRID] [rr_sim] 2248147.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988687953.95G/439.78G) 08:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851968.75 08:33:00 [---] pick_jobs_to_run() start 08:33:00 [---] pick_jobs_to_run() end 08:33:00 [---] [rr_sim] end 08:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:33:00 [---] [cpu_sched_debug] preliminary job list: 08:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:33:00 [---] [cpu_sched_debug] final job list: 08:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:33:00 [---] [cpu_sched_debug] enforce_run_list: end 08:33:00 [---] choose_project(): 30780.000000 08:33:00 [iThena.Measurements] doing RPC to get work 08:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404819.09 cpu 2150072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:33:00 [---] Got 0 tasks 08:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:34:00 [---] [rr_sim] doing sim: CPU sched 08:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988660363.60G/439.78G) 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248085.10 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 224808.51 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224808.510108 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023276.59 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 202327.66 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427136.169205 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820948.93 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 182094.89 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609231.062393 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638854.04 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 163885.40 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773116.466261 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474968.63 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 147496.86 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920613.329743 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327471.77 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 132747.18 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053360.506877 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194724.59 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 119472.46 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172832.966297 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075252.13 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 107525.21 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280358.179775 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967726.92 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 96772.69 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377130.871906 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870954.23 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 87095.42 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464226.294823 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783858.81 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 78385.88 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542612.175449 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705472.93 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 70547.29 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613159.468012 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634925.63 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 63492.56 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676652.031319 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571433.07 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 57143.31 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733795.338295 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514289.76 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 51428.98 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785224.314573 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462860.79 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 46286.08 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831510.393224 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416574.71 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 41657.47 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873167.864009 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374917.24 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 37491.72 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910659.587716 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337425.51 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 33742.55 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944402.139053 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303682.96 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 30368.30 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974770.435255 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273314.67 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 27331.47 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002101.901838 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245983.20 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 24598.32 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026700.221762 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221384.88 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 22138.49 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048838.709694 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199246.39 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 19924.64 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068763.348832 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179321.75 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 17932.18 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086695.524057 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161389.58 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 16138.96 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102834.481759 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145250.62 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 14525.06 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117359.543691 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130725.56 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 13072.56 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130432.099430 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117653.00 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 11765.30 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142197.399595 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105887.70 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 10588.77 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152786.169743 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95298.93 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 9529.89 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162316.062877 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85769.04 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 8576.90 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170892.966697 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77192.13 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 7719.21 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178612.180136 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69472.92 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 6947.29 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185559.472230 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62525.63 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 6252.56 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191812.035115 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56273.07 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 5627.31 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197439.341711 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50645.76 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 5064.58 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202503.917648 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45581.18 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 4558.12 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207062.035991 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41023.07 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 4102.31 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211164.342500 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36920.76 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3692.08 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214856.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33228.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218456.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29628.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222056.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26028.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225656.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22428.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229256.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18828.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232856.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15228.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236456.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11628.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240056.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8028.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243656.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4428.68 sec) 08:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247256.418358 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 828.68 sec) 08:34:00 [GPUGRID] [rr_sim] 2248085.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988660363.60G/439.78G) 08:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851966.01 08:34:00 [---] pick_jobs_to_run() start 08:34:00 [---] pick_jobs_to_run() end 08:34:00 [---] [rr_sim] end 08:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:34:00 [---] [cpu_sched_debug] preliminary job list: 08:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:34:00 [---] [cpu_sched_debug] final job list: 08:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:34:00 [---] [cpu_sched_debug] enforce_run_list: end 08:34:00 [---] choose_project(): 30840.000000 08:34:00 [iThena.Measurements] doing RPC to get work 08:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404759.09 cpu 2150012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2150012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2150012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:34:00 [---] Got 0 tasks 08:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:35:00 [---] [rr_sim] doing sim: CPU sched 08:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988632773.24G/439.78G) 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2248022.36 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 224802.24 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224802.236420 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023220.13 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 202322.01 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427124.249197 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820898.11 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 182089.81 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609214.060697 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638808.30 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 163880.83 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773094.891047 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474927.47 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 147492.75 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920587.638362 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327434.73 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 132743.47 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053331.110946 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194691.25 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 119469.13 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172800.236271 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075222.13 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 107522.21 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280322.449064 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967699.92 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 96769.99 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377092.440577 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870929.92 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 87092.99 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464185.432939 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783836.93 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 78383.69 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542569.126065 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705453.24 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 70545.32 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613114.449878 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634907.91 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 63490.79 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676605.241310 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571417.12 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 57141.71 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733746.953599 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514275.41 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 51427.54 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785174.494659 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462847.87 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 46284.79 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831459.281612 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416563.08 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 41656.31 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873115.589871 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374906.77 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 37490.68 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910606.267304 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337416.10 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 33741.61 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944347.876993 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303674.49 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 30367.45 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974715.325713 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273307.04 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 27330.70 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002046.029562 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245976.33 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 24597.63 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026643.663025 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221378.70 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 22137.87 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048781.533143 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199240.83 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 19924.08 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068705.616248 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179316.75 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 17931.67 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086637.291043 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161385.07 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 16138.51 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102775.798358 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145246.57 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 14524.66 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117300.454942 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130721.91 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 13072.19 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130372.645868 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117649.72 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 11764.97 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142137.617701 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105884.75 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 10588.47 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152726.092350 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95296.27 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 9529.63 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162255.719535 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85766.64 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 8576.66 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170832.384001 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77189.98 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 7719.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178551.382021 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69470.98 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 6947.10 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185498.480239 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62523.88 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 6252.39 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191750.868634 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56271.50 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 5627.15 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197378.018191 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50644.35 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 5064.43 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202442.452791 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45579.91 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 4557.99 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207000.443932 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41021.92 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 4102.19 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211102.635958 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36919.73 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3691.97 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214794.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33227.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218394.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29627.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221994.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26027.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225594.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22427.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229194.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18827.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232794.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15227.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236394.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11627.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239994.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8027.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243594.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4427.76 sec) 08:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247194.608782 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 827.76 sec) 08:35:00 [GPUGRID] [rr_sim] 2248022.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988632773.24G/439.78G) 08:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851963.28 08:35:00 [---] pick_jobs_to_run() start 08:35:00 [---] pick_jobs_to_run() end 08:35:00 [---] [rr_sim] end 08:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:35:00 [---] [cpu_sched_debug] preliminary job list: 08:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:35:00 [---] [cpu_sched_debug] final job list: 08:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:35:00 [---] [cpu_sched_debug] enforce_run_list: end 08:35:00 [---] choose_project(): 30900.000000 08:35:00 [iThena.Measurements] doing RPC to get work 08:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404699.09 cpu 2149952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:35:00 [---] Got 0 tasks 08:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:36:00 [---] [rr_sim] doing sim: CPU sched 08:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988605182.88G/439.78G) 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247959.63 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 224795.96 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224795.962732 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023163.66 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 202316.37 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427112.329190 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820847.30 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 182084.73 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609197.059002 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638762.57 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 163876.26 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773073.315834 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474886.31 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 147488.63 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920561.946982 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327397.68 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 132739.77 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053301.715015 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194657.91 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 119465.79 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172767.506245 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075192.12 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 107519.21 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280286.718352 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967672.91 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 96767.29 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377054.009248 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870905.62 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 87090.56 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464144.571055 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783815.06 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 78381.51 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542526.076681 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705433.55 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 70543.36 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613069.431745 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634890.20 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 63489.02 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676558.451302 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571401.18 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 57140.12 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733698.568903 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514261.06 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 51426.11 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785124.674744 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462834.95 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 46283.50 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831408.170001 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416551.46 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 41655.15 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873063.315733 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374896.31 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 37489.63 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910552.946891 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337406.68 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 33740.67 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944293.614933 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303666.01 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 30366.60 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974660.216171 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273299.41 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 27329.94 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001990.157286 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245969.47 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 24596.95 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026587.104289 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221372.52 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 22137.25 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048724.356591 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199235.27 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 19923.53 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068647.883664 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179311.74 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 17931.17 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086579.058029 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161380.57 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 16138.06 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102717.114958 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145242.51 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 14524.25 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117241.366193 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130718.26 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 13071.83 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130313.192305 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117646.44 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 11764.64 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142077.835806 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105881.79 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 10588.18 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152666.014957 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95293.61 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 9529.36 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162195.376193 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85764.25 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 8576.43 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170771.801305 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77187.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 7718.78 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178490.583906 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69469.04 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 6946.90 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185437.488247 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62522.14 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 6252.21 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191689.702154 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56269.93 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 5626.99 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197316.694670 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50642.93 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 5064.29 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202380.987935 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45578.64 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 4557.86 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206938.851873 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41020.78 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 4102.08 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211040.929417 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36918.70 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3691.87 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214732.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33226.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218332.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29626.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221932.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26026.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225532.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22426.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229132.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18826.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232732.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15226.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236332.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11626.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239932.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8026.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243532.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4426.83 sec) 08:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247132.799207 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 826.83 sec) 08:36:00 [GPUGRID] [rr_sim] 2247959.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988605182.88G/439.78G) 08:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851960.54 08:36:00 [---] pick_jobs_to_run() start 08:36:00 [---] pick_jobs_to_run() end 08:36:00 [---] [rr_sim] end 08:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:36:00 [---] [cpu_sched_debug] preliminary job list: 08:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:36:00 [---] [cpu_sched_debug] final job list: 08:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:36:00 [---] [cpu_sched_debug] enforce_run_list: end 08:36:00 [---] choose_project(): 30960.000000 08:36:00 [iThena.Measurements] doing RPC to get work 08:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404639.09 cpu 2149892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:36:00 [---] Got 0 tasks 08:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:37:00 [---] [rr_sim] doing sim: CPU sched 08:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988577592.53G/439.78G) 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247896.89 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 224789.69 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224789.689043 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023107.20 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 202310.72 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427100.409182 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820796.48 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 182079.65 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609180.057307 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638716.83 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 163871.68 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773051.740620 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474845.15 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 147484.51 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920536.255601 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327360.63 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 132736.06 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053272.319084 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194624.57 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 119462.46 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172734.776219 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075162.11 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 107516.21 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280250.987641 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967645.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 96764.59 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1377015.577920 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870881.31 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 87088.13 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464103.709171 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783793.18 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 78379.32 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542483.027297 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705413.86 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 70541.39 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1613024.413611 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634872.48 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 63487.25 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676511.661293 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571385.23 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 57138.52 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733650.184207 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514246.71 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 51424.67 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785074.854830 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462822.04 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 46282.20 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831357.058390 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416539.83 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 41653.98 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873011.041594 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374885.85 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 37488.58 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910499.626478 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337397.26 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 33739.73 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944239.352874 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303657.54 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 30365.75 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974605.106630 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273291.78 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 27329.18 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001934.285010 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245962.61 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 24596.26 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026530.545552 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221366.34 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 22136.63 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048667.180040 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199229.71 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 19922.97 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068590.151080 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179306.74 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 17930.67 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086520.825015 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161376.07 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 16137.61 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102658.431557 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145238.46 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 14523.85 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117182.277444 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130714.61 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 13071.46 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130253.738743 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117643.15 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 11764.32 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142018.053912 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105878.84 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 10587.88 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152605.937564 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95290.95 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 9529.10 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162135.032851 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85761.86 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 8576.19 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170711.218609 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77185.67 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 7718.57 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178429.785792 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69467.10 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 6946.71 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185376.496256 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62520.39 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 6252.04 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191628.535674 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56268.35 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 5626.84 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197255.371149 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50641.52 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 5064.15 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202319.523078 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45577.37 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 4557.74 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206877.259813 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41019.63 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 4101.96 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210979.222875 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36917.67 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3691.77 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214670.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33225.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218270.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29625.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221870.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26025.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225470.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22425.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229070.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18825.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232670.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15225.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236270.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11625.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239870.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8025.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243470.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4425.90 sec) 08:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247070.989631 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 825.90 sec) 08:37:00 [GPUGRID] [rr_sim] 2247896.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988577592.53G/439.78G) 08:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851957.80 08:37:00 [---] pick_jobs_to_run() start 08:37:00 [---] pick_jobs_to_run() end 08:37:00 [---] [rr_sim] end 08:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:37:00 [---] [cpu_sched_debug] preliminary job list: 08:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:37:00 [---] [cpu_sched_debug] final job list: 08:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:37:00 [---] [cpu_sched_debug] enforce_run_list: end 08:37:00 [---] choose_project(): 31020.000000 08:37:00 [iThena.Measurements] doing RPC to get work 08:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404579.09 cpu 2149832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:37:00 [---] Got 0 tasks 08:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:38:00 [---] [rr_sim] doing sim: CPU sched 08:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988550002.17G/439.78G) 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247834.15 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 224783.42 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224783.415355 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2023050.74 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 202305.07 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427088.489175 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820745.66 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 182074.57 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609163.055612 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638671.10 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 163867.11 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773030.165406 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474803.99 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 147480.40 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920510.564221 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327323.59 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 132732.36 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053242.923154 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194591.23 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 119459.12 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172702.046193 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075132.11 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 107513.21 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280215.256929 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967618.90 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 96761.89 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376977.146591 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870857.01 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 87085.70 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464062.847287 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783771.31 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 78377.13 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542439.977914 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705394.18 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 70539.42 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612979.395477 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634854.76 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 63485.48 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676464.871285 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571369.28 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 57136.93 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733601.799511 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514232.35 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 51423.24 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785025.034915 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462809.12 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 46280.91 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831305.946779 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416528.21 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 41652.82 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872958.767456 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374875.39 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 37487.54 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910446.306065 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337387.85 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 33738.78 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944185.090814 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303649.06 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 30364.91 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974549.997088 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273284.16 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 27328.42 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001878.412734 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245955.74 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 24595.57 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026473.986816 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221360.17 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 22136.02 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048610.003489 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199224.15 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 19922.42 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068532.418495 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179301.74 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 17930.17 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086462.592001 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161371.56 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 16137.16 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102599.748156 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145234.41 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 14523.44 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117123.188695 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130710.96 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 13071.10 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130194.285181 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117639.87 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 11763.99 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141958.272018 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105875.88 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 10587.59 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152545.860171 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95288.29 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 9528.83 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162074.689509 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85759.46 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 8575.95 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170650.635913 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77183.52 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 7718.35 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178368.987677 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69465.17 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 6946.52 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185315.504264 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62518.65 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 6251.86 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191567.369193 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56266.78 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 5626.68 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197194.047629 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50640.11 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 5064.01 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202258.058221 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45576.10 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 4557.61 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206815.667754 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41018.49 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 4101.85 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210917.516334 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36916.64 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3691.66 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214609.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33224.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218209.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29624.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221809.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26024.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225409.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22424.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229009.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18824.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232609.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15224.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236209.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11624.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239809.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8024.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243409.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4424.97 sec) 08:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2247009.180055 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 824.97 sec) 08:38:00 [GPUGRID] [rr_sim] 2247834.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988550002.17G/439.78G) 08:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851955.07 08:38:00 [---] pick_jobs_to_run() start 08:38:00 [---] pick_jobs_to_run() end 08:38:00 [---] [rr_sim] end 08:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:38:00 [---] [cpu_sched_debug] preliminary job list: 08:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:38:00 [---] [cpu_sched_debug] final job list: 08:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:38:00 [---] [cpu_sched_debug] enforce_run_list: end 08:38:00 [---] choose_project(): 31080.000000 08:38:00 [iThena.Measurements] doing RPC to get work 08:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404519.09 cpu 2149772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:38:00 [---] Got 0 tasks 08:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:39:00 [---] [rr_sim] doing sim: CPU sched 08:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988522411.82G/439.78G) 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247771.42 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 224777.14 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224777.141667 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022994.28 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 202299.43 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427076.569167 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820694.85 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 182069.48 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609146.053917 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638625.36 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 163862.54 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 773008.590192 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474762.83 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 147476.28 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920484.872840 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327286.54 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 132728.65 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053213.527223 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194557.89 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 119455.79 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172669.316167 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075102.10 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 107510.21 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280179.526218 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967591.89 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 96759.19 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376938.715263 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870832.70 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 87083.27 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1464021.985403 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783749.43 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 78374.94 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542396.928530 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705374.49 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 70537.45 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612934.377344 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634837.04 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 63483.70 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676418.081276 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571353.34 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 57135.33 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733553.414815 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514218.00 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 51421.80 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784975.215001 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462796.20 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 46279.62 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831254.835168 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416516.58 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 41651.66 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872906.493318 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374864.92 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 37486.49 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910392.985653 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337378.43 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 33737.84 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944130.828754 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303640.59 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 30364.06 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974494.887546 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273276.53 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 27327.65 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001822.540458 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245948.88 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 24594.89 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026417.428079 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221353.99 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 22135.40 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048552.826938 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199218.59 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 19921.86 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068474.685911 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179296.73 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 17929.67 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086404.358987 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161367.06 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 16136.71 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102541.064755 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145230.35 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 14523.04 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117064.099946 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130707.32 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 13070.73 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130134.831619 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117636.59 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 11763.66 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141898.490124 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105872.93 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 10587.29 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152485.782778 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95285.63 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 9528.56 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162014.346167 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85757.07 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 8575.71 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170590.053217 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77181.36 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 7718.14 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178308.189562 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69463.23 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 6946.32 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185254.512273 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62516.90 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 6251.69 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191506.202713 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56265.21 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 5626.52 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197132.724108 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50638.69 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 5063.87 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202196.593364 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45574.82 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 4557.48 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206754.075695 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41017.34 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 4101.73 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210855.809792 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36915.61 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3691.56 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214547.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33224.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218147.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29624.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221747.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26024.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225347.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22424.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228947.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18824.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232547.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15224.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236147.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11624.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239747.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8024.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243347.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4424.05 sec) 08:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246947.370480 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 824.05 sec) 08:39:00 [GPUGRID] [rr_sim] 2247771.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988522411.82G/439.78G) 08:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851952.33 08:39:00 [---] pick_jobs_to_run() start 08:39:00 [---] pick_jobs_to_run() end 08:39:00 [---] [rr_sim] end 08:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:39:00 [---] [cpu_sched_debug] preliminary job list: 08:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:39:00 [---] [cpu_sched_debug] final job list: 08:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:39:00 [---] [cpu_sched_debug] enforce_run_list: end 08:39:00 [---] choose_project(): 31140.000000 08:39:00 [iThena.Measurements] doing RPC to get work 08:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404459.09 cpu 2149712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:39:00 [---] Got 0 tasks 08:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:40:00 [---] [rr_sim] doing sim: CPU sched 08:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988494821.46G/439.78G) 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247708.68 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 224770.87 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224770.867979 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022937.81 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 202293.78 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427064.649159 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820644.03 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 182064.40 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609129.052222 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638579.63 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 163857.96 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772987.014979 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474721.66 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 147472.17 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920459.181459 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327249.50 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 132724.95 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053184.131292 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194524.55 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 119452.45 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172636.586142 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075072.09 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 107507.21 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280143.795506 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967564.88 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 96756.49 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376900.283934 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870808.40 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 87080.84 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463981.123519 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783727.56 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 78372.76 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542353.879146 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705354.80 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 70535.48 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612889.359210 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634819.32 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 63481.93 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676371.291268 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571337.39 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 57133.74 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733505.030120 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514203.65 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 51420.36 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784925.395086 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462783.28 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 46278.33 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831203.723556 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416504.96 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 41650.50 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872854.219179 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374854.46 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 37485.45 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910339.665240 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337369.01 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 33736.90 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944076.566695 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303632.11 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 30363.21 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974439.778004 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273268.90 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 27326.89 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001766.668182 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245942.01 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 24594.20 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026360.869343 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221347.81 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 22134.78 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048495.650387 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199213.03 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 19921.30 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068416.953327 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179291.73 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 17929.17 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086346.125973 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161362.55 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 16136.26 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102482.381354 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145226.30 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 14522.63 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117005.011197 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130703.67 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 13070.37 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130075.378056 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117633.30 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 11763.33 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141838.708229 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105869.97 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 10587.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152425.705385 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95282.97 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 9528.30 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161954.002825 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85754.68 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 8575.47 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170529.470521 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77179.21 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 7717.92 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178247.391448 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69461.29 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 6946.13 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185193.520282 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62515.16 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 6251.52 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191445.036232 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56263.64 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 5626.36 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197071.400588 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50637.28 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 5063.73 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202135.128508 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45573.55 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 4557.36 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206692.483635 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41016.20 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 4101.62 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210794.103251 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36914.58 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3691.46 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214485.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33223.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218085.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29623.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221685.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26023.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225285.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22423.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228885.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18823.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232485.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15223.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236085.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11623.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239685.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8023.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243285.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4423.12 sec) 08:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246885.560904 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 823.12 sec) 08:40:00 [GPUGRID] [rr_sim] 2247708.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988494821.46G/439.78G) 08:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851949.59 08:40:00 [---] pick_jobs_to_run() start 08:40:00 [---] pick_jobs_to_run() end 08:40:00 [---] [rr_sim] end 08:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:40:00 [---] [cpu_sched_debug] preliminary job list: 08:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:40:00 [---] [cpu_sched_debug] final job list: 08:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:40:00 [---] [cpu_sched_debug] enforce_run_list: end 08:40:00 [---] choose_project(): 31200.000000 08:40:00 [iThena.Measurements] doing RPC to get work 08:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404399.09 cpu 2149652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:40:00 [---] Got 0 tasks 08:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:41:00 [---] [rr_sim] doing sim: CPU sched 08:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988467231.10G/439.78G) 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247645.94 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 224764.59 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224764.594290 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022881.35 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 202288.13 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427052.729152 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820593.21 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 182059.32 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609112.050527 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638533.89 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 163853.39 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772965.439765 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474680.50 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 147468.05 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920433.490079 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327212.45 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 132721.25 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053154.735361 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194491.21 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 119449.12 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172603.856116 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075042.09 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 107504.21 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280108.064794 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967537.88 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 96753.79 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376861.852605 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870784.09 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 87078.41 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463940.261635 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783705.68 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 78370.57 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542310.829762 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705335.11 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 70533.51 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612844.341076 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634801.60 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 63480.16 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676324.501259 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571321.44 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 57132.14 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733456.645424 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514189.30 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 51418.93 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784875.575172 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462770.37 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 46277.04 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831152.611945 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416493.33 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 41649.33 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872801.945041 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374844.00 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 37484.40 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910286.344827 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337359.60 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 33735.96 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944022.304635 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303623.64 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 30362.36 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974384.668462 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273261.27 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 27326.13 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001710.795906 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245935.15 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 24593.51 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026304.310606 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221341.63 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 22134.16 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048438.473836 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199207.47 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 19920.75 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068359.220743 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179286.72 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 17928.67 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086287.892959 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161358.05 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 16135.80 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102423.697953 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145222.24 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 14522.22 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116945.922448 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130700.02 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 13070.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130015.924494 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117630.02 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 11763.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141778.926335 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105867.02 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 10586.70 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152365.627992 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95280.31 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 9528.03 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161893.659483 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85752.28 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 8575.23 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170468.887825 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77177.06 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 7717.71 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178186.593333 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69459.35 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 6945.93 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185132.528290 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62513.41 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 6251.34 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191383.869752 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56262.07 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 5626.21 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197010.077067 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50635.87 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 5063.59 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202073.663651 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45572.28 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 4557.23 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206630.891576 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41015.05 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 4101.51 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210732.396709 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36913.55 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3691.35 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214423.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33222.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218023.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29622.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221623.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26022.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225223.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22422.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228823.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18822.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232423.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15222.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236023.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11622.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239623.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8022.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243223.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4422.19 sec) 08:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246823.751328 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 822.19 sec) 08:41:00 [GPUGRID] [rr_sim] 2247645.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988467231.10G/439.78G) 08:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851946.86 08:41:00 [---] pick_jobs_to_run() start 08:41:00 [---] pick_jobs_to_run() end 08:41:00 [---] [rr_sim] end 08:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:41:00 [---] [cpu_sched_debug] preliminary job list: 08:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:41:00 [---] [cpu_sched_debug] final job list: 08:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:41:00 [---] [cpu_sched_debug] enforce_run_list: end 08:41:00 [---] choose_project(): 31260.000000 08:41:00 [iThena.Measurements] doing RPC to get work 08:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404339.09 cpu 2149592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:41:00 [---] Got 0 tasks 08:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:42:00 [---] [rr_sim] doing sim: CPU sched 08:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988439640.75G/439.78G) 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247583.21 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 224758.32 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224758.320602 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022824.89 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 202282.49 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427040.809144 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820542.40 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 182054.24 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609095.048832 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638488.16 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 163848.82 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772943.864551 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474639.34 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 147463.93 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920407.798698 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327175.41 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 132717.54 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053125.339431 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194457.87 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 119445.79 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172571.126090 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1075012.08 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 107501.21 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280072.334083 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967510.87 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 96751.09 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376823.421277 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870759.78 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 87075.98 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463899.399751 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783683.81 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 78368.38 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542267.780378 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705315.43 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 70531.54 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612799.322943 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634783.88 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 63478.39 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676277.711251 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571305.49 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 57130.55 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733408.260728 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514174.95 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 51417.49 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784825.755257 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462757.45 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 46275.75 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831101.500334 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416481.71 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 41648.17 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872749.670903 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374833.54 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 37483.35 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910233.024415 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337350.18 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 33735.02 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943968.042575 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303615.16 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 30361.52 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974329.558920 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273253.65 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 27325.36 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001654.923630 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245928.28 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 24592.83 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026247.751869 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221335.45 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 22133.55 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048381.297285 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199201.91 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 19920.19 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068301.488158 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179281.72 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 17928.17 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086229.659945 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161353.55 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 16135.35 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102365.014553 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145218.19 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 14521.82 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116886.833700 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130696.37 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 13069.64 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129956.470932 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117626.74 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 11762.67 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141719.144441 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105864.06 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 10586.41 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152305.550599 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95277.66 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 9527.77 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161833.316141 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85749.89 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 8574.99 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170408.305129 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77174.90 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 7717.49 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178125.795219 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69457.41 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 6945.74 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185071.536299 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62511.67 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 6251.17 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191322.703271 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56260.50 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 5626.05 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196948.753546 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50634.45 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 5063.45 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202012.198794 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45571.01 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 4557.10 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206569.299517 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41013.91 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 4101.39 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210670.690167 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36912.52 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3691.25 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214361.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33221.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217961.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29621.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221561.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26021.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225161.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22421.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228761.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18821.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232361.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15221.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235961.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11621.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239561.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8021.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243161.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4421.26 sec) 08:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246761.941753 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 821.26 sec) 08:42:00 [GPUGRID] [rr_sim] 2247583.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988439640.75G/439.78G) 08:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851944.12 08:42:00 [---] pick_jobs_to_run() start 08:42:00 [---] pick_jobs_to_run() end 08:42:00 [---] [rr_sim] end 08:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:42:00 [---] [cpu_sched_debug] preliminary job list: 08:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:42:00 [---] [cpu_sched_debug] final job list: 08:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:42:00 [---] [cpu_sched_debug] enforce_run_list: end 08:42:00 [---] choose_project(): 31320.000000 08:42:00 [iThena.Measurements] doing RPC to get work 08:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404279.09 cpu 2149532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:42:00 [---] Got 0 tasks 08:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:43:00 [---] [rr_sim] doing sim: CPU sched 08:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988412050.39G/439.78G) 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247520.47 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 224752.05 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224752.046914 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022768.42 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 202276.84 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427028.889137 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820491.58 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 182049.16 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609078.047137 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638442.42 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 163844.24 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772922.289337 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474598.18 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 147459.82 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920382.107318 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327138.36 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 132713.84 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053095.943500 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194424.53 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 119442.45 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172538.396064 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074982.07 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 107498.21 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280036.603371 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967483.87 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 96748.39 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376784.989948 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870735.48 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 87073.55 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463858.537867 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783661.93 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 78366.19 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542224.730995 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705295.74 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 70529.57 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612754.304809 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634766.16 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 63476.62 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676230.921242 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571289.55 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 57128.95 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733359.876032 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514160.59 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 51416.06 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784775.935343 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462744.53 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 46274.45 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831050.388723 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416470.08 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 41647.01 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872697.396764 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374823.07 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 37482.31 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910179.704002 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337340.77 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 33734.08 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943913.780516 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303606.69 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 30360.67 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974274.449378 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273246.02 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 27324.60 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001599.051354 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245921.42 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 24592.14 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026191.193133 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221329.28 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 22132.93 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048324.120734 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199196.35 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 19919.63 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068243.755574 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179276.71 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 17927.67 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086171.426931 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161349.04 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 16134.90 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102306.331152 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145214.14 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 14521.41 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116827.744951 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130692.72 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 13069.27 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129897.017369 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117623.45 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 11762.35 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141659.362547 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105861.11 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 10586.11 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152245.473206 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95275.00 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 9527.50 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161772.972799 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85747.50 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 8574.75 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170347.722433 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77172.75 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 7717.27 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178064.997104 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69455.47 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 6945.55 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185010.544308 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62509.92 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 6250.99 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191261.536791 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56258.93 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 5625.89 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196887.430026 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50633.04 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 5063.30 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201950.733937 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45569.74 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 4556.97 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206507.707457 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41012.76 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 4101.28 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210608.983626 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36911.49 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3691.15 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214300.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33220.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217900.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29620.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221500.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26020.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225100.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22420.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228700.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18820.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232300.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15220.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235900.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11620.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239500.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8020.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243100.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4420.34 sec) 08:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246700.132177 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 820.34 sec) 08:43:00 [GPUGRID] [rr_sim] 2247520.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988412050.39G/439.78G) 08:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851941.38 08:43:00 [---] pick_jobs_to_run() start 08:43:00 [---] pick_jobs_to_run() end 08:43:00 [---] [rr_sim] end 08:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:43:00 [---] [cpu_sched_debug] preliminary job list: 08:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:43:00 [---] [cpu_sched_debug] final job list: 08:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:43:00 [---] [cpu_sched_debug] enforce_run_list: end 08:43:00 [---] choose_project(): 31380.000000 08:43:00 [iThena.Measurements] doing RPC to get work 08:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 577019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404219.09 cpu 2149472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:43:00 [---] Got 0 tasks 08:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:44:00 [---] [rr_sim] doing sim: CPU sched 08:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988384460.04G/439.78G) 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247457.73 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 224745.77 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224745.773226 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022711.96 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 202271.20 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427016.969129 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820440.76 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 182044.08 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609061.045442 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638396.69 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 163839.67 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772900.714123 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474557.02 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 147455.70 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920356.415937 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327101.32 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 132710.13 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053066.547569 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194391.18 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 119439.12 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172505.666038 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074952.07 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 107495.21 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1280000.872660 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967456.86 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 96745.69 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376746.558620 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870711.17 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 87071.12 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463817.675983 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783640.06 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 78364.01 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542181.681611 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705276.05 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 70527.61 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612709.286676 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634748.45 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 63474.84 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676184.131234 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571273.60 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 57127.36 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733311.491336 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514146.24 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 51414.62 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784726.115428 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462731.62 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 46273.16 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830999.277111 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416458.46 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 41645.85 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872645.122626 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374812.61 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 37481.26 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910126.383589 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337331.35 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 33733.13 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943859.518456 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303598.21 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 30359.82 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974219.339836 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273238.39 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 27323.84 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001543.179078 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245914.55 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 24591.46 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026134.634396 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221323.10 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 22132.31 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048266.944182 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199190.79 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 19919.08 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068186.022990 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179271.71 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 17927.17 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086113.193917 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161344.54 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 16134.45 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102247.647751 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145210.08 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 14521.01 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116768.656202 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130689.08 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 13068.91 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129837.563807 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117620.17 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 11762.02 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141599.580652 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105858.15 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 10585.82 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152185.395813 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95272.34 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 9527.23 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161712.629457 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85745.10 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 8574.51 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170287.139737 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77170.59 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 7717.06 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178004.198989 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69453.53 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 6945.35 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184949.552316 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62508.18 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 6250.82 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191200.370310 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56257.36 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 5625.74 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196826.106505 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50631.63 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 5063.16 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201889.269080 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45568.46 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 4556.85 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206446.115398 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41011.62 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 4101.16 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210547.277084 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36910.46 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3691.05 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214238.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33219.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217838.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29619.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221438.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26019.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225038.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22419.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228638.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18819.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232238.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15219.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235838.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11619.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239438.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8019.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243038.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4419.41 sec) 08:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246638.322601 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 819.41 sec) 08:44:00 [GPUGRID] [rr_sim] 2247457.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988384460.04G/439.78G) 08:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851938.64 08:44:00 [---] pick_jobs_to_run() start 08:44:00 [---] pick_jobs_to_run() end 08:44:00 [---] [rr_sim] end 08:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:44:00 [---] [cpu_sched_debug] preliminary job list: 08:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:44:00 [---] [cpu_sched_debug] final job list: 08:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:44:00 [---] [cpu_sched_debug] enforce_run_list: end 08:44:00 [---] choose_project(): 31440.000000 08:44:00 [iThena.Measurements] doing RPC to get work 08:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404159.09 cpu 2149412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:44:00 [---] Got 0 tasks 08:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:45:00 [---] [rr_sim] doing sim: CPU sched 08:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988356869.68G/439.78G) 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247395.00 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 224739.50 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224739.499538 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022655.50 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 202265.55 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 427005.049121 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820389.95 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 182038.99 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609044.043747 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638350.95 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 163835.10 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772879.138910 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474515.86 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 147451.59 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920330.724556 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327064.27 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 132706.43 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053037.151638 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194357.84 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 119435.78 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172472.936012 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074922.06 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 107492.21 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279965.141948 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967429.85 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 96742.99 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376708.127291 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870686.87 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 87068.69 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463776.814100 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783618.18 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 78361.82 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542138.632227 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705256.36 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 70525.64 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612664.268542 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634730.73 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 63473.07 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676137.341225 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571257.65 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 57125.77 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733263.106640 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514131.89 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 51413.19 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784676.295514 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462718.70 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 46271.87 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830948.165500 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416446.83 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 41644.68 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872592.848488 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374802.15 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 37480.21 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910073.063176 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337321.93 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 33732.19 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943805.256396 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303589.74 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 30358.97 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974164.230294 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273230.77 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 27323.08 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001487.306802 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245907.69 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 24590.77 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026078.075660 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221316.92 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 22131.69 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048209.767631 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199185.23 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 19918.52 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068128.290406 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179266.70 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 17926.67 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086054.960903 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161340.03 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 16134.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102188.964350 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145206.03 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 14520.60 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116709.567453 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130685.43 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 13068.54 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129778.110245 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117616.89 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 11761.69 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141539.798758 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105855.20 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 10585.52 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152125.318420 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95269.68 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 9526.97 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161652.286115 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85742.71 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 8574.27 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170226.557041 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77168.44 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 7716.84 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177943.400875 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69451.59 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 6945.16 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184888.560325 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62506.44 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 6250.64 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191139.203830 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56255.79 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 5625.58 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196764.782985 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50630.21 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 5063.02 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201827.804224 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45567.19 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 4556.72 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206384.523339 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41010.47 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 4101.05 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210485.570543 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36909.42 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3690.94 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214176.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33218.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217776.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29618.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221376.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26018.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224976.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22418.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228576.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18818.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232176.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15218.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235776.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11618.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239376.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8018.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242976.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4418.48 sec) 08:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246576.513026 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 818.48 sec) 08:45:00 [GPUGRID] [rr_sim] 2247395.00: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988356869.68G/439.78G) 08:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851935.91 08:45:00 [---] pick_jobs_to_run() start 08:45:00 [---] pick_jobs_to_run() end 08:45:00 [---] [rr_sim] end 08:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:45:00 [---] [cpu_sched_debug] preliminary job list: 08:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:45:00 [---] [cpu_sched_debug] final job list: 08:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:45:00 [---] [cpu_sched_debug] enforce_run_list: end 08:45:00 [---] choose_project(): 31500.000000 08:45:00 [iThena.Measurements] doing RPC to get work 08:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404099.09 cpu 2149352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:45:00 [---] Got 0 tasks 08:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:46:00 [---] [rr_sim] doing sim: CPU sched 08:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988329279.32G/439.78G) 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247332.26 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 224733.23 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224733.225849 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022599.03 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 202259.90 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426993.129114 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820339.13 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 182033.91 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609027.042052 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638305.22 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 163830.52 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772857.563696 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474474.69 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 147447.47 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920305.033176 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1327027.23 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 132702.72 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1053007.755707 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194324.50 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 119432.45 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172440.205986 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074892.05 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 107489.21 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279929.411237 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967402.85 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 96740.28 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376669.695962 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870662.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 87066.26 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463735.952216 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783596.31 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 78359.63 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542095.582843 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705236.68 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 70523.67 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612619.250408 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634713.01 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 63471.30 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676090.551217 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571241.71 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 57124.17 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733214.721945 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514117.54 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 51411.75 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784626.475599 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462705.78 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 46270.58 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830897.053889 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416435.20 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 41643.52 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872540.574349 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374791.68 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 37479.17 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910019.742764 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337312.52 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 33731.25 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943750.994337 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303581.26 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 30358.13 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974109.120752 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273223.14 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 27322.31 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001431.434526 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245900.82 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 24590.08 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026021.516923 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221310.74 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 22131.07 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048152.591080 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199179.67 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 19917.97 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068070.557822 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179261.70 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 17926.17 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085996.727889 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161335.53 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 16133.55 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102130.280949 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145201.98 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 14520.20 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116650.478704 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130681.78 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 13068.18 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129718.656683 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117613.60 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 11761.36 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141480.016864 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105852.24 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 10585.22 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152065.241027 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95267.02 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 9526.70 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161591.942773 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85740.32 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 8574.03 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170165.974345 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77166.28 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 7716.63 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177882.602760 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69449.66 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 6944.97 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184827.568334 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62504.69 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 6250.47 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191078.037350 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56254.22 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 5625.42 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196703.459464 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50628.80 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 5062.88 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201766.339367 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45565.92 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 4556.59 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206322.931280 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41009.33 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 4100.93 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210423.864001 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36908.39 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3690.84 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214114.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33217.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217714.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29617.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221314.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26017.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224914.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22417.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228514.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18817.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232114.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15217.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235714.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11617.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239314.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8017.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242914.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4417.56 sec) 08:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246514.703450 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 817.56 sec) 08:46:00 [GPUGRID] [rr_sim] 2247332.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988329279.32G/439.78G) 08:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851933.17 08:46:00 [---] pick_jobs_to_run() start 08:46:00 [---] pick_jobs_to_run() end 08:46:00 [---] [rr_sim] end 08:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:46:00 [---] [cpu_sched_debug] preliminary job list: 08:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:46:00 [---] [cpu_sched_debug] final job list: 08:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:46:00 [---] [cpu_sched_debug] enforce_run_list: end 08:46:00 [---] choose_project(): 31560.000000 08:46:00 [iThena.Measurements] doing RPC to get work 08:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 404039.09 cpu 2149292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:46:00 [---] Got 0 tasks 08:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:47:00 [---] [rr_sim] doing sim: CPU sched 08:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988301688.97G/439.78G) 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247269.52 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 224726.95 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224726.952161 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022542.57 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 202254.26 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426981.209106 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820288.31 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 182028.83 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 609010.040357 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638259.48 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 163825.95 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772835.988482 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474433.53 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 147443.35 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920279.341795 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326990.18 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 132699.02 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052978.359777 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194291.16 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 119429.12 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172407.475960 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074862.05 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 107486.20 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279893.680525 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967375.84 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 96737.58 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376631.264634 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870638.26 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 87063.83 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463695.090332 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783574.43 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 78357.44 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542052.533460 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705216.99 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 70521.70 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612574.232275 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634695.29 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 63469.53 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1676043.761208 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571225.76 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 57122.58 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733166.337249 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514103.18 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 51410.32 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784576.655685 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462692.87 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 46269.29 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830845.942278 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416423.58 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 41642.36 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872488.300211 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374781.22 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 37478.12 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909966.422351 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337303.10 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 33730.31 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943696.732277 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303572.79 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 30357.28 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974054.011210 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273215.51 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 27321.55 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001375.562251 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245893.96 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 24589.40 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025964.958187 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221304.56 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 22130.46 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048095.414529 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199174.11 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 19917.41 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068012.825237 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179256.70 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 17925.67 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085938.494875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161331.03 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 16133.10 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102071.597548 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145197.92 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 14519.79 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116591.389955 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130678.13 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 13067.81 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129659.203120 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117610.32 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 11761.03 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141420.234969 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105849.29 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 10584.93 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152005.163634 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95264.36 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 9526.44 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161531.599431 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85737.92 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 8573.79 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170105.391649 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77164.13 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 7716.41 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177821.804646 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69447.72 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 6944.77 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184766.576342 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62502.95 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 6250.29 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191016.870869 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56252.65 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 5625.27 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196642.135943 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50627.39 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 5062.74 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201704.874510 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45564.65 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 4556.46 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206261.339220 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41008.18 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 4100.82 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210362.157459 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36907.36 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3690.74 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214052.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33216.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217652.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29616.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221252.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26016.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224852.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22416.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228452.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18816.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232052.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15216.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235652.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11616.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239252.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8016.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242852.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4416.63 sec) 08:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246452.893875 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 816.63 sec) 08:47:00 [GPUGRID] [rr_sim] 2247269.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988301688.97G/439.78G) 08:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851930.43 08:47:00 [---] pick_jobs_to_run() start 08:47:00 [---] pick_jobs_to_run() end 08:47:00 [---] [rr_sim] end 08:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:47:00 [---] [cpu_sched_debug] preliminary job list: 08:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:47:00 [---] [cpu_sched_debug] final job list: 08:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:47:00 [---] [cpu_sched_debug] enforce_run_list: end 08:47:00 [---] choose_project(): 31620.000000 08:47:00 [iThena.Measurements] doing RPC to get work 08:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403979.09 cpu 2149232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:47:00 [---] Got 0 tasks 08:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:48:00 [---] [rr_sim] doing sim: CPU sched 08:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988274098.61G/439.78G) 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247206.78 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 224720.68 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224720.678473 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022486.11 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 202248.61 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426969.289099 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820237.50 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 182023.75 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608993.038662 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638213.75 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 163821.37 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772814.413268 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474392.37 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 147439.24 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920253.650414 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326953.13 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 132695.31 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052948.963846 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194257.82 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 119425.78 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172374.745934 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074832.04 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 107483.20 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279857.949814 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967348.83 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 96734.88 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376592.833305 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870613.95 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 87061.40 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463654.228448 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783552.56 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 78355.26 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1542009.484076 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705197.30 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 70519.73 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612529.214141 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634677.57 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 63467.76 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675996.971200 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571209.81 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 57120.98 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733117.952553 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514088.83 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 51408.88 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784526.835770 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462679.95 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 46267.99 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830794.830666 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416411.95 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 41641.20 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872436.026073 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374770.76 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 37477.08 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909913.101938 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337293.68 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 33729.37 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943642.470217 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303564.31 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 30356.43 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973998.901669 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273207.88 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 27320.79 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001319.689975 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245887.09 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 24588.71 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025908.399450 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221298.39 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 22129.84 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048038.237978 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199168.55 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 19916.85 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067955.092653 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179251.69 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 17925.17 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085880.261861 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161326.52 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 16132.65 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102012.914148 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145193.87 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 14519.39 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116532.301206 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130674.48 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 13067.45 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129599.749558 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117607.04 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 11760.70 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141360.453075 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105846.33 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 10584.63 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151945.086241 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95261.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 9526.17 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161471.256089 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85735.53 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 8573.55 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170044.808953 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77161.98 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 7716.20 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177761.006531 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69445.78 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 6944.58 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184705.584351 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62501.20 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 6250.12 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190955.704389 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56251.08 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 5625.11 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196580.812423 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50625.97 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 5062.60 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201643.409653 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45563.38 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 4556.34 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206199.747161 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41007.04 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 4100.70 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210300.450918 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36906.33 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3690.63 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213991.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33215.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217591.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29615.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221191.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26015.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224791.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22415.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228391.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18815.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231991.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15215.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235591.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11615.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239191.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8015.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242791.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4415.70 sec) 08:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246391.084299 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 815.70 sec) 08:48:00 [GPUGRID] [rr_sim] 2247206.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988274098.61G/439.78G) 08:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851927.70 08:48:00 [---] pick_jobs_to_run() start 08:48:00 [---] pick_jobs_to_run() end 08:48:00 [---] [rr_sim] end 08:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:48:00 [---] [cpu_sched_debug] preliminary job list: 08:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:48:00 [---] [cpu_sched_debug] final job list: 08:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:48:00 [---] [cpu_sched_debug] enforce_run_list: end 08:48:00 [---] choose_project(): 31680.000000 08:48:00 [iThena.Measurements] doing RPC to get work 08:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403919.09 cpu 2149172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:48:00 [---] Got 0 tasks 08:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:49:00 [---] [rr_sim] doing sim: CPU sched 08:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988246508.25G/439.78G) 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247144.05 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 224714.40 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224714.404785 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022429.64 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 202242.96 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426957.369091 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820186.68 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 182018.67 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608976.036967 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638168.01 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 163816.80 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772792.838055 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474351.21 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 147435.12 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920227.959034 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326916.09 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 132691.61 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052919.567915 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194224.48 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 119422.45 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172342.015908 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074802.03 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 107480.20 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279822.219102 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967321.83 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 96732.18 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376554.401977 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870589.65 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 87058.96 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463613.366564 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783530.68 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 78353.07 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541966.434692 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705177.61 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 70517.76 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612484.196008 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634659.85 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 63465.99 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675950.181191 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571193.87 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 57119.39 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733069.567857 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514074.48 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 51407.45 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784477.015856 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462667.03 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 46266.70 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830743.719055 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416400.33 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 41640.03 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872383.751934 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374760.30 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 37476.03 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909859.781526 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337284.27 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 33728.43 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943588.208158 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303555.84 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 30355.58 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973943.792127 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273200.26 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 27320.03 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001263.817699 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245880.23 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 24588.02 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025851.840714 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221292.21 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 22129.22 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047981.061427 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199162.99 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 19916.30 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067897.360069 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179246.69 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 17924.67 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085822.028847 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161322.02 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 16132.20 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101954.230747 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145189.82 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 14518.98 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116473.212457 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130670.84 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 13067.08 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129540.295996 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117603.75 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 11760.38 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141300.671181 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105843.38 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 10584.34 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151885.008848 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95259.04 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 9525.90 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161410.912747 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85733.14 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 8573.31 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169984.226257 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77159.82 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 7715.98 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177700.208416 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69443.84 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 6944.38 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184644.592359 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62499.46 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 6249.95 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190894.537908 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56249.51 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 5624.95 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196519.488902 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50624.56 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 5062.46 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201581.944797 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45562.10 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 4556.21 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206138.155102 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41005.89 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 4100.59 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210238.744376 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36905.30 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3690.53 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213929.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33214.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217529.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29614.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221129.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26014.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224729.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22414.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228329.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18814.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231929.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15214.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235529.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11614.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239129.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8014.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242729.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4414.77 sec) 08:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246329.274723 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 814.77 sec) 08:49:00 [GPUGRID] [rr_sim] 2247144.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988246508.25G/439.78G) 08:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851924.96 08:49:00 [---] pick_jobs_to_run() start 08:49:00 [---] pick_jobs_to_run() end 08:49:00 [---] [rr_sim] end 08:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:49:00 [---] [cpu_sched_debug] preliminary job list: 08:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:49:00 [---] [cpu_sched_debug] final job list: 08:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:49:00 [---] [cpu_sched_debug] enforce_run_list: end 08:49:00 [---] choose_project(): 31740.000000 08:49:00 [iThena.Measurements] doing RPC to get work 08:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403859.09 cpu 2149112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:49:00 [---] Got 0 tasks 08:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:50:00 [---] [rr_sim] doing sim: CPU sched 08:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988218917.90G/439.78G) 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247081.31 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 224708.13 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224708.131096 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022373.18 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 202237.32 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426945.449083 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820135.86 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 182013.59 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608959.035271 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638122.28 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 163812.23 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772771.262841 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474310.05 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 147431.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920202.267653 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326879.04 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 132687.90 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052890.171984 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194191.14 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 119419.11 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172309.285882 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074772.03 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 107477.20 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279786.488391 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967294.82 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 96729.48 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376515.970648 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870565.34 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 87056.53 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463572.504680 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783508.81 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 78350.88 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541923.385308 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705157.93 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 70515.79 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612439.177874 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634642.13 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 63464.21 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675903.391183 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571177.92 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 57117.79 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1733021.183161 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514060.13 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 51406.01 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784427.195942 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462654.12 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 46265.41 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830692.607444 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416388.70 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 41638.87 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872331.477796 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374749.83 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 37474.98 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909806.461113 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337274.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 33727.48 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943533.946098 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303547.36 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 30354.74 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973888.682585 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273192.63 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 27319.26 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001207.945423 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245873.37 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 24587.34 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025795.281977 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221286.03 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 22128.60 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047923.884876 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199157.43 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 19915.74 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067839.627485 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179241.68 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 17924.17 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085763.795833 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161317.52 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 16131.75 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101895.547346 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145185.76 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 14518.58 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116414.123708 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130667.19 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 13066.72 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129480.842433 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117600.47 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 11760.05 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141240.889287 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105840.42 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 10584.04 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151824.931454 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95256.38 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 9525.64 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161350.569405 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85730.74 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 8573.07 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169923.643561 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77157.67 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 7715.77 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177639.410302 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69441.90 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 6944.19 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184583.600368 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62497.71 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 6249.77 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190833.371428 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56247.94 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 5624.79 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196458.165381 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50623.15 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 5062.31 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201520.479940 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45560.83 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 4556.08 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206076.563042 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41004.75 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 4100.47 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210177.037835 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36904.27 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3690.43 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213867.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33213.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217467.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29613.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221067.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26013.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224667.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22413.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228267.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18813.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231867.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15213.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235467.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11613.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239067.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8013.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242667.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4413.85 sec) 08:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246267.465148 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 813.85 sec) 08:50:00 [GPUGRID] [rr_sim] 2247081.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988218917.90G/439.78G) 08:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851922.22 08:50:00 [---] pick_jobs_to_run() start 08:50:00 [---] pick_jobs_to_run() end 08:50:00 [---] [rr_sim] end 08:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:50:00 [---] [cpu_sched_debug] preliminary job list: 08:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:50:00 [---] [cpu_sched_debug] final job list: 08:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:50:00 [---] [cpu_sched_debug] enforce_run_list: end 08:50:00 [---] choose_project(): 31800.000000 08:50:00 [iThena.Measurements] doing RPC to get work 08:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403799.09 cpu 2149052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2149052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2149052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:50:00 [---] Got 0 tasks 08:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:51:00 [---] [rr_sim] doing sim: CPU sched 08:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988191327.54G/439.78G) 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2247018.57 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 224701.86 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224701.857408 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022316.72 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 202231.67 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426933.529076 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820085.05 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 182008.50 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608942.033576 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638076.54 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 163807.65 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772749.687627 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474268.89 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 147426.89 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920176.576273 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326842.00 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 132684.20 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052860.776054 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194157.80 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 119415.78 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172276.555857 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074742.02 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 107474.20 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279750.757679 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967267.82 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 96726.78 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376477.539319 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870541.03 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 87054.10 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463531.642796 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783486.93 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 78348.69 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541880.335924 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705138.24 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 70513.82 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612394.159740 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634624.41 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 63462.44 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675856.601175 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571161.97 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 57116.20 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732972.798465 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514045.78 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 51404.58 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784377.376027 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462641.20 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 46264.12 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830641.495833 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416377.08 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 41637.71 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872279.203658 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374739.37 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 37473.94 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909753.140700 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337265.43 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 33726.54 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943479.684038 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303538.89 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 30353.89 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973833.573043 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273185.00 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 27318.50 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001152.073147 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245866.50 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 24586.65 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025738.723240 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221279.85 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 22127.99 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047866.708325 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199151.87 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 19915.19 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067781.894900 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179236.68 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 17923.67 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085705.562819 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161313.01 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 16131.30 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101836.863945 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145181.71 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 14518.17 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116355.034959 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130663.54 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 13066.35 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129421.388871 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117597.19 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 11759.72 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141181.107392 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105837.47 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 10583.75 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151764.854061 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95253.72 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 9525.37 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161290.226064 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85728.35 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 8572.83 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169863.060865 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77155.51 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 7715.55 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177578.612187 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69439.96 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 6944.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184522.608377 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62495.97 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 6249.60 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190772.204947 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56246.37 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 5624.64 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196396.841861 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50621.73 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 5062.17 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201459.015083 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45559.56 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 4555.96 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206014.970983 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41003.60 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 4100.36 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210115.331293 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36903.24 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3690.32 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213805.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33212.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217405.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29612.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221005.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26012.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224605.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22412.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228205.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18812.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231805.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15212.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235405.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11612.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239005.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8012.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242605.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4412.92 sec) 08:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246205.655572 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 812.92 sec) 08:51:00 [GPUGRID] [rr_sim] 2247018.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988191327.54G/439.78G) 08:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851919.49 08:51:00 [---] pick_jobs_to_run() start 08:51:00 [---] pick_jobs_to_run() end 08:51:00 [---] [rr_sim] end 08:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:51:00 [---] [cpu_sched_debug] preliminary job list: 08:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:51:00 [---] [cpu_sched_debug] final job list: 08:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:51:00 [---] [cpu_sched_debug] enforce_run_list: end 08:51:00 [---] choose_project(): 31860.000000 08:51:00 [iThena.Measurements] doing RPC to get work 08:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403739.09 cpu 2148992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:51:00 [---] Got 0 tasks 08:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:52:00 [---] [rr_sim] doing sim: CPU sched 08:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988163737.19G/439.78G) 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246955.84 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 224695.58 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224695.583720 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022260.25 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 202226.03 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426921.609068 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1820034.23 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 182003.42 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608925.031881 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1638030.81 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 163803.08 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772728.112413 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474227.72 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 147422.77 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920150.884892 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326804.95 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 132680.50 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052831.380123 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194124.46 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 119412.45 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172243.825831 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074712.01 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 107471.20 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279715.026968 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967240.81 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 96724.08 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376439.107991 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870516.73 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 87051.67 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463490.780912 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783465.06 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 78346.51 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541837.286541 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705118.55 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 70511.86 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612349.141607 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634606.70 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 63460.67 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675809.811166 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571146.03 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 57114.60 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732924.413770 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514031.42 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 51403.14 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784327.556113 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462628.28 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 46262.83 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830590.384221 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416365.45 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 41636.55 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872226.929519 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374728.91 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 37472.89 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909699.820287 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337256.02 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 33725.60 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943425.421979 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303530.42 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 30353.04 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973778.463501 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273177.37 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 27317.74 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001096.200871 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245859.64 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 24585.96 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025682.164504 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221273.67 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 22127.37 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047809.531773 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199146.31 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 19914.63 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067724.162316 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179231.67 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 17923.17 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085647.329805 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161308.51 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 16130.85 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101778.180544 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145177.66 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 14517.77 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116295.946210 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130659.89 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 13065.99 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129361.935309 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117593.90 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 11759.39 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141121.325498 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105834.51 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 10583.45 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151704.776668 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95251.06 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 9525.11 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161229.882722 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85725.95 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 8572.60 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169802.478169 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77153.36 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 7715.34 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177517.814073 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69438.02 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 6943.80 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184461.616385 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62494.22 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 6249.42 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190711.038467 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56244.80 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 5624.48 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196335.518340 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50620.32 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 5062.03 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201397.550226 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45558.29 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 4555.83 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205953.378924 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41002.46 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 4100.25 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210053.624751 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36902.21 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3690.22 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213743.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33211.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217343.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29611.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220943.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26011.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224543.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22411.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228143.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18811.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231743.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15211.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235343.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11611.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238943.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8011.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242543.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4411.99 sec) 08:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246143.845996 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 811.99 sec) 08:52:00 [GPUGRID] [rr_sim] 2246955.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988163737.19G/439.78G) 08:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851916.75 08:52:00 [---] pick_jobs_to_run() start 08:52:00 [---] pick_jobs_to_run() end 08:52:00 [---] [rr_sim] end 08:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:52:00 [---] [cpu_sched_debug] preliminary job list: 08:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:52:00 [---] [cpu_sched_debug] final job list: 08:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:52:00 [---] [cpu_sched_debug] enforce_run_list: end 08:52:00 [---] choose_project(): 31920.000000 08:52:00 [iThena.Measurements] doing RPC to get work 08:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403679.09 cpu 2148932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:52:00 [---] Got 0 tasks 08:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:53:00 [---] [rr_sim] doing sim: CPU sched 08:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988136146.83G/439.78G) 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246893.10 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 224689.31 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224689.310032 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022203.79 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 202220.38 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426909.689060 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819983.41 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 181998.34 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608908.030186 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637985.07 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 163798.51 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772706.537199 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474186.56 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 147418.66 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920125.193511 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326767.91 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 132676.79 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052801.984192 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194091.12 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 119409.11 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172211.095805 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074682.00 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 107468.20 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279679.296256 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967213.80 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 96721.38 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376400.676662 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870492.42 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 87049.24 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463449.919028 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783443.18 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 78344.32 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541794.237157 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705098.86 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 70509.89 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612304.123473 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634588.98 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 63458.90 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675763.021158 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571130.08 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 57113.01 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732876.029074 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514017.07 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 51401.71 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784277.736198 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462615.36 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 46261.54 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830539.272610 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416353.83 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 41635.38 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872174.655381 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374718.44 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 37471.84 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909646.499875 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337246.60 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 33724.66 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943371.159919 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303521.94 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 30352.19 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973723.353959 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273169.75 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 27316.97 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001040.328595 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245852.77 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 24585.28 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025625.605767 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221267.49 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 22126.75 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047752.355222 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199140.75 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 19914.07 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067666.429732 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179226.67 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 17922.67 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085589.096791 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161304.00 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 16130.40 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101719.497143 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145173.60 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 14517.36 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116236.857461 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130656.24 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 13065.62 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129302.481747 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117590.62 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 11759.06 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141061.543604 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105831.56 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 10583.16 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151644.699275 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95248.40 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 9524.84 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161169.539380 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85723.56 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 8572.36 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169741.895473 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77151.20 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 7715.12 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177457.015958 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69436.08 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 6943.61 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184400.624394 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62492.48 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 6249.25 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190649.871986 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56243.23 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 5624.32 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196274.194820 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50618.91 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 5061.89 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201336.085369 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45557.01 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 4555.70 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205891.786864 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41001.31 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 4100.13 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209991.918210 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36901.18 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3690.12 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213682.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33211.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217282.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29611.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220882.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26011.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224482.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22411.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228082.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18811.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231682.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15211.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235282.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11611.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238882.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8011.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242482.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4411.06 sec) 08:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246082.036421 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 811.06 sec) 08:53:00 [GPUGRID] [rr_sim] 2246893.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988136146.83G/439.78G) 08:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851914.01 08:53:00 [---] pick_jobs_to_run() start 08:53:00 [---] pick_jobs_to_run() end 08:53:00 [---] [rr_sim] end 08:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:53:00 [---] [cpu_sched_debug] preliminary job list: 08:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:53:00 [---] [cpu_sched_debug] final job list: 08:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:53:00 [---] [cpu_sched_debug] enforce_run_list: end 08:53:00 [---] choose_project(): 31980.000000 08:53:00 [iThena.Measurements] doing RPC to get work 08:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403619.09 cpu 2148872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:53:00 [---] Got 0 tasks 08:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:54:00 [---] [rr_sim] doing sim: CPU sched 08:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988136146.83G/439.78G) 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246893.10 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 224689.31 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224689.310032 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022203.79 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 202220.38 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426909.689060 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819983.41 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 181998.34 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608908.030186 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637985.07 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 163798.51 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772706.537199 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474186.56 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 147418.66 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920125.193511 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326767.91 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 132676.79 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052801.984192 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194091.12 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 119409.11 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172211.095805 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074682.00 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 107468.20 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279679.296256 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967213.80 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 96721.38 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376400.676662 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870492.42 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 87049.24 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463449.919028 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783443.18 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 78344.32 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541794.237157 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705098.86 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 70509.89 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612304.123473 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634588.98 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 63458.90 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675763.021158 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571130.08 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 57113.01 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732876.029074 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514017.07 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 51401.71 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784277.736198 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462615.36 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 46261.54 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830539.272610 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416353.83 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 41635.38 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872174.655381 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374718.44 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 37471.84 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909646.499875 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337246.60 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 33724.66 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943371.159919 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303521.94 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 30352.19 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973723.353959 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273169.75 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 27316.97 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001040.328595 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245852.77 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 24585.28 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025625.605767 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221267.49 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 22126.75 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047752.355222 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199140.75 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 19914.07 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067666.429732 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179226.67 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 17922.67 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085589.096791 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161304.00 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 16130.40 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101719.497143 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145173.60 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 14517.36 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116236.857461 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130656.24 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 13065.62 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129302.481747 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117590.62 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 11759.06 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141061.543604 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105831.56 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 10583.16 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151644.699275 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95248.40 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 9524.84 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161169.539380 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85723.56 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 8572.36 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169741.895473 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77151.20 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 7715.12 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177457.015958 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69436.08 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 6943.61 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184400.624394 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62492.48 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 6249.25 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190649.871986 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56243.23 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 5624.32 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196274.194820 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50618.91 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 5061.89 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201336.085369 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45557.01 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 4555.70 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205891.786864 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41001.31 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 4100.13 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209991.918210 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36901.18 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3690.12 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213682.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33211.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217282.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29611.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220882.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26011.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224482.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22411.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228082.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18811.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231682.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15211.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235282.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11611.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238882.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8011.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242482.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4411.06 sec) 08:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246082.036421 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 811.06 sec) 08:54:00 [GPUGRID] [rr_sim] 2246893.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988136146.83G/439.78G) 08:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851974.01 08:54:00 [---] pick_jobs_to_run() start 08:54:00 [---] pick_jobs_to_run() end 08:54:00 [---] [rr_sim] end 08:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:54:00 [---] [cpu_sched_debug] preliminary job list: 08:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:54:00 [---] [cpu_sched_debug] final job list: 08:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:54:00 [---] [cpu_sched_debug] enforce_run_list: end 08:54:00 [---] choose_project(): 32040.000000 08:54:00 [iThena.Measurements] doing RPC to get work 08:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403559.09 cpu 2148872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:54:00 [---] Got 0 tasks 08:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:55:00 [---] [rr_sim] doing sim: CPU sched 08:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988108556.47G/439.78G) 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246830.36 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 224683.04 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224683.036344 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022147.33 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 202214.73 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426897.769053 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819932.59 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 181993.26 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608891.028491 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637939.33 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 163793.93 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772684.961986 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474145.40 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 147414.54 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920099.502131 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326730.86 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 132673.09 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052772.588261 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194057.78 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 119405.78 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172178.365779 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074652.00 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 107465.20 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279643.565545 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967186.80 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 96718.68 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376362.245334 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870468.12 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 87046.81 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463409.057144 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783421.31 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 78342.13 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541751.187773 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705079.18 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 70507.92 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612259.105339 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634571.26 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 63457.13 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675716.231149 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571114.13 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 57111.41 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732827.644378 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514002.72 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 51400.27 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784227.916284 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462602.45 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 46260.24 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830488.160999 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416342.20 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 41634.22 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872122.381243 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374707.98 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 37470.80 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909593.179462 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337237.18 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 33723.72 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943316.897859 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303513.47 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 30351.35 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973668.244417 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273162.12 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 27316.21 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000984.456319 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245845.91 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 24584.59 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025569.047031 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221261.32 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 22126.13 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047695.178671 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199135.18 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 19913.52 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067608.697148 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179221.67 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 17922.17 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085530.863777 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161299.50 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 16129.95 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101660.813743 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145169.55 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 14516.95 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116177.768712 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130652.59 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 13065.26 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129243.028184 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117587.34 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 11758.73 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141001.761710 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105828.60 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 10582.86 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151584.621882 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95245.74 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 9524.57 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161109.196038 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85721.17 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 8572.12 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169681.312777 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77149.05 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 7714.91 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177396.217843 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69434.15 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 6943.41 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184339.632403 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62490.73 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 6249.07 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190588.705506 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56241.66 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 5624.17 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196212.871299 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50617.49 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 5061.75 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201274.620513 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45555.74 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 4555.57 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205830.194805 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41000.17 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 4100.02 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209930.211668 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36900.15 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3690.02 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213620.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33210.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217220.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29610.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220820.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26010.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224420.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22410.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228020.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18810.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231620.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15210.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235220.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11610.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238820.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8010.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242420.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4410.14 sec) 08:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246020.226845 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 810.14 sec) 08:55:00 [GPUGRID] [rr_sim] 2246830.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988108556.47G/439.78G) 08:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1851971.28 08:55:00 [---] pick_jobs_to_run() start 08:55:00 [---] pick_jobs_to_run() end 08:55:00 [---] [rr_sim] end 08:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:55:00 [---] [cpu_sched_debug] preliminary job list: 08:55:00 [---] [cpu_sched_debug] final job list: 08:55:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 08:55:00 [---] [work_fetch] Request work fetch: CPUs idle 08:55:00 [---] [cpu_sched_debug] enforce_run_list: end 08:55:00 [---] choose_project(): 32100.000000 08:55:00 [iThena.Measurements] doing RPC to get work 08:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403499.09 cpu 2148812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:55:00 [---] Got 0 tasks 08:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:56:00 [---] [rr_sim] doing sim: CPU sched 08:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988108556.47G/439.78G) 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246830.36 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 224683.04 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224683.036344 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022147.33 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 202214.73 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426897.769053 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819932.59 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 181993.26 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608891.028491 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637939.33 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 163793.93 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772684.961986 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474145.40 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 147414.54 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920099.502131 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326730.86 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 132673.09 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052772.588261 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194057.78 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 119405.78 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172178.365779 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074652.00 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 107465.20 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279643.565545 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967186.80 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 96718.68 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376362.245334 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870468.12 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 87046.81 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463409.057144 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783421.31 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 78342.13 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541751.187773 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705079.18 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 70507.92 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612259.105339 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634571.26 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 63457.13 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675716.231149 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571114.13 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 57111.41 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732827.644378 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514002.72 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 51400.27 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784227.916284 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462602.45 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 46260.24 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830488.160999 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416342.20 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 41634.22 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872122.381243 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374707.98 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 37470.80 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909593.179462 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337237.18 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 33723.72 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943316.897859 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303513.47 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 30351.35 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973668.244417 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273162.12 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 27316.21 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000984.456319 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245845.91 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 24584.59 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025569.047031 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221261.32 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 22126.13 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047695.178671 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199135.18 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 19913.52 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067608.697148 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179221.67 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 17922.17 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085530.863777 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161299.50 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 16129.95 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101660.813743 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145169.55 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 14516.95 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116177.768712 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130652.59 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 13065.26 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129243.028184 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117587.34 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 11758.73 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141001.761710 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105828.60 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 10582.86 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151584.621882 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95245.74 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 9524.57 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161109.196038 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85721.17 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 8572.12 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169681.312777 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77149.05 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 7714.91 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177396.217843 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69434.15 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 6943.41 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184339.632403 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62490.73 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 6249.07 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190588.705506 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56241.66 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 5624.17 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196212.871299 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50617.49 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 5061.75 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201274.620513 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45555.74 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 4555.57 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205830.194805 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41000.17 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 4100.02 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209930.211668 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36900.15 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3690.02 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213620.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33210.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217220.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29610.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220820.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26010.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224420.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22410.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228020.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18810.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231620.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15210.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235220.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11610.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238820.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8010.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242420.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4410.14 sec) 08:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2246020.226845 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 810.14 sec) 08:56:00 [GPUGRID] [rr_sim] 2246830.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988108556.47G/439.78G) 08:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852031.28 08:56:00 [---] pick_jobs_to_run() start 08:56:00 [---] pick_jobs_to_run() end 08:56:00 [---] [rr_sim] end 08:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:56:00 [---] [cpu_sched_debug] preliminary job list: 08:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:56:00 [---] [cpu_sched_debug] final job list: 08:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:56:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 988108556.47G 08:56:00 [---] [cpu_sched_debug] enforce_run_list: end 08:56:00 [---] choose_project(): 32160.000000 08:56:00 [iThena.Measurements] doing RPC to get work 08:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403439.09 cpu 2148812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:56:00 [---] Got 0 tasks 08:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:57:00 [---] [rr_sim] doing sim: CPU sched 08:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988080966.12G/439.78G) 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246767.63 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 224676.76 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224676.762655 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022090.86 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 202209.09 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426885.849045 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819881.78 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 181988.18 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608874.026796 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637893.60 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 163789.36 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772663.386772 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474104.24 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 147410.42 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920073.810750 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326693.82 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 132669.38 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052743.192331 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194024.43 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 119402.44 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172145.635753 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074621.99 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 107462.20 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279607.834833 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967159.79 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 96715.98 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376323.814005 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870443.81 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 87044.38 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463368.195260 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783399.43 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 78339.94 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541708.138389 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705059.49 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 70505.95 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612214.087206 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634553.54 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 63455.35 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675669.441141 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571098.19 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 57109.82 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732779.259682 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513988.37 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 51398.84 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784178.096369 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462589.53 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 46258.95 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830437.049388 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416330.58 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 41633.06 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872070.107104 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374697.52 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 37469.75 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909539.859049 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337227.77 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 33722.78 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943262.635800 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303504.99 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 30350.50 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973613.134875 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273154.49 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 27315.45 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000928.584043 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245839.04 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 24583.90 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025512.488294 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221255.14 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 22125.51 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047638.002120 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199129.62 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 19912.96 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067550.964564 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179216.66 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 17921.67 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085472.630763 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161295.00 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 16129.50 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101602.130342 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145165.50 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 14516.55 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116118.679963 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130648.95 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 13064.89 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129183.574622 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117584.05 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 11758.41 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140941.979815 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105825.65 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 10582.56 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151524.544489 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95243.08 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 9524.31 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161048.852696 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85718.77 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 8571.88 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169620.730081 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77146.90 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 7714.69 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177335.419729 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69432.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 6943.22 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184278.640411 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62488.99 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 6248.90 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190527.539026 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56240.09 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 5624.01 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196151.547778 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50616.08 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 5061.61 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201213.155656 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45554.47 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 4555.45 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205768.602746 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40999.02 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 4099.90 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209868.505127 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36899.12 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3689.91 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213558.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33209.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217158.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29609.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220758.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26009.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224358.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22409.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227958.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18809.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231558.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15209.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235158.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11609.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238758.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8009.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242358.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4409.21 sec) 08:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245958.417269 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 809.21 sec) 08:57:00 [GPUGRID] [rr_sim] 2246767.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988080966.12G/439.78G) 08:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852028.54 08:57:00 [---] pick_jobs_to_run() start 08:57:00 [---] pick_jobs_to_run() end 08:57:00 [---] [rr_sim] end 08:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:57:00 [---] [cpu_sched_debug] preliminary job list: 08:57:00 [---] [cpu_sched_debug] final job list: 08:57:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 08:57:00 [---] [work_fetch] Request work fetch: CPUs idle 08:57:00 [---] [cpu_sched_debug] enforce_run_list: end 08:57:00 [---] choose_project(): 32220.000000 08:57:00 [iThena.Measurements] doing RPC to get work 08:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403379.09 cpu 2148752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:57:00 [---] Got 0 tasks 08:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:58:00 [---] [rr_sim] doing sim: CPU sched 08:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988080966.12G/439.78G) 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246767.63 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 224676.76 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224676.762655 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022090.86 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 202209.09 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426885.849045 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819881.78 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 181988.18 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608874.026796 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637893.60 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 163789.36 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772663.386772 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474104.24 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 147410.42 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920073.810750 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326693.82 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 132669.38 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052743.192331 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1194024.43 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 119402.44 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172145.635753 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074621.99 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 107462.20 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279607.834833 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967159.79 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 96715.98 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376323.814005 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870443.81 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 87044.38 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463368.195260 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783399.43 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 78339.94 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541708.138389 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705059.49 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 70505.95 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612214.087206 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634553.54 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 63455.35 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675669.441141 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571098.19 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 57109.82 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732779.259682 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513988.37 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 51398.84 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784178.096369 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462589.53 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 46258.95 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830437.049388 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416330.58 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 41633.06 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872070.107104 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374697.52 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 37469.75 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909539.859049 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337227.77 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 33722.78 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943262.635800 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303504.99 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 30350.50 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973613.134875 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273154.49 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 27315.45 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000928.584043 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245839.04 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 24583.90 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025512.488294 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221255.14 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 22125.51 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047638.002120 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199129.62 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 19912.96 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067550.964564 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179216.66 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 17921.67 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085472.630763 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161295.00 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 16129.50 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101602.130342 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145165.50 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 14516.55 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116118.679963 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130648.95 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 13064.89 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129183.574622 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117584.05 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 11758.41 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140941.979815 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105825.65 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 10582.56 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151524.544489 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95243.08 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 9524.31 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161048.852696 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85718.77 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 8571.88 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169620.730081 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77146.90 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 7714.69 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177335.419729 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69432.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 6943.22 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184278.640411 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62488.99 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 6248.90 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190527.539026 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56240.09 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 5624.01 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196151.547778 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50616.08 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 5061.61 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201213.155656 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45554.47 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 4555.45 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205768.602746 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40999.02 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 4099.90 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209868.505127 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36899.12 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3689.91 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213558.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33209.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217158.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29609.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220758.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26009.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224358.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22409.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227958.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18809.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231558.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15209.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235158.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11609.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238758.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8009.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242358.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4409.21 sec) 08:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245958.417269 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 809.21 sec) 08:58:00 [GPUGRID] [rr_sim] 2246767.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988080966.12G/439.78G) 08:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852088.54 08:58:00 [---] pick_jobs_to_run() start 08:58:00 [---] pick_jobs_to_run() end 08:58:00 [---] [rr_sim] end 08:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:58:00 [---] [cpu_sched_debug] preliminary job list: 08:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:58:00 [---] [cpu_sched_debug] final job list: 08:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:58:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 988080966.12G 08:58:00 [---] [cpu_sched_debug] enforce_run_list: end 08:58:00 [---] choose_project(): 32280.000000 08:58:00 [iThena.Measurements] doing RPC to get work 08:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403319.09 cpu 2148752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:58:00 [---] Got 0 tasks 08:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 08:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 08:59:00 [---] [rr_sim] doing sim: CPU sched 08:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988053375.76G/439.78G) 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246704.89 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 224670.49 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224670.488967 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022034.40 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 202203.44 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426873.929038 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819830.96 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 181983.10 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608857.025101 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637847.86 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 163784.79 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772641.811558 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474063.08 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 147406.31 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920048.119370 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326656.77 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 132665.68 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052713.796400 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193991.09 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 119399.11 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172112.905727 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074591.98 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 107459.20 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279572.104121 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967132.79 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 96713.28 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376285.382676 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870419.51 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 87041.95 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463327.333376 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783377.56 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 78337.76 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541665.089006 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705039.80 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 70503.98 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612169.069072 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634535.82 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 63453.58 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675622.651132 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571082.24 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 57108.22 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732730.874986 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513974.01 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 51397.40 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784128.276455 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462576.61 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 46257.66 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830385.937776 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416318.95 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 41631.90 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872017.832966 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374687.06 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 37468.71 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909486.538637 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337218.35 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 33721.84 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943208.373740 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303496.52 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 30349.65 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973558.025333 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273146.86 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 27314.69 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000872.711767 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245832.18 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 24583.22 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025455.929558 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221248.96 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 22124.90 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047580.825569 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199124.06 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 19912.41 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067493.231979 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179211.66 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 17921.17 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085414.397749 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161290.49 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 16129.05 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101543.446941 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145161.44 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 14516.14 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116059.591214 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130645.30 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 13064.53 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129124.121060 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117580.77 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 11758.08 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140882.197921 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105822.69 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 10582.27 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151464.467096 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95240.42 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 9524.04 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160988.509354 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85716.38 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 8571.64 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169560.147385 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77144.74 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 7714.47 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177274.621614 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69430.27 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 6943.03 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184217.648420 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62487.24 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 6248.72 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190466.372545 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56238.52 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 5623.85 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196090.224258 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50614.67 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 5061.47 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201151.690799 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45553.20 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 4555.32 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205707.010686 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40997.88 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 4099.79 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209806.798585 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36898.09 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3689.81 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213496.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33208.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217096.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29608.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220696.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26008.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224296.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22408.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227896.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18808.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231496.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15208.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235096.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11608.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238696.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8008.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242296.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4408.28 sec) 08:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245896.607694 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 808.28 sec) 08:59:00 [GPUGRID] [rr_sim] 2246704.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988053375.76G/439.78G) 08:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852085.80 08:59:00 [---] pick_jobs_to_run() start 08:59:00 [---] pick_jobs_to_run() end 08:59:00 [---] [rr_sim] end 08:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 08:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 08:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 08:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 08:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 08:59:00 [---] [cpu_sched_debug] preliminary job list: 08:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 08:59:00 [---] [cpu_sched_debug] final job list: 08:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 08:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 08:59:00 [---] [cpu_sched_debug] enforce_run_list: end 08:59:00 [---] choose_project(): 32340.000000 08:59:00 [iThena.Measurements] doing RPC to get work 08:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 576059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403259.09 cpu 2148692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 08:59:00 [---] Got 0 tasks 09:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:00:00 [---] [rr_sim] doing sim: CPU sched 09:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988053375.76G/439.78G) 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246704.89 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 224670.49 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224670.488967 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2022034.40 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 202203.44 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426873.929038 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819830.96 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 181983.10 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608857.025101 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637847.86 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 163784.79 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772641.811558 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474063.08 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 147406.31 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920048.119370 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326656.77 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 132665.68 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052713.796400 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193991.09 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 119399.11 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172112.905727 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074591.98 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 107459.20 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279572.104121 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967132.79 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 96713.28 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376285.382676 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870419.51 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 87041.95 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463327.333376 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783377.56 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 78337.76 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541665.089006 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705039.80 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 70503.98 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612169.069072 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634535.82 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 63453.58 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675622.651132 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571082.24 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 57108.22 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732730.874986 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513974.01 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 51397.40 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784128.276455 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462576.61 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 46257.66 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830385.937776 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416318.95 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 41631.90 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872017.832966 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374687.06 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 37468.71 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909486.538637 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337218.35 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 33721.84 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943208.373740 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303496.52 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 30349.65 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973558.025333 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273146.86 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 27314.69 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000872.711767 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245832.18 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 24583.22 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025455.929558 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221248.96 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 22124.90 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047580.825569 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199124.06 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 19912.41 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067493.231979 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179211.66 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 17921.17 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085414.397749 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161290.49 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 16129.05 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101543.446941 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145161.44 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 14516.14 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116059.591214 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130645.30 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 13064.53 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129124.121060 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117580.77 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 11758.08 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140882.197921 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105822.69 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 10582.27 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151464.467096 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95240.42 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 9524.04 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160988.509354 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85716.38 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 8571.64 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169560.147385 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77144.74 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 7714.47 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177274.621614 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69430.27 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 6943.03 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184217.648420 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62487.24 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 6248.72 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190466.372545 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56238.52 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 5623.85 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196090.224258 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50614.67 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 5061.47 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201151.690799 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45553.20 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 4555.32 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205707.010686 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40997.88 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 4099.79 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209806.798585 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36898.09 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3689.81 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213496.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33208.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217096.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29608.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220696.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26008.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224296.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22408.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227896.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18808.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231496.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15208.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235096.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11608.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238696.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8008.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242296.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4408.28 sec) 09:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245896.607694 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 808.28 sec) 09:00:00 [GPUGRID] [rr_sim] 2246704.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988053375.76G/439.78G) 09:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852145.80 09:00:00 [---] pick_jobs_to_run() start 09:00:00 [---] pick_jobs_to_run() end 09:00:00 [---] [rr_sim] end 09:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:00:00 [---] [cpu_sched_debug] preliminary job list: 09:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:00:00 [---] [cpu_sched_debug] final job list: 09:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:00:00 [---] [cpu_sched_debug] enforce_run_list: end 09:00:00 [---] choose_project(): 32400.000000 09:00:00 [iThena.Measurements] doing RPC to get work 09:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403199.09 cpu 2148692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:00:00 [---] Got 0 tasks 09:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:01:00 [---] [rr_sim] doing sim: CPU sched 09:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988025785.40G/439.78G) 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246642.15 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 224664.22 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224664.215279 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021977.94 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 202197.79 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426862.009030 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819780.14 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 181978.01 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608840.023406 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637802.13 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 163780.21 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772620.236344 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474021.92 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 147402.19 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920022.427989 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326619.72 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 132661.97 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052684.400469 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193957.75 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 119395.78 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172080.175701 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074561.98 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 107456.20 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279536.373410 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967105.78 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 96710.58 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376246.951348 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870395.20 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 87039.52 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463286.471492 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783355.68 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 78335.57 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541622.039622 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705020.11 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 70502.01 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612124.050939 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634518.10 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 63451.81 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675575.861124 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571066.29 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 57106.63 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732682.490290 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513959.66 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 51395.97 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784078.456540 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462563.70 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 46256.37 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830334.826165 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416307.33 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 41630.73 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871965.558828 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374676.59 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 37467.66 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909433.218224 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337208.93 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 33720.89 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943154.111680 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303488.04 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 30348.80 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973502.915791 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273139.24 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 27313.92 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000816.839491 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245825.31 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 24582.53 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025399.370821 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221242.78 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 22124.28 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047523.649018 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199118.50 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 19911.85 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067435.499395 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179206.65 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 17920.67 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085356.164735 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161285.99 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 16128.60 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101484.763540 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145157.39 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 14515.74 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116000.502465 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130641.65 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 13064.17 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129064.667497 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117577.49 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 11757.75 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140822.416027 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105819.74 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 10581.97 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151404.389703 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95237.76 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 9523.78 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160928.166012 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85713.99 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 8571.40 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169499.564689 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77142.59 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 7714.26 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177213.823499 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69428.33 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 6942.83 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184156.656429 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62485.50 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 6248.55 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190405.206065 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56236.95 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 5623.69 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196028.900737 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50613.25 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 5061.33 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201090.225942 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45551.93 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 4555.19 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205645.418627 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40996.73 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 4099.67 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209745.092043 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36897.06 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3689.71 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213434.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33207.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217034.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29607.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220634.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26007.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224234.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22407.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227834.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18807.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231434.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15207.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235034.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11607.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238634.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8007.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242234.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4407.35 sec) 09:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245834.798118 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 807.35 sec) 09:01:00 [GPUGRID] [rr_sim] 2246642.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988025785.40G/439.78G) 09:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852143.07 09:01:00 [---] pick_jobs_to_run() start 09:01:00 [---] pick_jobs_to_run() end 09:01:00 [---] [rr_sim] end 09:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:01:00 [---] [cpu_sched_debug] preliminary job list: 09:01:00 [---] [cpu_sched_debug] final job list: 09:01:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 09:01:00 [---] [work_fetch] Request work fetch: CPUs idle 09:01:00 [---] [cpu_sched_debug] enforce_run_list: end 09:01:00 [---] choose_project(): 32460.000000 09:01:00 [iThena.Measurements] doing RPC to get work 09:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403139.09 cpu 2148632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:01:00 [---] Got 0 tasks 09:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:02:00 [---] [rr_sim] doing sim: CPU sched 09:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (988025785.40G/439.78G) 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246642.15 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 224664.22 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224664.215279 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021977.94 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 202197.79 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426862.009030 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819780.14 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 181978.01 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608840.023406 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637802.13 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 163780.21 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772620.236344 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1474021.92 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 147402.19 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 920022.427989 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326619.72 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 132661.97 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052684.400469 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193957.75 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 119395.78 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172080.175701 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074561.98 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 107456.20 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279536.373410 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967105.78 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 96710.58 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376246.951348 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870395.20 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 87039.52 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463286.471492 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783355.68 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 78335.57 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541622.039622 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705020.11 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 70502.01 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612124.050939 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634518.10 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 63451.81 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675575.861124 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571066.29 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 57106.63 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732682.490290 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513959.66 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 51395.97 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784078.456540 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462563.70 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 46256.37 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830334.826165 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416307.33 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 41630.73 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871965.558828 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374676.59 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 37467.66 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909433.218224 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337208.93 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 33720.89 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943154.111680 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303488.04 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 30348.80 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973502.915791 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273139.24 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 27313.92 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000816.839491 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245825.31 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 24582.53 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025399.370821 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221242.78 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 22124.28 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047523.649018 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199118.50 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 19911.85 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067435.499395 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179206.65 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 17920.67 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085356.164735 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161285.99 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 16128.60 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101484.763540 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145157.39 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 14515.74 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116000.502465 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130641.65 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 13064.17 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129064.667497 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117577.49 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 11757.75 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140822.416027 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105819.74 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 10581.97 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151404.389703 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95237.76 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 9523.78 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160928.166012 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85713.99 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 8571.40 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169499.564689 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77142.59 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 7714.26 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177213.823499 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69428.33 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 6942.83 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184156.656429 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62485.50 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 6248.55 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190405.206065 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56236.95 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 5623.69 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196028.900737 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50613.25 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 5061.33 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201090.225942 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45551.93 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 4555.19 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205645.418627 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40996.73 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 4099.67 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209745.092043 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36897.06 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3689.71 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213434.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33207.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217034.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29607.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220634.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26007.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224234.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22407.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227834.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18807.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231434.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15207.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235034.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11607.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238634.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8007.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242234.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4407.35 sec) 09:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245834.798118 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 807.35 sec) 09:02:00 [GPUGRID] [rr_sim] 2246642.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (988025785.40G/439.78G) 09:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852203.07 09:02:00 [---] pick_jobs_to_run() start 09:02:00 [---] pick_jobs_to_run() end 09:02:00 [---] [rr_sim] end 09:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:02:00 [---] [cpu_sched_debug] preliminary job list: 09:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:02:00 [---] [cpu_sched_debug] final job list: 09:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 09:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:02:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 988025785.40G 09:02:00 [---] [cpu_sched_debug] enforce_run_list: end 09:02:00 [---] choose_project(): 32520.000000 09:02:00 [iThena.Measurements] doing RPC to get work 09:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403079.09 cpu 2148632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:02:00 [---] Got 0 tasks 09:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:03:00 [---] [rr_sim] doing sim: CPU sched 09:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987998195.05G/439.78G) 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246579.42 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 224657.94 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224657.941591 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021921.47 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 202192.15 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426850.089022 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819729.33 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 181972.93 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608823.021711 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637756.39 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 163775.64 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772598.661131 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473980.75 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 147398.08 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919996.736608 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326582.68 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 132658.27 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052655.004538 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193924.41 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 119392.44 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172047.445675 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074531.97 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 107453.20 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279500.642698 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967078.77 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 96707.88 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376208.520019 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870370.90 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 87037.09 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463245.609608 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783333.81 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 78333.38 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541578.990238 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705000.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 70500.04 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612079.032805 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634500.38 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 63450.04 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675529.071115 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571050.34 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 57105.03 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732634.105594 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513945.31 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 51394.53 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1784028.636626 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462550.78 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 46255.08 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830283.714554 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416295.70 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 41629.57 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871913.284689 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374666.13 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 37466.61 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909379.897811 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337199.52 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 33719.95 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943099.849621 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303479.57 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 30347.96 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973447.806249 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273131.61 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 27313.16 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000760.967215 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245818.45 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 24581.84 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025342.812084 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221236.60 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 22123.66 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047466.472467 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199112.94 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 19911.29 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067377.766811 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179201.65 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 17920.16 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085297.931721 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161281.48 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 16128.15 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101426.080139 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145153.34 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 14515.33 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115941.413716 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130638.00 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 13063.80 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129005.213935 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117574.20 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 11757.42 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140762.634132 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105816.78 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 10581.68 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151344.312310 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95235.10 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 9523.51 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160867.822670 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85711.59 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 8571.16 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169438.981993 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77140.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 7714.04 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177153.025385 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69426.39 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 6942.64 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184095.664437 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62483.75 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 6248.38 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190344.039584 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56235.38 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 5623.54 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195967.577217 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50611.84 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 5061.18 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201028.761086 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45550.65 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 4555.07 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205583.826568 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40995.59 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 4099.56 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209683.385502 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36896.03 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3689.60 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213372.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33206.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216972.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29606.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220572.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26006.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224172.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22406.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227772.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18806.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231372.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15206.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234972.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11606.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238572.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8006.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242172.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4406.43 sec) 09:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245772.988542 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 806.43 sec) 09:03:00 [GPUGRID] [rr_sim] 2246579.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987998195.05G/439.78G) 09:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852200.33 09:03:00 [---] pick_jobs_to_run() start 09:03:00 [---] pick_jobs_to_run() end 09:03:00 [---] [rr_sim] end 09:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:03:00 [---] [cpu_sched_debug] preliminary job list: 09:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 09:03:00 [---] [cpu_sched_debug] final job list: 09:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:03:00 [---] [cpu_sched_debug] enforce_run_list: end 09:03:00 [---] choose_project(): 32580.000000 09:03:00 [iThena.Measurements] doing RPC to get work 09:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 403019.09 cpu 2148572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:03:00 [---] Got 0 tasks 09:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:04:00 [---] [rr_sim] doing sim: CPU sched 09:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987970604.69G/439.78G) 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246516.68 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 224651.67 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224651.667903 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021865.01 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 202186.50 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426838.169015 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819678.51 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 181967.85 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608806.020016 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637710.66 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 163771.07 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772577.085917 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473939.59 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 147393.96 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919971.045228 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326545.63 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 132654.56 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052625.608607 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193891.07 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 119389.11 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1172014.715649 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074501.96 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 107450.20 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279464.911987 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967051.77 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 96705.18 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376170.088691 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870346.59 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 87034.66 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463204.747724 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783311.93 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 78331.19 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541535.940854 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704980.74 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 70498.07 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1612034.014671 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634482.66 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 63448.27 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675482.281107 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571034.40 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 57103.44 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732585.720899 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513930.96 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 51393.10 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783978.816711 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462537.86 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 46253.79 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830232.602943 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416284.08 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 41628.41 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871861.010551 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374655.67 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 37465.57 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909326.577398 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337190.10 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 33719.01 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943045.587561 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303471.09 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 30347.11 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973392.696708 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273123.98 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 27312.40 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000705.094939 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245811.58 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 24581.16 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025286.253348 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221230.43 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 22123.04 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047409.295916 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199107.38 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 19910.74 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067320.034227 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179196.64 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 17919.66 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085239.698706 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161276.98 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 16127.70 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101367.396738 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145149.28 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 14514.93 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115882.324967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130634.35 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 13063.44 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128945.760373 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117570.92 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 11757.09 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140702.852238 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105813.83 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 10581.38 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151284.234917 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95232.44 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 9523.24 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160807.479328 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85709.20 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 8570.92 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169378.399297 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77138.28 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 7713.83 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177092.227270 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69424.45 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 6942.45 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184034.672446 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62482.01 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 6248.20 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190282.873104 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56233.81 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 5623.38 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195906.253696 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50610.43 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 5061.04 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200967.296229 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45549.38 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 4554.94 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205522.234508 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40994.44 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 4099.44 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209621.678960 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36895.00 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3689.50 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213311.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33205.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216911.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29605.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220511.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26005.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224111.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22405.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227711.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18805.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231311.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15205.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234911.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11605.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238511.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8005.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242111.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4405.50 sec) 09:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245711.178967 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 805.50 sec) 09:04:00 [GPUGRID] [rr_sim] 2246516.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987970604.69G/439.78G) 09:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852197.59 09:04:00 [---] pick_jobs_to_run() start 09:04:00 [---] pick_jobs_to_run() end 09:04:00 [---] [rr_sim] end 09:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:04:00 [---] [cpu_sched_debug] preliminary job list: 09:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:04:00 [---] [cpu_sched_debug] final job list: 09:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:04:00 [---] [cpu_sched_debug] enforce_run_list: end 09:04:00 [---] choose_project(): 32640.000000 09:04:00 [iThena.Measurements] doing RPC to get work 09:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402959.09 cpu 2148512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:04:00 [---] Got 0 tasks 09:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:05:00 [---] [rr_sim] doing sim: CPU sched 09:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987943014.34G/439.78G) 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246453.94 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 224645.39 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224645.394214 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021808.55 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 202180.85 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426826.249007 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819627.69 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 181962.77 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608789.018321 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637664.92 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 163766.49 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772555.510703 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473898.43 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 147389.84 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919945.353847 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326508.59 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 132650.86 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052596.212677 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193857.73 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 119385.77 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171981.985623 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074471.96 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 107447.20 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279429.181275 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 967024.76 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 96702.48 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376131.657362 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870322.28 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 87032.23 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463163.885840 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783290.06 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 78329.01 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541492.891470 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704961.05 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 70496.11 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611988.996538 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634464.95 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 63446.49 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675435.491098 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571018.45 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 57101.85 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732537.336203 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513916.61 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 51391.66 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783928.996797 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462524.95 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 46252.49 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830181.491331 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416272.45 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 41627.25 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871808.736413 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374645.21 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 37464.52 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909273.256986 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337180.69 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 33718.07 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942991.325501 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303462.62 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 30346.26 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973337.587166 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273116.35 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 27311.64 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000649.222663 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245804.72 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 24580.47 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025229.694611 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221224.25 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 22122.42 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047352.119365 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199101.82 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 19910.18 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067262.301642 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179191.64 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 17919.16 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085181.465692 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161272.48 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 16127.25 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101308.713338 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145145.23 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 14514.52 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115823.236218 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130630.71 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 13063.07 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128886.306811 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117567.64 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 11756.76 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140643.070344 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105810.87 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 10581.09 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151224.157524 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95229.78 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 9522.98 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160747.135986 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85706.81 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 8570.68 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169317.816601 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77136.13 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 7713.61 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177031.429156 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69422.51 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 6942.25 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183973.680454 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62480.26 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 6248.03 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190221.706623 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56232.24 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 5623.22 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195844.930175 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50609.01 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 5060.90 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200905.831372 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45548.11 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 4554.81 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205460.642449 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40993.30 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 4099.33 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209559.972419 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36893.97 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3689.40 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213249.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33204.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216849.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29604.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220449.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26004.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224049.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22404.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227649.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18804.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231249.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15204.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234849.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11604.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238449.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8004.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242049.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4404.57 sec) 09:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245649.369391 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 804.57 sec) 09:05:00 [GPUGRID] [rr_sim] 2246453.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987943014.34G/439.78G) 09:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852194.85 09:05:00 [---] pick_jobs_to_run() start 09:05:00 [---] pick_jobs_to_run() end 09:05:00 [---] [rr_sim] end 09:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:05:00 [---] [cpu_sched_debug] preliminary job list: 09:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:05:00 [---] [cpu_sched_debug] final job list: 09:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:05:00 [---] [cpu_sched_debug] enforce_run_list: end 09:05:00 [---] choose_project(): 32700.000000 09:05:00 [iThena.Measurements] doing RPC to get work 09:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402899.09 cpu 2148452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:05:00 [---] Got 0 tasks 09:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:06:00 [---] [rr_sim] doing sim: CPU sched 09:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987915423.98G/439.78G) 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246391.21 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 224639.12 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224639.120526 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021752.08 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 202175.21 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426814.329000 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819576.88 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 181957.69 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608772.016626 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637619.19 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 163761.92 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772533.935489 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473857.27 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 147385.73 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919919.662466 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326471.54 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 132647.15 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052566.816746 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193824.39 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 119382.44 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171949.255597 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074441.95 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 107444.19 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279393.450564 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966997.75 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 96699.78 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376093.226033 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870297.98 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 87029.80 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463123.023956 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783268.18 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 78326.82 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541449.842087 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704941.36 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 70494.14 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611943.978404 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634447.23 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 63444.72 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675388.701090 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571002.50 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 57100.25 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732488.951507 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513902.25 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 51390.23 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783879.176882 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462512.03 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 46251.20 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830130.379720 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416260.83 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 41626.08 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871756.462274 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374634.74 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 37463.47 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909219.936573 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337171.27 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 33717.13 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942937.063442 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303454.14 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 30345.41 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973282.477624 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273108.73 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 27310.87 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000593.350387 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245797.85 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 24579.79 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025173.135875 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221218.07 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 22121.81 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047294.942813 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199096.26 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 19909.63 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067204.569058 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179186.64 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 17918.66 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085123.232678 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161267.97 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 16126.80 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101250.029937 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145141.18 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 14514.12 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115764.147469 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130627.06 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 13062.71 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128826.853248 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117564.35 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 11756.44 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140583.288450 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105807.92 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 10580.79 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151164.080131 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95227.13 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 9522.71 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160686.792644 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85704.41 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 8570.44 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169257.233905 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77133.97 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 7713.40 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176970.631041 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69420.57 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 6942.06 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183912.688463 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62478.52 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 6247.85 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190160.540143 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56230.67 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 5623.07 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195783.606655 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50607.60 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 5060.76 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200844.366515 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45546.84 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 4554.68 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205399.050390 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40992.15 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 4099.22 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209498.265877 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36892.94 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3689.29 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213187.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33203.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216787.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29603.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220387.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26003.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223987.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22403.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227587.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18803.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231187.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15203.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234787.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11603.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238387.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8003.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241987.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4403.65 sec) 09:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245587.559815 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 803.65 sec) 09:06:00 [GPUGRID] [rr_sim] 2246391.21: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987915423.98G/439.78G) 09:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852192.12 09:06:00 [---] pick_jobs_to_run() start 09:06:00 [---] pick_jobs_to_run() end 09:06:00 [---] [rr_sim] end 09:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:06:00 [---] [cpu_sched_debug] preliminary job list: 09:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:06:00 [---] [cpu_sched_debug] final job list: 09:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:06:00 [---] [cpu_sched_debug] enforce_run_list: end 09:06:00 [---] choose_project(): 32760.000000 09:06:00 [iThena.Measurements] doing RPC to get work 09:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402839.09 cpu 2148392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:06:00 [---] Got 0 tasks 09:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:07:00 [---] [rr_sim] doing sim: CPU sched 09:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987887833.62G/439.78G) 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246328.47 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 224632.85 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224632.846838 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021695.62 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 202169.56 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426802.408992 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819526.06 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 181952.61 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608755.014931 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637573.45 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 163757.35 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772512.360275 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473816.11 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 147381.61 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919893.971086 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326434.50 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 132643.45 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052537.420815 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193791.05 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 119379.10 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171916.525571 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074411.94 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 107441.19 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279357.719852 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966970.75 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 96697.07 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376054.794705 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870273.67 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 87027.37 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463082.162072 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783246.31 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 78324.63 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541406.792703 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704921.68 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 70492.17 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611898.960271 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634429.51 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 63442.95 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675341.911081 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570986.56 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 57098.66 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732440.566811 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513887.90 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 51388.79 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783829.356968 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462499.11 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 46249.91 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830079.268109 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416249.20 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 41624.92 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871704.188136 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374624.28 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 37462.43 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909166.616160 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337161.85 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 33716.19 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942882.801382 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303445.67 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 30344.57 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973227.368082 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273101.10 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 27310.11 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000537.478112 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245790.99 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 24579.10 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025116.577138 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221211.89 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 22121.19 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047237.766262 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199090.70 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 19909.07 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067146.836474 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179181.63 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 17918.16 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085064.999664 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161263.47 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 16126.35 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101191.346536 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145137.12 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 14513.71 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115705.058720 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130623.41 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 13062.34 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128767.399686 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117561.07 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 11756.11 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140523.506555 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105804.96 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 10580.50 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151104.002738 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95224.47 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 9522.45 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160626.449302 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85702.02 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 8570.20 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169196.651210 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77131.82 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 7713.18 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176909.832926 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69418.64 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 6941.86 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183851.696472 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62476.77 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 6247.68 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190099.373662 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56229.09 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 5622.91 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195722.283134 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50606.19 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 5060.62 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200782.901659 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45545.57 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 4554.56 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205337.458331 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40991.01 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 4099.10 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209436.559335 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36891.91 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3689.19 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213125.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33202.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216725.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29602.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220325.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26002.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223925.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22402.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227525.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18802.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231125.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15202.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234725.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11602.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238325.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8002.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241925.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4402.72 sec) 09:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245525.750240 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 802.72 sec) 09:07:00 [GPUGRID] [rr_sim] 2246328.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987887833.62G/439.78G) 09:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852189.38 09:07:00 [---] pick_jobs_to_run() start 09:07:00 [---] pick_jobs_to_run() end 09:07:00 [---] [rr_sim] end 09:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:07:00 [---] [cpu_sched_debug] preliminary job list: 09:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:07:00 [---] [cpu_sched_debug] final job list: 09:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:07:00 [---] [cpu_sched_debug] enforce_run_list: end 09:07:00 [---] choose_project(): 32820.000000 09:07:00 [iThena.Measurements] doing RPC to get work 09:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402779.09 cpu 2148332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:07:00 [---] Got 0 tasks 09:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:08:00 [---] [rr_sim] doing sim: CPU sched 09:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987860243.27G/439.78G) 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246265.73 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 224626.57 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224626.573150 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021639.16 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 202163.92 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426790.488984 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819475.24 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 181947.52 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608738.013236 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637527.72 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 163752.77 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772490.785062 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473774.95 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 147377.49 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919868.279705 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326397.45 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 132639.75 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052508.024884 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193757.71 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 119375.77 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171883.795546 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074381.94 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 107438.19 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279321.989141 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966943.74 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 96694.37 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1376016.363376 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870249.37 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 87024.94 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463041.300188 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783224.43 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 78322.44 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541363.743319 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704901.99 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 70490.20 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611853.942137 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634411.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 63441.18 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675295.121073 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570970.61 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 57097.06 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732392.182115 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513873.55 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 51387.35 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783779.537053 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462486.19 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 46248.62 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830028.156498 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416237.57 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 41623.76 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871651.913998 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374613.82 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 37461.38 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909113.295748 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337152.44 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 33715.24 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942828.539322 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303437.19 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 30343.72 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973172.258540 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273093.47 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 27309.35 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000481.605836 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245784.13 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 24578.41 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025060.018402 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221205.71 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 22120.57 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047180.589711 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199085.14 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 19908.51 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067089.103890 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179176.63 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 17917.66 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085006.766650 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161258.96 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 16125.90 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101132.663135 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145133.07 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 14513.31 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115645.969971 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130619.76 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 13061.98 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128707.946124 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117557.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 11755.78 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140463.724661 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105802.01 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 10580.20 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151043.925345 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95221.81 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 9522.18 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160566.105960 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85699.63 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 8569.96 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169136.068514 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77129.66 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 7712.97 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176849.034812 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69416.70 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 6941.67 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183790.704480 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62475.03 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 6247.50 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190038.207182 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56227.52 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 5622.75 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195660.959613 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50604.77 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 5060.48 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200721.436802 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45544.29 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 4554.43 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205275.866271 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40989.87 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 4098.99 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209374.852794 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36890.88 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3689.09 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213063.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33201.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216663.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29601.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220263.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26001.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223863.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22401.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227463.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18801.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231063.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15201.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234663.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11601.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238263.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8001.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241863.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4401.79 sec) 09:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245463.940664 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 801.79 sec) 09:08:00 [GPUGRID] [rr_sim] 2246265.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987860243.27G/439.78G) 09:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852186.64 09:08:00 [---] pick_jobs_to_run() start 09:08:00 [---] pick_jobs_to_run() end 09:08:00 [---] [rr_sim] end 09:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:08:00 [---] [cpu_sched_debug] preliminary job list: 09:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:08:00 [---] [cpu_sched_debug] final job list: 09:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:08:00 [---] [cpu_sched_debug] enforce_run_list: end 09:08:00 [---] choose_project(): 32880.000000 09:08:00 [iThena.Measurements] doing RPC to get work 09:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402719.09 cpu 2148272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:08:00 [---] Got 0 tasks 09:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:09:00 [---] [rr_sim] doing sim: CPU sched 09:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987832652.91G/439.78G) 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246202.99 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 224620.30 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224620.299461 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021582.70 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 202158.27 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426778.568977 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819424.43 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 181942.44 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608721.011541 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637481.98 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 163748.20 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772469.209848 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473733.78 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 147373.38 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919842.588325 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326360.41 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 132636.04 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052478.628954 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193724.37 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 119372.44 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171851.065520 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074351.93 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 107435.19 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279286.258429 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966916.74 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 96691.67 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375977.932048 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870225.06 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 87022.51 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1463000.438304 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783202.56 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 78320.26 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541320.693935 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704882.30 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 70488.23 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611808.924003 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634394.07 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 63439.41 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675248.331064 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570954.66 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 57095.47 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732343.797419 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513859.20 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 51385.92 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783729.717139 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462473.28 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 46247.33 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829977.044887 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416225.95 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 41622.59 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871599.639859 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374603.35 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 37460.34 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909059.975335 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337143.02 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 33714.30 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942774.277263 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303428.72 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 30342.87 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973117.148998 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273085.85 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 27308.58 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000425.733560 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245777.26 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 24577.73 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025003.459665 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221199.53 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 22119.95 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047123.413160 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199079.58 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 19907.96 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067031.371306 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179171.62 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 17917.16 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084948.533636 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161254.46 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 16125.45 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101073.979734 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145129.01 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 14512.90 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115586.881222 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130616.11 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 13061.61 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128648.492561 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117554.50 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 11755.45 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140403.942767 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105799.05 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 10579.91 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150983.847952 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95219.15 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 9521.91 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160505.762618 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85697.23 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 8569.72 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169075.485818 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77127.51 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 7712.75 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176788.236697 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69414.76 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 6941.48 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183729.712489 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62473.28 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 6247.33 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189977.040702 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56225.95 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 5622.60 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195599.636093 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50603.36 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 5060.34 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200659.971945 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45543.02 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 4554.30 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205214.274212 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40988.72 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 4098.87 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209313.146252 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36889.85 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3688.98 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213002.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33200.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216602.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29600.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220202.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26000.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223802.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22400.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227402.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18800.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231002.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15200.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234602.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11600.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238202.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8000.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241802.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4400.86 sec) 09:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245402.131088 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 800.86 sec) 09:09:00 [GPUGRID] [rr_sim] 2246202.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987832652.91G/439.78G) 09:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852183.91 09:09:00 [---] pick_jobs_to_run() start 09:09:00 [---] pick_jobs_to_run() end 09:09:00 [---] [rr_sim] end 09:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:09:00 [---] [cpu_sched_debug] preliminary job list: 09:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:09:00 [---] [cpu_sched_debug] final job list: 09:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:09:00 [---] [cpu_sched_debug] enforce_run_list: end 09:09:00 [---] choose_project(): 32940.000000 09:09:00 [iThena.Measurements] doing RPC to get work 09:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402659.09 cpu 2148212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:09:00 [---] Got 0 tasks 09:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:10:00 [---] [rr_sim] doing sim: CPU sched 09:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987805062.55G/439.78G) 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246140.26 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 224614.03 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224614.025773 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021526.23 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 202152.62 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426766.648969 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819373.61 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 181937.36 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608704.009845 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637436.25 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 163743.62 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772447.634634 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473692.62 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 147369.26 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919816.896944 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326323.36 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 132632.34 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052449.233023 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193691.02 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 119369.10 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171818.335494 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074321.92 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 107432.19 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279250.527718 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966889.73 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 96688.97 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375939.500719 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870200.76 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 87020.08 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462959.576420 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783180.68 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 78318.07 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541277.644552 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704862.61 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 70486.26 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611763.905870 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634376.35 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 63437.64 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675201.541056 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570938.72 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 57093.87 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732295.412724 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513844.85 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 51384.48 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783679.897224 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462460.36 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 46246.04 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829925.933275 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416214.32 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 41621.43 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871547.365721 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374592.89 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 37459.29 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909006.654922 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337133.60 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 33713.36 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942720.015203 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303420.24 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 30342.02 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973062.039456 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273078.22 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 27307.82 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000369.861284 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245770.40 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 24577.04 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024946.900929 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221193.36 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 22119.34 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047066.236609 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199074.02 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 19907.40 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066973.638721 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179166.62 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 17916.66 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084890.300622 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161249.96 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 16125.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101015.296333 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145124.96 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 14512.50 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115527.792473 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130612.47 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 13061.25 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128589.038999 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117551.22 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 11755.12 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140344.160873 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105796.10 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 10579.61 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150923.770558 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95216.49 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 9521.65 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160445.419276 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85694.84 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 8569.48 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169014.903122 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77125.35 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 7712.54 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176727.438583 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69412.82 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 6941.28 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183668.720498 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62471.54 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 6247.15 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189915.874221 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56224.38 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 5622.44 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195538.312572 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50601.95 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 5060.19 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200598.507088 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45541.75 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 4554.18 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205152.682153 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40987.58 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 4098.76 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209251.439711 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36888.82 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3688.88 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212940.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33199.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216540.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29599.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220140.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25999.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223740.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22399.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227340.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18799.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230940.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15199.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234540.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11599.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238140.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7999.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241740.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4399.94 sec) 09:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245340.321513 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 799.94 sec) 09:10:00 [GPUGRID] [rr_sim] 2246140.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987805062.55G/439.78G) 09:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852181.17 09:10:00 [---] pick_jobs_to_run() start 09:10:00 [---] pick_jobs_to_run() end 09:10:00 [---] [rr_sim] end 09:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:10:00 [---] [cpu_sched_debug] preliminary job list: 09:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:10:00 [---] [cpu_sched_debug] final job list: 09:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:10:00 [---] [cpu_sched_debug] enforce_run_list: end 09:10:00 [---] choose_project(): 33000.000000 09:10:00 [iThena.Measurements] doing RPC to get work 09:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402599.09 cpu 2148152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:10:00 [---] Got 0 tasks 09:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:11:00 [---] [rr_sim] doing sim: CPU sched 09:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987777472.20G/439.78G) 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246077.52 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 224607.75 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224607.752085 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021469.77 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 202146.98 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426754.728962 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819322.79 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 181932.28 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608687.008150 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637390.51 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 163739.05 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772426.059420 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473651.46 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 147365.15 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919791.205563 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326286.32 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 132628.63 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052419.837092 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193657.68 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 119365.77 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171785.605468 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074291.92 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 107429.19 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279214.797006 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966862.72 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 96686.27 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375901.069391 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870176.45 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 87017.65 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462918.714536 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783158.81 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 78315.88 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541234.595168 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704842.93 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 70484.29 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611718.887736 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634358.63 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 63435.86 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675154.751047 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570922.77 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 57092.28 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732247.028028 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513830.49 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 51383.05 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783630.077310 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462447.44 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 46244.74 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829874.821664 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416202.70 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 41620.27 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871495.091583 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374582.43 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 37458.24 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908953.334509 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337124.19 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 33712.42 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942665.753143 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303411.77 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 30341.18 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973006.929914 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273070.59 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 27307.06 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000313.989008 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245763.53 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 24576.35 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024890.342192 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221187.18 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 22118.72 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047009.060058 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199068.46 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 19906.85 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066915.906137 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179161.61 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 17916.16 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084832.067608 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161245.45 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 16124.55 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100956.612933 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145120.91 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 14512.09 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115468.703724 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130608.82 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 13060.88 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128529.585437 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117547.94 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 11754.79 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140284.378978 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105793.14 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 10579.31 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150863.693165 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95213.83 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 9521.38 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160385.075934 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85692.44 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 8569.24 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168954.320426 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77123.20 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 7712.32 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176666.640468 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69410.88 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 6941.09 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183607.728506 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62469.79 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 6246.98 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189854.707741 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56222.81 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 5622.28 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195476.989052 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50600.53 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 5060.05 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200537.042231 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45540.48 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 4554.05 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205091.090093 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40986.43 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 4098.64 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209189.733169 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36887.79 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3688.78 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212878.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33199.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216478.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29599.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220078.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25999.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223678.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22399.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227278.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18799.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230878.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15199.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234478.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11599.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238078.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7999.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241678.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4399.01 sec) 09:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245278.511937 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 799.01 sec) 09:11:00 [GPUGRID] [rr_sim] 2246077.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987777472.20G/439.78G) 09:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852178.43 09:11:00 [---] pick_jobs_to_run() start 09:11:00 [---] pick_jobs_to_run() end 09:11:00 [---] [rr_sim] end 09:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:11:00 [---] [cpu_sched_debug] preliminary job list: 09:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:11:00 [---] [cpu_sched_debug] final job list: 09:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:11:00 [---] [cpu_sched_debug] enforce_run_list: end 09:11:00 [---] choose_project(): 33060.000000 09:11:00 [iThena.Measurements] doing RPC to get work 09:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402539.09 cpu 2148092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:11:00 [---] Got 0 tasks 09:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:12:00 [---] [rr_sim] doing sim: CPU sched 09:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987749881.84G/439.78G) 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2246014.78 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 224601.48 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224601.478397 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021413.31 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 202141.33 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426742.808954 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819271.98 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 181927.20 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608670.006455 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637344.78 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 163734.48 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772404.484207 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473610.30 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 147361.03 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919765.514183 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326249.27 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 132624.93 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052390.441161 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193624.34 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 119362.43 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171752.875442 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074261.91 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 107426.19 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279179.066295 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966835.72 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 96683.57 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375862.638062 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870152.15 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 87015.21 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462877.852653 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783136.93 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 78313.69 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541191.545784 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704823.24 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 70482.32 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611673.869602 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634340.91 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 63434.09 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675107.961039 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570906.82 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 57090.68 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732198.643332 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513816.14 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 51381.61 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783580.257396 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462434.53 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 46243.45 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829823.710053 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416191.07 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 41619.11 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871442.817444 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374571.97 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 37457.20 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908900.014097 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337114.77 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 33711.48 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942611.491084 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303403.29 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 30340.33 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972951.820372 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273062.96 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 27306.30 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000258.116732 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245756.67 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 24575.67 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024833.783455 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221181.00 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 22118.10 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046951.883507 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199062.90 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 19906.29 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066858.173553 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179156.61 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 17915.66 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084773.834594 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161240.95 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 16124.09 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100897.929532 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145116.85 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 14511.69 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115409.614975 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130605.17 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 13060.52 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128470.131875 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117544.65 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 11754.47 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140224.597084 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105790.19 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 10579.02 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150803.615772 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95211.17 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 9521.12 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160324.732592 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85690.05 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 8569.01 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168893.737730 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77121.05 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 7712.10 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176605.842353 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69408.94 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 6940.89 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183546.736515 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62468.05 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 6246.80 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189793.541260 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56221.24 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 5622.12 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195415.665531 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50599.12 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 5059.91 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200475.577375 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45539.21 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 4553.92 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205029.498034 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40985.29 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 4098.53 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209128.026627 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36886.76 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3688.68 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212816.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33198.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216416.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29598.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220016.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25998.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223616.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22398.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227216.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18798.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230816.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15198.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234416.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11598.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238016.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7998.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241616.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4398.08 sec) 09:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245216.702361 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 798.08 sec) 09:12:00 [GPUGRID] [rr_sim] 2246014.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987749881.84G/439.78G) 09:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852175.70 09:12:00 [---] pick_jobs_to_run() start 09:12:00 [---] pick_jobs_to_run() end 09:12:00 [---] [rr_sim] end 09:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:12:00 [---] [cpu_sched_debug] preliminary job list: 09:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:12:00 [---] [cpu_sched_debug] final job list: 09:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:12:00 [---] [cpu_sched_debug] enforce_run_list: end 09:12:00 [---] choose_project(): 33120.000000 09:12:00 [iThena.Measurements] doing RPC to get work 09:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402479.09 cpu 2148032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2148032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2148032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:12:00 [---] Got 0 tasks 09:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:13:00 [---] [rr_sim] doing sim: CPU sched 09:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987722291.49G/439.78G) 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245952.05 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 224595.20 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224595.204709 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021356.84 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 202135.68 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426730.888946 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819221.16 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 181922.12 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608653.004760 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637299.04 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 163729.90 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772382.908993 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473569.14 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 147356.91 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919739.822802 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326212.22 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 132621.22 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052361.045231 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193591.00 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 119359.10 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171720.145416 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074231.90 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 107423.19 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279143.335583 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966808.71 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 96680.87 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375824.206733 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870127.84 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 87012.78 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462836.990769 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783115.06 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 78311.51 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541148.496400 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704803.55 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 70480.36 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611628.851469 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634323.20 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 63432.32 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675061.171031 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570890.88 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 57089.09 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732150.258636 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513801.79 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 51380.18 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783530.437481 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462421.61 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 46242.16 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829772.598442 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416179.45 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 41617.94 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871390.543306 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374561.50 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 37456.15 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908846.693684 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337105.35 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 33710.54 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942557.229024 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303394.82 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 30339.48 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972896.710830 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273055.34 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 27305.53 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000202.244456 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245749.80 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 24574.98 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024777.224719 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221174.82 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 22117.48 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046894.706956 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199057.34 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 19905.73 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066800.440969 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179151.61 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 17915.16 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084715.601580 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161236.45 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 16123.64 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100839.246131 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145112.80 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 14511.28 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115350.526226 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130601.52 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 13060.15 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128410.678312 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117541.37 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 11754.14 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140164.815190 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105787.23 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 10578.72 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150743.538379 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95208.51 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 9520.85 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160264.389250 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85687.66 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 8568.77 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168833.155034 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77118.89 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 7711.89 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176545.044239 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69407.00 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 6940.70 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183485.744523 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62466.30 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 6246.63 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189732.374780 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56219.67 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 5621.97 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195354.342010 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50597.71 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 5059.77 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200414.112518 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45537.93 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 4553.79 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204967.905975 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40984.14 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 4098.41 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209066.320086 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36885.73 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3688.57 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212754.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33197.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216354.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29597.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219954.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25997.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223554.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22397.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227154.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18797.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230754.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15197.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234354.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11597.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237954.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7997.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241554.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4397.15 sec) 09:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245154.892786 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 797.15 sec) 09:13:00 [GPUGRID] [rr_sim] 2245952.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987722291.49G/439.78G) 09:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852172.96 09:13:00 [---] pick_jobs_to_run() start 09:13:00 [---] pick_jobs_to_run() end 09:13:00 [---] [rr_sim] end 09:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:13:00 [---] [cpu_sched_debug] preliminary job list: 09:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:13:00 [---] [cpu_sched_debug] final job list: 09:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:13:00 [---] [cpu_sched_debug] enforce_run_list: end 09:13:00 [---] choose_project(): 33180.000000 09:13:00 [iThena.Measurements] doing RPC to get work 09:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402419.09 cpu 2147972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:13:00 [---] Got 0 tasks 09:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:14:00 [---] [rr_sim] doing sim: CPU sched 09:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987694701.13G/439.78G) 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245889.31 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 224588.93 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224588.931020 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021300.38 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 202130.04 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426718.968939 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819170.34 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 181917.03 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608636.003065 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637253.31 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 163725.33 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772361.333779 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473527.98 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 147352.80 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919714.131422 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326175.18 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 132617.52 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052331.649300 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193557.66 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 119355.77 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171687.415390 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074201.89 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 107420.19 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279107.604871 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966781.71 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 96678.17 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375785.775405 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870103.53 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 87010.35 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462796.128885 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783093.18 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 78309.32 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541105.447017 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704783.86 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 70478.39 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611583.833335 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634305.48 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 63430.55 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1675014.381022 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570874.93 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 57087.49 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732101.873940 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513787.44 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 51378.74 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783480.617567 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462408.69 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 46240.87 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829721.486830 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416167.82 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 41616.78 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871338.269168 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374551.04 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 37455.10 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908793.373271 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337095.94 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 33709.59 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942502.966964 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303386.34 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 30338.63 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972841.601288 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273047.71 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 27304.77 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000146.372180 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245742.94 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 24574.29 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024720.665982 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221168.64 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 22116.86 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046837.530404 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199051.78 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 19905.18 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066742.708384 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179146.60 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 17914.66 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084657.368566 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161231.94 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 16123.19 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100780.562730 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145108.75 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 14510.87 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115291.437477 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130597.87 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 13059.79 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128351.224750 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117538.09 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 11753.81 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140105.033295 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105784.28 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 10578.43 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150683.460986 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95205.85 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 9520.58 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160204.045908 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85685.26 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 8568.53 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168772.572338 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77116.74 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 7711.67 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176484.246124 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69405.06 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 6940.51 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183424.752532 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62464.56 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 6246.46 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189671.208299 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56218.10 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 5621.81 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195293.018490 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50596.29 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 5059.63 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200352.647661 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45536.66 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 4553.67 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204906.313915 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40983.00 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 4098.30 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209004.613544 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36884.70 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3688.47 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212693.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33196.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216293.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29596.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219893.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25996.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223493.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22396.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227093.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18796.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230693.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15196.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234293.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11596.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237893.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7996.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241493.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4396.23 sec) 09:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245093.083210 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 796.23 sec) 09:14:00 [GPUGRID] [rr_sim] 2245889.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987694701.13G/439.78G) 09:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852170.22 09:14:00 [---] pick_jobs_to_run() start 09:14:00 [---] pick_jobs_to_run() end 09:14:00 [---] [rr_sim] end 09:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:14:00 [---] [cpu_sched_debug] preliminary job list: 09:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:14:00 [---] [cpu_sched_debug] final job list: 09:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:14:00 [---] [cpu_sched_debug] enforce_run_list: end 09:14:00 [---] choose_project(): 33240.000000 09:14:00 [iThena.Measurements] doing RPC to get work 09:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402359.09 cpu 2147912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:14:00 [---] Got 0 tasks 09:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:15:00 [---] [rr_sim] doing sim: CPU sched 09:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987667110.77G/439.78G) 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245826.57 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 224582.66 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224582.657332 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021243.92 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 202124.39 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426707.048931 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819119.52 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 181911.95 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608619.001370 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637207.57 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 163720.76 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772339.758565 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473486.81 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 147348.68 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919688.440041 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326138.13 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 132613.81 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052302.253369 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193524.32 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 119352.43 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171654.685364 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074171.89 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 107417.19 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279071.874160 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966754.70 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 96675.47 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375747.344076 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870079.23 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 87007.92 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462755.267001 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783071.31 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 78307.13 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541062.397633 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704764.18 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 70476.42 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611538.815202 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634287.76 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 63428.78 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674967.591014 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570858.98 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 57085.90 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732053.489244 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513773.08 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 51377.31 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783430.797652 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462395.78 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 46239.58 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829670.375219 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416156.20 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 41615.62 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871285.995029 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374540.58 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 37454.06 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908740.052859 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337086.52 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 33708.65 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942448.704905 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303377.87 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 30337.79 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972786.491747 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273040.08 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 27304.01 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000090.499904 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245736.07 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 24573.61 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024664.107246 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221162.47 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 22116.25 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046780.353853 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199046.22 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 19904.62 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066684.975800 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179141.60 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 17914.16 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084599.135552 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161227.44 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 16122.74 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100721.879329 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145104.69 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 14510.47 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115232.348728 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130594.22 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 13059.42 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128291.771188 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117534.80 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 11753.48 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140045.251401 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105781.32 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 10578.13 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150623.383593 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95203.19 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 9520.32 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160143.702566 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85682.87 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 8568.29 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168711.989642 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77114.58 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 7711.46 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176423.448010 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69403.13 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 6940.31 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183363.760541 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62462.81 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 6246.28 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189610.041819 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56216.53 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 5621.65 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195231.694969 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50594.88 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 5059.49 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200291.182804 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45535.39 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 4553.54 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204844.721856 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40981.85 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 4098.19 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208942.907003 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36883.67 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3688.37 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212631.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33195.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216231.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29595.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219831.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25995.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223431.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22395.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227031.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18795.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230631.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15195.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234231.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11595.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237831.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7995.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241431.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4395.30 sec) 09:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2245031.273635 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 795.30 sec) 09:15:00 [GPUGRID] [rr_sim] 2245826.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987667110.77G/439.78G) 09:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852167.49 09:15:00 [---] pick_jobs_to_run() start 09:15:00 [---] pick_jobs_to_run() end 09:15:00 [---] [rr_sim] end 09:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:15:00 [---] [cpu_sched_debug] preliminary job list: 09:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:15:00 [---] [cpu_sched_debug] final job list: 09:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:15:00 [---] [cpu_sched_debug] enforce_run_list: end 09:15:00 [---] choose_project(): 33300.000000 09:15:00 [iThena.Measurements] doing RPC to get work 09:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402299.09 cpu 2147852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:15:00 [---] Got 0 tasks 09:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:16:00 [---] [rr_sim] doing sim: CPU sched 09:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987639520.42G/439.78G) 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245763.84 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 224576.38 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224576.383644 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021187.45 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 202118.75 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426695.128923 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819068.71 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 181906.87 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608601.999675 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637161.84 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 163716.18 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772318.183351 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473445.65 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 147344.57 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919662.748660 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326101.09 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 132610.11 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052272.857438 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193490.98 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 119349.10 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171621.955338 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074141.88 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 107414.19 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279036.143448 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966727.69 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 96672.77 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375708.912748 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870054.92 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 87005.49 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462714.405117 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783049.43 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 78304.94 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1541019.348249 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704744.49 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 70474.45 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611493.797068 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634270.04 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 63427.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674920.801005 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570843.04 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 57084.30 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1732005.104549 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513758.73 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 51375.87 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783380.977738 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462382.86 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 46238.29 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829619.263608 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416144.57 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 41614.46 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871233.720891 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374530.12 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 37453.01 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908686.732446 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337077.10 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 33707.71 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942394.442845 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303369.39 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 30336.94 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972731.382205 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273032.45 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 27303.25 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000034.627628 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245729.21 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 24572.92 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024607.548509 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221156.29 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 22115.63 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046723.177302 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199040.66 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 19904.07 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066627.243216 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179136.59 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 17913.66 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084540.902538 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161222.93 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 16122.29 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100663.195928 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145100.64 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 14510.06 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115173.259979 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130590.58 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 13059.06 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128232.317625 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117531.52 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 11753.15 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139985.469507 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105778.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 10577.84 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150563.306200 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95200.53 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 9520.05 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160083.359224 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85680.48 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 8568.05 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168651.406946 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77112.43 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 7711.24 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176362.649895 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69401.19 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 6940.12 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183302.768549 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62461.07 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 6246.11 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189548.875338 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56214.96 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 5621.50 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195170.371448 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50593.46 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 5059.35 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200229.717948 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45534.12 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 4553.41 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204783.129797 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40980.71 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 4098.07 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208881.200461 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36882.64 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3688.26 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212569.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33194.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216169.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29594.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219769.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25994.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223369.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22394.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226969.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18794.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230569.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15194.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234169.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11594.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237769.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7994.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241369.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4394.37 sec) 09:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244969.464059 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 794.37 sec) 09:16:00 [GPUGRID] [rr_sim] 2245763.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987639520.42G/439.78G) 09:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852164.75 09:16:00 [---] pick_jobs_to_run() start 09:16:00 [---] pick_jobs_to_run() end 09:16:00 [---] [rr_sim] end 09:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:16:00 [---] [cpu_sched_debug] preliminary job list: 09:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:16:00 [---] [cpu_sched_debug] final job list: 09:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:16:00 [---] [cpu_sched_debug] enforce_run_list: end 09:16:00 [---] choose_project(): 33360.000000 09:16:00 [iThena.Measurements] doing RPC to get work 09:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 575039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402239.09 cpu 2147792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:16:00 [---] Got 0 tasks 09:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:17:00 [---] [rr_sim] doing sim: CPU sched 09:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987611930.06G/439.78G) 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245701.10 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 224570.11 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224570.109956 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021130.99 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 202113.10 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426683.208916 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1819017.89 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 181901.79 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608584.997980 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637116.10 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 163711.61 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772296.608138 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473404.49 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 147340.45 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919637.057280 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326064.04 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 132606.40 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052243.461507 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193457.64 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 119345.76 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171589.225312 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074111.87 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 107411.19 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1279000.412737 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966700.69 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 96670.07 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375670.481419 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870030.62 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 87003.06 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462673.543233 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783027.56 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 78302.76 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540976.298865 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704724.80 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 70472.48 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611448.778934 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634252.32 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 63425.23 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674874.010997 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570827.09 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 57082.71 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731956.719853 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513744.38 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 51374.44 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783331.157823 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462369.94 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 46236.99 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829568.151997 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416132.95 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 41613.29 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871181.446753 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374519.65 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 37451.97 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908633.412033 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337067.69 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 33706.77 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942340.180786 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303360.92 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 30336.09 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972676.272663 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273024.83 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 27302.48 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999978.755352 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245722.34 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 24572.23 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024550.989773 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221150.11 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 22115.01 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046666.000751 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199035.10 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 19903.51 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066569.510632 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179131.59 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 17913.16 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084482.669524 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161218.43 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 16121.84 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100604.512528 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145096.59 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 14509.66 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115114.171231 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130586.93 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 13058.69 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128172.864063 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117528.24 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 11752.82 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139925.687613 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105775.41 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 10577.54 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150503.228807 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95197.87 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 9519.79 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160023.015882 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85678.08 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 8567.81 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168590.824250 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77110.28 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 7711.03 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176301.851780 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69399.25 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 6939.92 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183241.776558 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62459.32 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 6245.93 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189487.708858 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56213.39 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 5621.34 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195109.047928 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50592.05 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 5059.21 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200168.253091 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45532.85 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 4553.28 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204721.537737 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40979.56 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 4097.96 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208819.493919 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36881.61 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3688.16 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212507.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33193.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216107.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29593.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219707.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25993.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223307.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22393.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226907.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18793.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230507.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15193.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234107.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11593.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237707.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7993.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241307.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4393.45 sec) 09:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244907.654483 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 793.45 sec) 09:17:00 [GPUGRID] [rr_sim] 2245701.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987611930.06G/439.78G) 09:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852162.01 09:17:00 [---] pick_jobs_to_run() start 09:17:00 [---] pick_jobs_to_run() end 09:17:00 [---] [rr_sim] end 09:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:17:00 [---] [cpu_sched_debug] preliminary job list: 09:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:17:00 [---] [cpu_sched_debug] final job list: 09:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:17:00 [---] [cpu_sched_debug] enforce_run_list: end 09:17:00 [---] choose_project(): 33420.000000 09:17:00 [iThena.Measurements] doing RPC to get work 09:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402179.09 cpu 2147732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:17:00 [---] Got 0 tasks 09:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:18:00 [---] [rr_sim] doing sim: CPU sched 09:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987584339.70G/439.78G) 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245638.36 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 224563.84 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224563.836268 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021074.53 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 202107.45 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426671.288908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818967.07 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 181896.71 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608567.996285 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637070.37 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 163707.04 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772275.032924 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473363.33 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 147336.33 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919611.365899 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1326027.00 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 132602.70 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052214.065577 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193424.30 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 119342.43 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171556.495286 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074081.87 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 107408.19 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278964.682025 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966673.68 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 96667.37 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375632.050090 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 870006.31 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 87000.63 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462632.681349 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783005.68 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 78300.57 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540933.249481 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704705.11 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 70470.51 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611403.760801 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634234.60 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 63423.46 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674827.220988 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570811.14 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 57081.11 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731908.335157 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513730.03 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 51373.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783281.337909 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462357.02 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 46235.70 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829517.040385 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416121.32 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 41612.13 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871129.172614 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374509.19 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 37450.92 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908580.091620 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337058.27 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 33705.83 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942285.918726 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303352.44 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 30335.24 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972621.163121 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273017.20 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 27301.72 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999922.883076 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245715.48 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 24571.55 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024494.431036 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221143.93 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 22114.39 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046608.824200 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199029.54 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 19902.95 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066511.778047 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179126.58 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 17912.66 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084424.436510 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161213.93 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 16121.39 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100545.829127 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145092.53 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 14509.25 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115055.082482 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130583.28 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 13058.33 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128113.410501 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117524.95 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 11752.50 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139865.905718 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105772.46 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 10577.25 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150443.151414 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95195.21 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 9519.52 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159962.672540 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85675.69 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 8567.57 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168530.241554 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77108.12 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 7710.81 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176241.053666 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69397.31 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 6939.73 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183180.784567 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62457.58 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 6245.76 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189426.542378 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56211.82 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 5621.18 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195047.724407 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50590.64 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 5059.06 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200106.788234 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45531.57 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 4553.16 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204659.945678 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40978.42 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 4097.84 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208757.787378 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36880.58 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3688.06 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212445.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33192.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216045.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29592.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219645.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25992.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223245.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22392.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226845.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18792.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230445.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15192.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234045.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11592.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237645.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7992.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241245.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4392.52 sec) 09:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244845.844908 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 792.52 sec) 09:18:00 [GPUGRID] [rr_sim] 2245638.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987584339.70G/439.78G) 09:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852159.28 09:18:00 [---] pick_jobs_to_run() start 09:18:00 [---] pick_jobs_to_run() end 09:18:00 [---] [rr_sim] end 09:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:18:00 [---] [cpu_sched_debug] preliminary job list: 09:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:18:00 [---] [cpu_sched_debug] final job list: 09:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:18:00 [---] [cpu_sched_debug] enforce_run_list: end 09:18:00 [---] choose_project(): 33480.000000 09:18:00 [iThena.Measurements] doing RPC to get work 09:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402119.09 cpu 2147672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:18:00 [---] Got 0 tasks 09:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:19:00 [---] [rr_sim] doing sim: CPU sched 09:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987556749.35G/439.78G) 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245575.63 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 224557.56 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224557.562579 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2021018.06 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 202101.81 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426659.368901 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818916.26 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 181891.63 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608550.994590 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1637024.63 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 163702.46 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772253.457710 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473322.17 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 147332.22 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919585.674518 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325989.95 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 132599.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052184.669646 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193390.96 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 119339.10 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171523.765261 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074051.86 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 107405.19 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278928.951314 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966646.67 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 96664.67 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375593.618762 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869982.01 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 86998.20 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462591.819465 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782983.81 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 78298.38 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540890.200098 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704685.43 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 70468.54 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611358.742667 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634216.88 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 63421.69 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674780.430980 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570795.19 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 57079.52 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731859.950461 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513715.68 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 51371.57 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783231.517994 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462344.11 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 46234.41 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829465.928774 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416109.70 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 41610.97 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871076.898476 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374498.73 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 37449.87 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908526.771208 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337048.85 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 33704.89 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942231.656666 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303343.97 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 30334.40 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972566.053579 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273009.57 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 27300.96 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999867.010800 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245708.61 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 24570.86 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024437.872300 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221137.75 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 22113.78 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046551.647649 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199023.98 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 19902.40 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066454.045463 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179121.58 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 17912.16 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084366.203496 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161209.42 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 16120.94 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100487.145726 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145088.48 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 14508.85 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114995.993733 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130579.63 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 13057.96 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128053.956939 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117521.67 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 11752.17 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139806.123824 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105769.50 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 10576.95 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150383.074021 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95192.55 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 9519.26 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159902.329198 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85673.30 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 8567.33 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168469.658858 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77105.97 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 7710.60 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176180.255551 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69395.37 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 6939.54 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183119.792575 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62455.83 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 6245.58 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189365.375897 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56210.25 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 5621.02 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194986.400887 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50589.22 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 5058.92 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200045.323377 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45530.30 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 4553.03 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204598.353619 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40977.27 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 4097.73 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208696.080836 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36879.54 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3687.95 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212384.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33191.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215984.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29591.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219584.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25991.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223184.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22391.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226784.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18791.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230384.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15191.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233984.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11591.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237584.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7991.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241184.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4391.59 sec) 09:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244784.035332 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 791.59 sec) 09:19:00 [GPUGRID] [rr_sim] 2245575.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987556749.35G/439.78G) 09:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852156.54 09:19:00 [---] pick_jobs_to_run() start 09:19:00 [---] pick_jobs_to_run() end 09:19:00 [---] [rr_sim] end 09:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:19:00 [---] [cpu_sched_debug] preliminary job list: 09:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:19:00 [---] [cpu_sched_debug] final job list: 09:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:19:00 [---] [cpu_sched_debug] enforce_run_list: end 09:19:00 [---] choose_project(): 33540.000000 09:19:00 [iThena.Measurements] doing RPC to get work 09:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 402059.09 cpu 2147612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:19:00 [---] Got 0 tasks 09:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:20:00 [---] [rr_sim] doing sim: CPU sched 09:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987529158.99G/439.78G) 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245512.89 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 224551.29 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224551.288891 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020961.60 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 202096.16 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426647.448893 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818865.44 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 181886.54 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608533.992895 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636978.90 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 163697.89 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772231.882496 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473281.01 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 147328.10 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919559.983138 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325952.91 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 132595.29 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052155.273715 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193357.62 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 119335.76 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171491.035235 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1074021.85 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 107402.19 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278893.220602 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966619.67 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 96661.97 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375555.187433 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869957.70 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 86995.77 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462550.957581 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782961.93 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 78296.19 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540847.150714 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704665.74 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 70466.57 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611313.724534 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634199.16 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 63419.92 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674733.640971 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570779.25 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 57077.92 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731811.565765 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513701.32 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 51370.13 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783181.698080 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462331.19 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 46233.12 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829414.817163 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416098.07 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 41609.81 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871024.624338 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374488.26 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 37448.83 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908473.450795 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337039.44 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 33703.94 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942177.394607 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303335.49 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 30333.55 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972510.944037 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273001.94 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 27300.19 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999811.138524 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245701.75 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 24570.18 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024381.313563 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221131.58 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 22113.16 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046494.471098 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199018.42 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 19901.84 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066396.312879 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179116.58 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 17911.66 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084307.970482 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161204.92 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 16120.49 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100428.462325 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145084.43 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 14508.44 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114936.904984 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130575.98 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 13057.60 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127994.503376 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117518.39 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 11751.84 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139746.341930 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105766.55 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 10576.65 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150322.996628 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95189.89 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 9518.99 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159841.985856 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85670.90 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 8567.09 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168409.076162 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77103.81 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 7710.38 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176119.457437 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69393.43 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 6939.34 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183058.800584 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62454.09 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 6245.41 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189304.209417 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56208.68 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 5620.87 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194925.077366 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50587.81 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 5058.78 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199983.858520 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45529.03 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 4552.90 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204536.761560 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40976.13 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 4097.61 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208634.374295 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36878.51 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3687.85 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212322.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33190.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215922.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29590.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219522.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25990.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223122.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22390.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226722.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18790.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230322.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15190.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233922.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11590.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237522.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7990.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241122.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4390.66 sec) 09:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244722.225756 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 790.66 sec) 09:20:00 [GPUGRID] [rr_sim] 2245512.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987529158.99G/439.78G) 09:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852153.80 09:20:00 [---] pick_jobs_to_run() start 09:20:00 [---] pick_jobs_to_run() end 09:20:00 [---] [rr_sim] end 09:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:20:00 [---] [cpu_sched_debug] preliminary job list: 09:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:20:00 [---] [cpu_sched_debug] final job list: 09:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:20:00 [---] [cpu_sched_debug] enforce_run_list: end 09:20:00 [---] choose_project(): 33600.000000 09:20:00 [iThena.Measurements] doing RPC to get work 09:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401999.09 cpu 2147552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:20:00 [---] Got 0 tasks 09:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:21:00 [---] [rr_sim] doing sim: CPU sched 09:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987501568.64G/439.78G) 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245450.15 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 224545.02 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224545.015203 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020905.14 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 202090.51 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426635.528885 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818814.62 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 181881.46 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608516.991200 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636933.16 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 163693.32 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772210.307283 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473239.84 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 147323.98 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919534.291757 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325915.86 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 132591.59 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052125.877784 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193324.27 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 119332.43 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171458.305209 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073991.85 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 107399.18 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278857.489891 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966592.66 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 96659.27 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375516.756105 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869933.40 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 86993.34 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462510.095697 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782940.06 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 78294.01 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540804.101330 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704646.05 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 70464.61 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611268.706400 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634181.45 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 63418.14 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674686.850963 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570763.30 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 57076.33 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731763.181069 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513686.97 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 51368.70 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783131.878165 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462318.27 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 46231.83 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829363.705552 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416086.45 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 41608.64 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870972.350199 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374477.80 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 37447.78 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908420.130382 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337030.02 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 33703.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942123.132547 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303327.02 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 30332.70 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972455.834495 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272994.32 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 27299.43 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999755.266248 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245694.89 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 24569.49 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024324.754826 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221125.40 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 22112.54 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046437.294547 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199012.86 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 19901.29 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066338.580295 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179111.57 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 17911.16 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084249.737468 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161200.41 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 16120.04 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100369.778924 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145080.37 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 14508.04 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114877.816235 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130572.34 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 13057.23 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127935.049814 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117515.10 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 11751.51 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139686.560035 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105763.59 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 10576.36 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150262.919235 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95187.23 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 9518.72 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159781.642514 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85668.51 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 8566.85 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168348.493466 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77101.66 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 7710.17 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176058.659322 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69391.49 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 6939.15 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182997.808593 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62452.34 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 6245.23 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189243.042936 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56207.11 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 5620.71 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194863.753845 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50586.40 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 5058.64 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199922.393664 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45527.76 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 4552.78 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204475.169500 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40974.98 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 4097.50 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208572.667753 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36877.48 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3687.75 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212260.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33189.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215860.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29589.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219460.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25989.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223060.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22389.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226660.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18789.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230260.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15189.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233860.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11589.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237460.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7989.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241060.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4389.74 sec) 09:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244660.416181 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 789.74 sec) 09:21:00 [GPUGRID] [rr_sim] 2245450.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987501568.64G/439.78G) 09:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852151.06 09:21:00 [---] pick_jobs_to_run() start 09:21:00 [---] pick_jobs_to_run() end 09:21:00 [---] [rr_sim] end 09:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:21:00 [---] [cpu_sched_debug] preliminary job list: 09:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:21:00 [---] [cpu_sched_debug] final job list: 09:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:21:00 [---] [cpu_sched_debug] enforce_run_list: end 09:21:00 [---] choose_project(): 33660.000000 09:21:00 [iThena.Measurements] doing RPC to get work 09:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401939.09 cpu 2147492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:21:00 [---] Got 0 tasks 09:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:22:00 [---] [rr_sim] doing sim: CPU sched 09:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987473978.28G/439.78G) 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245387.42 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 224538.74 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224538.741515 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020848.67 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 202084.87 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426623.608878 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818763.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 181876.38 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608499.989505 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636887.43 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 163688.74 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772188.732069 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473198.68 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 147319.87 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919508.600377 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325878.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 132587.88 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052096.481854 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193290.93 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 119329.09 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171425.575183 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073961.84 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 107396.18 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278821.759179 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966565.66 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 96656.57 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375478.324776 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869909.09 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 86990.91 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462469.233813 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782918.18 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 78291.82 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540761.051946 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704626.36 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 70462.64 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611223.688266 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634163.73 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 63416.37 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674640.060954 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570747.35 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 57074.74 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731714.796374 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513672.62 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 51367.26 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783082.058251 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462305.36 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 46230.54 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829312.593940 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416074.82 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 41607.48 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870920.076061 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374467.34 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 37446.73 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908366.809970 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337020.61 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 33702.06 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942068.870487 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303318.54 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 30331.85 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972400.724953 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272986.69 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 27298.67 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999699.393972 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245688.02 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 24568.80 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024268.196090 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221119.22 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 22111.92 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046380.117995 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199007.30 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 19900.73 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066280.847711 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179106.57 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 17910.66 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084191.504454 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161195.91 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 16119.59 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100311.095523 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145076.32 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 14507.63 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114818.727486 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130568.69 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 13056.87 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127875.596252 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117511.82 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 11751.18 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139626.778141 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105760.64 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 10576.06 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150202.841842 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95184.57 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 9518.46 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159721.299172 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85666.12 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 8566.61 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168287.910770 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77099.50 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 7709.95 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175997.861207 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69389.55 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 6938.96 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182936.816601 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62450.60 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 6245.06 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189181.876456 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56205.54 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 5620.55 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194802.430325 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50584.98 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 5058.50 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199860.928807 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45526.49 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 4552.65 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204413.577441 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40973.84 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 4097.38 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208510.961211 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36876.45 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3687.65 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212198.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33188.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215798.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29588.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219398.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25988.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222998.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22388.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226598.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18788.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230198.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15188.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233798.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11588.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237398.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7988.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240998.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4388.81 sec) 09:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244598.606605 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 788.81 sec) 09:22:00 [GPUGRID] [rr_sim] 2245387.42: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987473978.28G/439.78G) 09:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852148.33 09:22:00 [---] pick_jobs_to_run() start 09:22:00 [---] pick_jobs_to_run() end 09:22:00 [---] [rr_sim] end 09:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:22:00 [---] [cpu_sched_debug] preliminary job list: 09:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:22:00 [---] [cpu_sched_debug] final job list: 09:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:22:00 [---] [cpu_sched_debug] enforce_run_list: end 09:22:00 [---] choose_project(): 33720.000000 09:22:00 [iThena.Measurements] doing RPC to get work 09:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401879.09 cpu 2147432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:22:00 [---] Got 0 tasks 09:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:23:00 [---] [rr_sim] doing sim: CPU sched 09:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987446387.92G/439.78G) 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245324.68 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 224532.47 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224532.467826 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020792.21 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 202079.22 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426611.688870 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818712.99 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 181871.30 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608482.987810 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636841.69 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 163684.17 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772167.156855 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473157.52 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 147315.75 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919482.908996 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325841.77 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 132584.18 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052067.085923 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193257.59 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 119325.76 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171392.845157 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073931.83 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 107393.18 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278786.028468 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966538.65 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 96653.86 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375439.893447 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869884.78 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 86988.48 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462428.371929 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782896.31 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 78289.63 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540718.002563 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704606.68 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 70460.67 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611178.670133 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634146.01 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 63414.60 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674593.270946 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570731.41 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 57073.14 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731666.411678 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513658.27 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 51365.83 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1783032.238336 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462292.44 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 46229.24 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829261.482329 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416063.20 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 41606.32 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870867.801923 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374456.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 37445.69 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908313.489557 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337011.19 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 33701.12 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942014.608428 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303310.07 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 30331.01 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972345.615411 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272979.06 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 27297.91 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999643.521697 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245681.16 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 24568.12 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024211.637353 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221113.04 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 22111.30 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046322.941444 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199001.74 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 19900.17 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066223.115126 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179101.56 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 17910.16 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084133.271440 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161191.41 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 16119.14 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100252.412123 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145072.27 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 14507.23 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114759.638737 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130565.04 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 13056.50 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127816.142689 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117508.54 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 11750.85 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139566.996247 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105757.68 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 10575.77 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150142.764449 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95181.91 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 9518.19 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159660.955830 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85663.72 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 8566.37 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168227.328074 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77097.35 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 7709.74 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175937.063093 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69387.62 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 6938.76 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182875.824610 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62448.85 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 6244.89 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189120.709975 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56203.97 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 5620.40 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194741.106804 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50583.57 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 5058.36 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199799.463950 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45525.21 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 4552.52 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204351.985382 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40972.69 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 4097.27 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208449.254670 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36875.42 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3687.54 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212136.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33187.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215736.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29587.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219336.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25987.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222936.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22387.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226536.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18787.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230136.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15187.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233736.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11587.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237336.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7987.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240936.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4387.88 sec) 09:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244536.797029 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 787.88 sec) 09:23:00 [GPUGRID] [rr_sim] 2245324.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987446387.92G/439.78G) 09:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852145.59 09:23:00 [---] pick_jobs_to_run() start 09:23:00 [---] pick_jobs_to_run() end 09:23:00 [---] [rr_sim] end 09:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:23:00 [---] [cpu_sched_debug] preliminary job list: 09:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:23:00 [---] [cpu_sched_debug] final job list: 09:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:23:00 [---] [cpu_sched_debug] enforce_run_list: end 09:23:00 [---] choose_project(): 33780.000000 09:23:00 [iThena.Measurements] doing RPC to get work 09:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401819.09 cpu 2147372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:23:00 [---] Got 0 tasks 09:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:24:00 [---] [rr_sim] doing sim: CPU sched 09:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987418797.57G/439.78G) 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245261.94 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 224526.19 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224526.194138 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020735.75 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 202073.57 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426599.768863 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818662.17 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 181866.22 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608465.986115 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636795.96 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 163679.60 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772145.581641 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473116.36 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 147311.64 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919457.217615 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325804.72 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 132580.47 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052037.689992 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193224.25 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 119322.43 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171360.115131 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073901.83 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 107390.18 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278750.297756 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966511.64 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 96651.16 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375401.462119 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869860.48 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 86986.05 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462387.510045 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782874.43 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 78287.44 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540674.953179 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704586.99 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 70458.70 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611133.651999 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634128.29 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 63412.83 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674546.480937 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570715.46 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 57071.55 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731618.026982 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513643.91 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 51364.39 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782982.418422 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462279.52 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 46227.95 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829210.370718 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416051.57 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 41605.16 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870815.527784 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374446.41 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 37444.64 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908260.169144 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337001.77 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 33700.18 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941960.346368 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303301.60 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 30330.16 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972290.505869 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272971.44 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 27297.14 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999587.649421 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245674.29 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 24567.43 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024155.078617 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221106.86 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 22110.69 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046265.764893 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198996.18 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 19899.62 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066165.382542 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179096.56 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 17909.66 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084075.038426 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161186.90 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 16118.69 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100193.728722 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145068.21 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 14506.82 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114700.549988 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130561.39 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 13056.14 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127756.689127 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117505.25 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 11750.53 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139507.214353 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105754.73 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 10575.47 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150082.687056 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95179.25 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 9517.93 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159600.612488 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85661.33 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 8566.13 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168166.745378 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77095.20 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 7709.52 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175876.264978 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69385.68 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 6938.57 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182814.832618 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62447.11 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 6244.71 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189059.543495 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56202.40 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 5620.24 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194679.783284 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50582.16 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 5058.22 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199737.999093 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45523.94 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 4552.39 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204290.393322 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40971.55 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 4097.15 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208387.548128 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36874.39 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3687.44 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212074.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33186.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215674.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29586.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219274.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25986.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222874.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22386.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226474.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18786.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230074.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15186.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233674.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11586.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237274.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7986.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240874.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4386.95 sec) 09:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244474.987454 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786.95 sec) 09:24:00 [GPUGRID] [rr_sim] 2245261.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987418797.57G/439.78G) 09:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852142.85 09:24:00 [---] pick_jobs_to_run() start 09:24:00 [---] pick_jobs_to_run() end 09:24:00 [---] [rr_sim] end 09:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:24:00 [---] [cpu_sched_debug] preliminary job list: 09:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:24:00 [---] [cpu_sched_debug] final job list: 09:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:24:00 [---] [cpu_sched_debug] enforce_run_list: end 09:24:00 [---] choose_project(): 33840.000000 09:24:00 [iThena.Measurements] doing RPC to get work 09:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401759.09 cpu 2147312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:24:00 [---] Got 0 tasks 09:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:25:00 [---] [rr_sim] doing sim: CPU sched 09:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987391207.21G/439.78G) 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245199.20 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 224519.92 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224519.920450 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020679.28 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 202067.93 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426587.848855 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818611.36 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 181861.14 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608448.984419 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636750.22 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 163675.02 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772124.006428 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473075.20 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 147307.52 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919431.526235 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325767.68 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 132576.77 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1052008.294061 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193190.91 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 119319.09 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171327.385105 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073871.82 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 107387.18 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278714.567045 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966484.64 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 96648.46 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375363.030790 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869836.17 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 86983.62 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462346.648161 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782852.56 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 78285.26 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540631.903795 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704567.30 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 70456.73 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611088.633865 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634110.57 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 63411.06 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674499.690929 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570699.51 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 57069.95 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731569.642286 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513629.56 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 51362.96 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782932.598507 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462266.61 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 46226.66 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829159.259107 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416039.95 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 41603.99 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870763.253646 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374435.95 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 37443.60 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908206.848731 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336992.36 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 33699.24 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941906.084308 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303293.12 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 30329.31 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972235.396327 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272963.81 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 27296.38 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999531.777145 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245667.43 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 24566.74 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024098.519880 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221100.68 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 22110.07 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046208.588342 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198990.62 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 19899.06 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066107.649958 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179091.55 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 17909.16 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084016.805412 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161182.40 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 16118.24 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100135.045321 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145064.16 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 14506.42 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114641.461239 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130557.74 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 13055.77 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127697.235565 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117501.97 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 11750.20 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139447.432458 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105751.77 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 10575.18 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150022.609663 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95176.59 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 9517.66 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159540.269146 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85658.94 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 8565.89 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168106.162682 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77093.04 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 7709.30 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175815.466863 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69383.74 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 6938.37 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182753.840627 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62445.36 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 6244.54 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188998.377014 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56200.83 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 5620.08 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194618.459763 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50580.74 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 5058.07 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199676.534237 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45522.67 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 4552.27 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204228.801263 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40970.40 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 4097.04 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208325.841587 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36873.36 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3687.34 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212013.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33186.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215613.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29586.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219213.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25986.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222813.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22386.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226413.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18786.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230013.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15186.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233613.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11586.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237213.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7986.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240813.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4386.03 sec) 09:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244413.177878 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 786.03 sec) 09:25:00 [GPUGRID] [rr_sim] 2245199.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987391207.21G/439.78G) 09:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852140.12 09:25:00 [---] pick_jobs_to_run() start 09:25:00 [---] pick_jobs_to_run() end 09:25:00 [---] [rr_sim] end 09:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:25:00 [---] [cpu_sched_debug] preliminary job list: 09:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:25:00 [---] [cpu_sched_debug] final job list: 09:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:25:00 [---] [cpu_sched_debug] enforce_run_list: end 09:25:00 [---] choose_project(): 33900.000000 09:25:00 [iThena.Measurements] doing RPC to get work 09:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401699.09 cpu 2147252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:25:00 [---] Got 0 tasks 09:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:26:00 [---] [rr_sim] doing sim: CPU sched 09:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987363616.86G/439.78G) 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245136.47 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 224513.65 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224513.646762 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020622.82 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 202062.28 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426575.928847 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818560.54 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 181856.05 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608431.982724 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636704.48 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 163670.45 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772102.431214 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1473034.04 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 147303.40 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919405.834854 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325730.63 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 132573.06 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051978.898130 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193157.57 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 119315.76 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171294.655079 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073841.81 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 107384.18 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278678.836333 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966457.63 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 96645.76 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375324.599462 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869811.87 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 86981.19 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462305.786277 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782830.68 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 78283.07 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540588.854411 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704547.61 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 70454.76 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1611043.615732 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634092.85 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 63409.29 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674452.900920 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570683.57 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 57068.36 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731521.257590 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513615.21 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 51361.52 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782882.778593 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462253.69 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 46225.37 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829108.147495 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416028.32 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 41602.83 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870710.979508 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374425.49 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 37442.55 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908153.528319 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336982.94 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 33698.29 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941851.822249 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303284.65 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 30328.46 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972180.286785 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272956.18 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 27295.62 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999475.904869 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245660.56 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 24566.06 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024041.961144 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221094.51 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 22109.45 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046151.411791 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198985.06 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 19898.51 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066049.917374 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179086.55 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 17908.66 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083958.572398 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161177.90 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 16117.79 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100076.361920 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145060.11 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 14506.01 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114582.372490 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130554.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 13055.41 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127637.782003 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117498.69 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 11749.87 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139387.650564 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105748.82 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 10574.88 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149962.532269 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95173.94 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 9517.39 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159479.925804 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85656.54 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 8565.65 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168045.579986 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77090.89 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 7709.09 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175754.668749 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69381.80 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 6938.18 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182692.848636 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62443.62 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 6244.36 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188937.210534 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56199.26 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 5619.93 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194557.136242 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50579.33 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 5057.93 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199615.069380 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45521.40 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 4552.14 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204167.209204 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40969.26 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 4096.93 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208264.135045 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36872.33 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3687.23 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211951.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33185.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215551.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29585.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219151.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25985.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222751.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22385.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226351.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18785.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229951.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15185.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233551.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11585.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237151.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7985.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240751.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4385.10 sec) 09:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244351.368302 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 785.10 sec) 09:26:00 [GPUGRID] [rr_sim] 2245136.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987363616.86G/439.78G) 09:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852137.38 09:26:00 [---] pick_jobs_to_run() start 09:26:00 [---] pick_jobs_to_run() end 09:26:00 [---] [rr_sim] end 09:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:26:00 [---] [cpu_sched_debug] preliminary job list: 09:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:26:00 [---] [cpu_sched_debug] final job list: 09:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:26:00 [---] [cpu_sched_debug] enforce_run_list: end 09:26:00 [---] choose_project(): 33960.000000 09:26:00 [iThena.Measurements] doing RPC to get work 09:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401639.09 cpu 2147192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:26:00 [---] Got 0 tasks 09:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:27:00 [---] [rr_sim] doing sim: CPU sched 09:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987336026.50G/439.78G) 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245073.73 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 224507.37 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224507.373074 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020566.36 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 202056.64 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426564.008840 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818509.72 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 181850.97 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608414.981029 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636658.75 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 163665.87 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772080.856000 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472992.87 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 147299.29 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919380.143474 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325693.59 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 132569.36 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051949.502200 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193124.23 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 119312.42 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171261.925053 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073811.81 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 107381.18 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278643.105622 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966430.63 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 96643.06 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375286.168133 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869787.56 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 86978.76 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462264.924393 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782808.81 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 78280.88 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540545.805027 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704527.93 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 70452.79 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610998.597598 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634075.13 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 63407.51 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674406.110912 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570667.62 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 57066.76 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731472.872894 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513600.86 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 51360.09 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782832.958678 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462240.77 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 46224.08 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829057.035884 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416016.69 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 41601.67 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870658.705369 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374415.03 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 37441.50 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908100.207906 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336973.52 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 33697.35 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941797.560189 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303276.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 30327.62 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972125.177244 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272948.55 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 27294.86 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999420.032593 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245653.70 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 24565.37 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023985.402407 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221088.33 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 22108.83 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046094.235240 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198979.50 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 19897.95 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065992.184789 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179081.55 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 17908.15 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083900.339384 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161173.39 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 16117.34 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100017.678519 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145056.05 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 14505.61 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114523.283741 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130550.45 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 13055.04 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127578.328440 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117495.40 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 11749.54 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139327.868670 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105745.86 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 10574.59 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149902.454876 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95171.28 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 9517.13 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159419.582462 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85654.15 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 8565.41 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167984.997290 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77088.73 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 7708.87 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175693.870634 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69379.86 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 6937.99 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182631.856644 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62441.87 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 6244.19 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188876.044053 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56197.69 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 5619.77 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194495.812722 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50577.92 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 5057.79 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199553.604523 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45520.13 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 4552.01 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204105.617144 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40968.11 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 4096.81 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208202.428503 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36871.30 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3687.13 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211889.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33184.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215489.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29584.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219089.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25984.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222689.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22384.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226289.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18784.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229889.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15184.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233489.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11584.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237089.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7984.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240689.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4384.17 sec) 09:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244289.558727 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 784.17 sec) 09:27:00 [GPUGRID] [rr_sim] 2245073.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987336026.50G/439.78G) 09:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852134.64 09:27:00 [---] pick_jobs_to_run() start 09:27:00 [---] pick_jobs_to_run() end 09:27:00 [---] [rr_sim] end 09:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:27:00 [---] [cpu_sched_debug] preliminary job list: 09:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:27:00 [---] [cpu_sched_debug] final job list: 09:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:27:00 [---] [cpu_sched_debug] enforce_run_list: end 09:27:00 [---] choose_project(): 34020.000000 09:27:00 [iThena.Measurements] doing RPC to get work 09:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401579.09 cpu 2147132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:27:00 [---] Got 0 tasks 09:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:28:00 [---] [rr_sim] doing sim: CPU sched 09:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987308436.14G/439.78G) 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2245010.99 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 224501.10 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224501.099385 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020509.89 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 202050.99 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426552.088832 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818458.91 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 181845.89 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608397.979334 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636613.01 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 163661.30 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772059.280786 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472951.71 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 147295.17 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919354.452093 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325656.54 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 132565.65 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051920.106269 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193090.89 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 119309.09 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171229.195027 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073781.80 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 107378.18 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278607.374910 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966403.62 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 96640.36 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375247.736804 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869763.26 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 86976.33 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462224.062509 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782786.93 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 78278.69 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540502.755644 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704508.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 70450.82 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610953.579465 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634057.41 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 63405.74 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674359.320904 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570651.67 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 57065.17 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731424.488198 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513586.51 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 51358.65 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782783.138764 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462227.86 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 46222.79 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829005.924273 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416005.07 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 41600.51 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870606.431231 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374404.56 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 37440.46 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908046.887493 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336964.11 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 33696.41 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941743.298129 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303267.70 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 30326.77 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972070.067702 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272940.93 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 27294.09 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999364.160317 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245646.83 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 24564.68 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023928.843670 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221082.15 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 22108.22 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046037.058689 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198973.94 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 19897.39 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065934.452205 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179076.54 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 17907.65 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083842.106370 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161168.89 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 16116.89 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099958.995118 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145052.00 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 14505.20 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114464.194992 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130546.80 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 13054.68 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127518.874878 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117492.12 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 11749.21 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139268.086776 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105742.91 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 10574.29 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149842.377483 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95168.62 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 9516.86 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159359.239120 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85651.75 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 8565.18 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167924.414594 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77086.58 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 7708.66 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175633.072520 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69377.92 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 6937.79 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182570.864653 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62440.13 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 6244.01 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188814.877573 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56196.12 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 5619.61 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194434.489201 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50576.50 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 5057.65 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199492.139666 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45518.85 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 4551.89 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204044.025085 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40966.97 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 4096.70 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208140.721962 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36870.27 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3687.03 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211827.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33183.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215427.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29583.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219027.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25983.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222627.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22383.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226227.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18783.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229827.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15183.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233427.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11583.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237027.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7983.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240627.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4383.24 sec) 09:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244227.749151 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 783.24 sec) 09:28:00 [GPUGRID] [rr_sim] 2245010.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987308436.14G/439.78G) 09:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852131.91 09:28:00 [---] pick_jobs_to_run() start 09:28:00 [---] pick_jobs_to_run() end 09:28:00 [---] [rr_sim] end 09:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:28:00 [---] [cpu_sched_debug] preliminary job list: 09:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:28:00 [---] [cpu_sched_debug] final job list: 09:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:28:00 [---] [cpu_sched_debug] enforce_run_list: end 09:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:29:00 [---] [rr_sim] doing sim: CPU sched 09:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987280845.79G/439.78G) 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244948.26 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 224494.83 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224494.825697 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020453.43 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 202045.34 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426540.168825 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818408.09 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 181840.81 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608380.977639 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636567.28 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 163656.73 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772037.705572 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472910.55 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 147291.06 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919328.760712 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325619.50 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 132561.95 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051890.710338 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193057.55 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 119305.75 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171196.465001 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073751.79 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 107375.18 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278571.644198 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966376.61 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 96637.66 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375209.305476 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869738.95 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 86973.90 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462183.200625 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782765.06 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 78276.51 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540459.706260 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704488.55 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 70448.86 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610908.561331 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634039.70 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 63403.97 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674312.530895 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570635.73 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 57063.57 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731376.103503 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513572.15 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 51357.22 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782733.318850 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462214.94 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 46221.49 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828954.812662 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415993.44 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 41599.34 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870554.157093 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374394.10 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 37439.41 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907993.567080 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336954.69 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 33695.47 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941689.036070 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303259.22 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 30325.92 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972014.958160 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272933.30 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 27293.33 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999308.288041 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245639.97 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 24564.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023872.284934 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221075.97 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 22107.60 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045979.882138 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198968.37 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 19896.84 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065876.719621 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179071.54 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 17907.15 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083783.873356 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161164.38 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 16116.44 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099900.311718 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145047.95 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 14504.79 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114405.106243 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130543.15 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 13054.32 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127459.421316 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117488.84 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 11748.88 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139208.304881 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105739.95 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 10574.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149782.300090 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95165.96 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 9516.60 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159298.895778 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85649.36 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 8564.94 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167863.831898 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77084.43 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 7708.44 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175572.274405 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69375.98 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 6937.60 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182509.872662 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62438.38 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 6243.84 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188753.711093 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56194.55 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 5619.45 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194373.165680 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50575.09 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 5057.51 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199430.674810 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45517.58 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 4551.76 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203982.433026 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40965.82 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 4096.58 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208079.015420 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36869.24 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3686.92 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211765.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33182.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215365.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29582.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218965.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25982.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222565.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22382.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226165.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18782.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229765.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15182.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233365.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11582.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236965.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7982.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240565.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4382.32 sec) 09:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244165.939575 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782.32 sec) 09:29:00 [GPUGRID] [rr_sim] 2244948.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987280845.79G/439.78G) 09:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852129.17 09:29:00 [---] pick_jobs_to_run() start 09:29:00 [---] pick_jobs_to_run() end 09:29:00 [---] [rr_sim] end 09:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:29:00 [---] [cpu_sched_debug] preliminary job list: 09:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:29:00 [---] [cpu_sched_debug] final job list: 09:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:29:00 [---] [cpu_sched_debug] enforce_run_list: end 09:29:00 [---] choose_project(): 34140.000000 09:29:00 [iThena.Measurements] doing RPC to get work 09:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401459.09 cpu 2147012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2147012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2147012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:29:00 [---] Got 0 tasks 09:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:30:00 [---] [rr_sim] doing sim: CPU sched 09:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987253255.43G/439.78G) 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244885.52 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 224488.55 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224488.552009 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020396.97 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 202039.70 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426528.248817 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818357.27 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 181835.73 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608363.975944 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636521.54 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 163652.15 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 772016.130359 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472869.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 147286.94 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919303.069332 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325582.45 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 132558.25 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051861.314407 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1193024.21 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 119302.42 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171163.734976 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073721.79 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 107372.18 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278535.913487 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966349.61 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 96634.96 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375170.874147 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869714.65 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 86971.46 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462142.338741 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782743.18 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 78274.32 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540416.656876 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704468.86 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 70446.89 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610863.543197 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634021.98 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 63402.20 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674265.740887 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570619.78 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 57061.98 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731327.718807 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513557.80 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 51355.78 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782683.498935 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462202.02 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 46220.20 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828903.701050 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415981.82 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 41598.18 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870501.882954 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374383.64 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 37438.36 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907940.246668 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336945.27 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 33694.53 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941634.774010 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303250.75 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 30325.07 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971959.848618 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272925.67 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 27292.57 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999252.415765 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245633.10 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 24563.31 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023815.726197 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221069.79 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 22106.98 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045922.705587 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198962.81 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 19896.28 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065818.987037 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179066.53 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 17906.65 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083725.640342 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161159.88 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 16115.99 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099841.628317 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145043.89 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 14504.39 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114346.017494 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130539.50 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 13053.95 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127399.967753 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117485.55 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 11748.56 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139148.522987 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105737.00 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 10573.70 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149722.222697 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95163.30 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 9516.33 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159238.552436 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85646.97 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 8564.70 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167803.249202 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77082.27 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 7708.23 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175511.476290 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69374.04 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 6937.40 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182448.880670 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62436.64 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 6243.66 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188692.544612 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56192.98 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 5619.30 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194311.842160 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50573.68 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 5057.37 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199369.209953 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45516.31 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 4551.63 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203920.840966 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40964.68 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 4096.47 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208017.308879 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36868.21 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3686.82 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211704.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33181.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215304.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29581.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218904.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25981.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222504.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22381.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226104.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18781.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229704.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15181.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233304.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11581.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236904.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7981.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240504.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4381.39 sec) 09:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244104.130000 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781.39 sec) 09:30:00 [GPUGRID] [rr_sim] 2244885.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987253255.43G/439.78G) 09:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852126.43 09:30:00 [---] pick_jobs_to_run() start 09:30:00 [---] pick_jobs_to_run() end 09:30:00 [---] [rr_sim] end 09:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:30:00 [---] [cpu_sched_debug] preliminary job list: 09:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:30:00 [---] [cpu_sched_debug] final job list: 09:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:30:00 [---] [cpu_sched_debug] enforce_run_list: end 09:30:00 [---] choose_project(): 34200.000000 09:30:00 [iThena.Measurements] doing RPC to get work 09:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401399.09 cpu 2146952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:30:00 [---] Got 0 tasks 09:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:31:00 [---] [rr_sim] doing sim: CPU sched 09:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987225665.07G/439.78G) 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244822.78 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 224482.28 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224482.278321 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020340.50 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 202034.05 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426516.328809 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818306.45 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 181830.65 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608346.974249 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636475.81 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 163647.58 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771994.555145 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472828.23 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 147282.82 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919277.377951 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325545.41 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 132554.54 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051831.918477 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192990.86 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 119299.09 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171131.004950 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073691.78 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 107369.18 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278500.182775 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966322.60 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 96632.26 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375132.442819 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869690.34 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 86969.03 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462101.476857 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782721.31 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 78272.13 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540373.607492 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704449.18 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 70444.92 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610818.525064 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634004.26 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 63400.43 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674218.950878 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570603.83 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 57060.38 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731279.334111 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513543.45 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 51354.34 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782633.679021 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462189.10 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 46218.91 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828852.589439 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415970.19 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 41597.02 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870449.608816 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374373.17 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 37437.32 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907886.926255 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336935.86 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 33693.59 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941580.511950 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303242.27 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 30324.23 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971904.739076 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272918.04 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 27291.80 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999196.543489 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245626.24 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 24562.62 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023759.167461 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221063.62 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 22106.36 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045865.529035 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198957.25 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 19895.73 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065761.254453 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179061.53 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 17906.15 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083667.407328 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161155.38 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 16115.54 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099782.944916 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145039.84 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 14503.98 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114286.928745 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130535.85 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 13053.59 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127340.514191 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117482.27 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 11748.23 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139088.741093 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105734.04 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 10573.40 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149662.145304 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95160.64 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 9516.06 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159178.209094 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85644.57 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 8564.46 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167742.666506 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77080.12 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 7708.01 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175450.678176 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69372.11 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 6937.21 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182387.888679 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62434.89 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 6243.49 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188631.378132 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56191.41 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 5619.14 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194250.518639 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50572.26 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 5057.23 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199307.745096 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45515.04 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 4551.50 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203859.248907 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40963.53 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 4096.35 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207955.602337 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36867.18 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3686.72 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211642.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33180.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215242.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29580.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218842.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25980.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222442.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22380.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226042.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18780.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229642.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15180.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233242.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11580.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236842.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7980.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240442.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4380.46 sec) 09:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2244042.320424 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780.46 sec) 09:31:00 [GPUGRID] [rr_sim] 2244822.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987225665.07G/439.78G) 09:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852123.70 09:31:00 [---] pick_jobs_to_run() start 09:31:00 [---] pick_jobs_to_run() end 09:31:00 [---] [rr_sim] end 09:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:31:00 [---] [cpu_sched_debug] preliminary job list: 09:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:31:00 [---] [cpu_sched_debug] final job list: 09:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:31:00 [---] [cpu_sched_debug] enforce_run_list: end 09:31:00 [---] choose_project(): 34260.000000 09:31:00 [iThena.Measurements] doing RPC to get work 09:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401339.09 cpu 2146892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:31:00 [---] Got 0 tasks 09:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:32:00 [---] [rr_sim] doing sim: CPU sched 09:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987198074.72G/439.78G) 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244760.05 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 224476.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224476.004632 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020284.04 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 202028.40 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426504.408802 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818255.64 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 181825.56 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608329.972554 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636430.07 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 163643.01 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771972.979931 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472787.07 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 147278.71 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919251.686570 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325508.36 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 132550.84 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051802.522546 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192957.52 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 119295.75 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171098.274924 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073661.77 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 107366.18 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278464.452064 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966295.59 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 96629.56 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375094.011490 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869666.03 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 86966.60 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462060.614973 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782699.43 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 78269.94 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540330.558109 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704429.49 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 70442.95 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610773.506930 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633986.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 63398.65 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674172.160870 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570587.89 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 57058.79 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731230.949415 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513529.10 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 51352.91 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782583.859106 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462176.19 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 46217.62 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828801.477828 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415958.57 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 41595.86 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870397.334678 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374362.71 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 37436.27 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907833.605842 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336926.44 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 33692.64 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941526.249891 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303233.80 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 30323.38 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971849.629534 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272910.42 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 27291.04 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999140.671213 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245619.38 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 24561.94 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023702.608724 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221057.44 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 22105.74 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045808.352484 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198951.69 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 19895.17 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065703.521868 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179056.52 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 17905.65 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083609.174314 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161150.87 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 16115.09 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099724.261515 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145035.78 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 14503.58 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114227.839996 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130532.21 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 13053.22 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127281.060629 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117478.99 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 11747.90 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139028.959198 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105731.09 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 10573.11 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149602.067911 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95157.98 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 9515.80 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159117.865752 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85642.18 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 8564.22 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167682.083810 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77077.96 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 7707.80 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175389.880061 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69370.17 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 6937.02 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182326.896688 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62433.15 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 6243.31 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188570.211651 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56189.83 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 5618.98 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194189.195119 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50570.85 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 5057.09 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199246.280239 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45513.77 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 4551.38 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203797.656848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40962.39 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 4096.24 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207893.895795 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36866.15 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3686.62 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211580.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33179.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215180.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29579.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218780.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25979.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222380.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22379.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225980.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18779.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229580.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15179.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233180.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11579.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236780.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7979.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240380.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4379.54 sec) 09:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243980.510848 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779.54 sec) 09:32:00 [GPUGRID] [rr_sim] 2244760.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987198074.72G/439.78G) 09:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852120.96 09:32:00 [---] pick_jobs_to_run() start 09:32:00 [---] pick_jobs_to_run() end 09:32:00 [---] [rr_sim] end 09:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:32:00 [---] [cpu_sched_debug] preliminary job list: 09:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:32:00 [---] [cpu_sched_debug] final job list: 09:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:32:00 [---] [cpu_sched_debug] enforce_run_list: end 09:32:00 [---] choose_project(): 34320.000000 09:32:00 [iThena.Measurements] doing RPC to get work 09:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401279.09 cpu 2146832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:32:00 [---] Got 0 tasks 09:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:33:00 [---] [rr_sim] doing sim: CPU sched 09:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987170484.36G/439.78G) 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244697.31 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 224469.73 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224469.730944 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020227.58 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 202022.76 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426492.488794 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818204.82 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 181820.48 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608312.970859 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636384.34 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 163638.43 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771951.404717 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472745.90 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 147274.59 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919225.995190 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325471.31 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 132547.13 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051773.126615 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192924.18 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 119292.42 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171065.544898 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073631.76 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 107363.18 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278428.721352 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966268.59 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 96626.86 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375055.580161 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869641.73 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 86964.17 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1462019.753089 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782677.56 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 78267.76 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540287.508725 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704409.80 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 70440.98 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610728.488797 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633968.82 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 63396.88 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674125.370861 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570571.94 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 57057.19 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731182.564719 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513514.74 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 51351.47 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782534.039192 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462163.27 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 46216.33 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828750.366217 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415946.94 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 41594.69 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870345.060539 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374352.25 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 37435.22 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907780.285430 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336917.02 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 33691.70 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941471.987831 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303225.32 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 30322.53 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971794.519992 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272902.79 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 27290.28 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999084.798937 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245612.51 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 24561.25 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023646.049988 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221051.26 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 22105.13 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045751.175933 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198946.13 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 19894.61 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065645.789284 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179051.52 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 17905.15 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083550.941300 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161146.37 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 16114.64 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099665.578114 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145031.73 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 14503.17 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114168.751247 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130528.56 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 13052.86 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127221.607067 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117475.70 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 11747.57 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138969.177304 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105728.13 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 10572.81 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149541.990518 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95155.32 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 9515.53 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159057.522410 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85639.79 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 8563.98 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167621.501114 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77075.81 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 7707.58 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175329.081947 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69368.23 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 6936.82 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182265.904696 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62431.40 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 6243.14 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188509.045171 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56188.26 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 5618.83 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194127.871598 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50569.44 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 5056.94 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199184.815382 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45512.49 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 4551.25 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203736.064788 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40961.24 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 4096.12 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207832.189254 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36865.12 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3686.51 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211518.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33178.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215118.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29578.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218718.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25978.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222318.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22378.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225918.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18778.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229518.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15178.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233118.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11578.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236718.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7978.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240318.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4378.61 sec) 09:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243918.701273 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778.61 sec) 09:33:00 [GPUGRID] [rr_sim] 2244697.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987170484.36G/439.78G) 09:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852118.22 09:33:00 [---] pick_jobs_to_run() start 09:33:00 [---] pick_jobs_to_run() end 09:33:00 [---] [rr_sim] end 09:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:33:00 [---] [cpu_sched_debug] preliminary job list: 09:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:33:00 [---] [cpu_sched_debug] final job list: 09:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:33:00 [---] [cpu_sched_debug] enforce_run_list: end 09:33:00 [---] choose_project(): 34380.000000 09:33:00 [iThena.Measurements] doing RPC to get work 09:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 574019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401219.09 cpu 2146772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:33:00 [---] Got 0 tasks 09:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:35:00 [---] [rr_sim] doing sim: CPU sched 09:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987142894.01G/439.78G) 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244634.57 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 224463.46 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224463.457256 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020171.12 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 202017.11 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426480.568786 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818154.00 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 181815.40 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608295.969164 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636338.60 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 163633.86 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771929.829504 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472704.74 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 147270.47 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919200.303809 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325434.27 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 132543.43 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051743.730684 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192890.84 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 119289.08 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171032.814872 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073601.76 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 107360.18 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278392.990641 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966241.58 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 96624.16 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1375017.148833 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869617.42 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 86961.74 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461978.891206 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782655.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 78265.57 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540244.459341 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704390.11 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 70439.01 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610683.470663 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633951.10 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 63395.11 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674078.580853 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570555.99 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 57055.60 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731134.180023 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513500.39 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 51350.04 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782484.219277 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462150.35 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 46215.04 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828699.254605 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415935.32 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 41593.53 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870292.786401 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374341.79 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 37434.18 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907726.965017 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336907.61 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 33690.76 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941417.725771 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303216.85 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 30321.68 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971739.410450 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272895.16 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 27289.52 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999028.926661 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245605.65 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 24560.56 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023589.491251 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221045.08 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 22104.51 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045693.999382 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198940.57 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 19894.06 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065588.056700 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179046.52 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 17904.65 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083492.708286 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161141.86 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 16114.19 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099606.894713 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145027.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 14502.77 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114109.662498 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130524.91 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 13052.49 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127162.153504 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117472.42 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 11747.24 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138909.395410 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105725.18 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 10572.52 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149481.913125 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95152.66 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 9515.27 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158997.179069 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85637.39 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 8563.74 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167560.918418 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77073.65 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 7707.37 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175268.283832 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69366.29 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 6936.63 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182204.912705 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62429.66 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 6242.97 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188447.878690 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56186.69 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 5618.67 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194066.548077 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50568.02 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 5056.80 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199123.350526 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45511.22 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 4551.12 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203674.472729 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40960.10 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 4096.01 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207770.482712 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36864.09 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3686.41 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211456.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33177.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215056.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29577.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218656.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25977.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222256.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22377.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225856.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18777.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229456.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15177.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233056.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11577.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236656.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7977.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240256.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4377.68 sec) 09:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243856.891697 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777.68 sec) 09:35:00 [GPUGRID] [rr_sim] 2244634.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987142894.01G/439.78G) 09:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852175.49 09:35:00 [---] pick_jobs_to_run() start 09:35:00 [---] pick_jobs_to_run() end 09:35:00 [---] [rr_sim] end 09:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:35:00 [---] [cpu_sched_debug] preliminary job list: 09:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:35:00 [---] [cpu_sched_debug] final job list: 09:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:35:00 [---] [cpu_sched_debug] enforce_run_list: end 09:35:00 [---] choose_project(): 34500.000000 09:35:00 [iThena.Measurements] doing RPC to get work 09:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401099.09 cpu 2146712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:35:00 [---] Got 0 tasks 09:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:36:00 [---] [rr_sim] doing sim: CPU sched 09:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987115303.65G/439.78G) 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244571.84 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 224457.18 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224457.183568 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020114.65 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 202011.47 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426468.648779 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818103.19 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 181810.32 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608278.967469 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636292.87 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 163629.29 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771908.254290 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472663.58 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 147266.36 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919174.612429 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325397.22 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 132539.72 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051714.334754 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192857.50 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 119285.75 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1171000.084846 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073571.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 107357.18 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278357.259929 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966214.58 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 96621.46 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374978.717504 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869593.12 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 86959.31 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461938.029322 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782633.81 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 78263.38 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540201.409957 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704370.43 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 70437.04 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610638.452529 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633933.38 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 63393.34 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1674031.790844 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570540.04 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 57054.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731085.795328 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513486.04 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 51348.60 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782434.399363 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462137.44 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 46213.74 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828648.142994 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415923.69 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 41592.37 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870240.512263 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374331.32 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 37433.13 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907673.644604 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336898.19 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 33689.82 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941363.463712 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303208.37 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 30320.84 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971684.300908 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272887.53 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 27288.75 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998973.054385 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245598.78 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 24559.88 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023532.932515 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221038.90 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 22103.89 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045636.822831 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198935.01 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 19893.50 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065530.324116 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179041.51 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 17904.15 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083434.475272 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161137.36 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 16113.74 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099548.211313 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145023.62 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 14502.36 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114050.573749 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130521.26 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 13052.13 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127102.699942 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117469.14 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 11746.91 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138849.613516 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105722.22 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 10572.22 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149421.835732 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95150.00 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 9515.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158936.835727 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85635.00 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 8563.50 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167500.335722 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77071.50 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 7707.15 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175207.485717 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69364.35 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 6936.43 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182143.920713 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62427.91 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 6242.79 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188386.712210 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56185.12 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 5618.51 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194005.224557 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50566.61 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 5056.66 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199061.885669 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45509.95 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 4551.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203612.880670 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40958.96 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 4095.90 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207708.776171 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36863.06 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3686.31 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211395.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33176.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214995.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29576.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218595.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25976.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222195.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22376.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225795.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18776.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229395.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15176.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232995.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11576.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236595.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7976.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240195.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4376.75 sec) 09:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243795.082121 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776.75 sec) 09:36:00 [GPUGRID] [rr_sim] 2244571.84: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987115303.65G/439.78G) 09:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852172.75 09:36:00 [---] pick_jobs_to_run() start 09:36:00 [---] pick_jobs_to_run() end 09:36:00 [---] [rr_sim] end 09:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:36:00 [---] [cpu_sched_debug] preliminary job list: 09:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:36:00 [---] [cpu_sched_debug] final job list: 09:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:36:00 [---] [cpu_sched_debug] enforce_run_list: end 09:36:00 [---] choose_project(): 34560.000000 09:36:00 [iThena.Measurements] doing RPC to get work 09:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 401039.09 cpu 2146652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:36:00 [---] Got 0 tasks 09:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:37:00 [---] [rr_sim] doing sim: CPU sched 09:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987087713.29G/439.78G) 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244509.10 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 224450.91 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224450.909880 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020058.19 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 202005.82 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426456.728771 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818052.37 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 181805.24 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608261.965774 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636247.13 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 163624.71 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771886.679076 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472622.42 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 147262.24 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919148.921048 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325360.18 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 132536.02 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051684.938823 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192824.16 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 119282.42 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170967.354820 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073541.74 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 107354.17 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278321.529218 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966187.57 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 96618.76 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374940.286176 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869568.81 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 86956.88 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461897.167438 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782611.93 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 78261.19 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540158.360573 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704350.74 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 70435.07 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610593.434396 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633915.66 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 63391.57 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673985.000836 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570524.10 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 57052.41 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1731037.410632 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513471.69 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 51347.17 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782384.579448 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462124.52 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 46212.45 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828597.031383 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415912.07 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 41591.21 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870188.238124 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374320.86 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 37432.09 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907620.324191 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336888.77 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 33688.88 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941309.201652 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303199.90 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 30319.99 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971629.191366 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272879.91 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 27287.99 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998917.182109 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245591.92 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 24559.19 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023476.373778 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221032.73 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 22103.27 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045579.646280 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198929.45 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 19892.95 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065472.591531 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179036.51 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 17903.65 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083376.242258 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161132.86 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 16113.29 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099489.527912 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145019.57 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 14501.96 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113991.485000 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130517.61 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 13051.76 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127043.246380 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117465.85 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 11746.59 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138789.831621 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105719.27 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 10571.93 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149361.758339 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95147.34 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 9514.73 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158876.492385 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85632.61 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 8563.26 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167439.753026 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77069.35 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 7706.93 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175146.687603 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69362.41 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 6936.24 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182082.928722 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62426.17 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 6242.62 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188325.545729 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56183.55 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 5618.36 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193943.901036 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50565.20 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 5056.52 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199000.420812 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45508.68 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 4550.87 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203551.288611 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40957.81 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 4095.78 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207647.069629 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36862.03 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3686.20 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211333.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33175.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214933.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29575.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218533.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25975.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222133.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22375.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225733.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18775.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229333.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15175.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232933.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11575.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236533.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7975.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240133.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4375.83 sec) 09:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243733.272546 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775.83 sec) 09:37:00 [GPUGRID] [rr_sim] 2244509.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987087713.29G/439.78G) 09:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852170.01 09:37:00 [---] pick_jobs_to_run() start 09:37:00 [---] pick_jobs_to_run() end 09:37:00 [---] [rr_sim] end 09:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:37:00 [---] [cpu_sched_debug] preliminary job list: 09:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:37:00 [---] [cpu_sched_debug] final job list: 09:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:37:00 [---] [cpu_sched_debug] enforce_run_list: end 09:37:00 [---] choose_project(): 34620.000000 09:37:00 [iThena.Measurements] doing RPC to get work 09:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400979.09 cpu 2146592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:37:00 [---] Got 0 tasks 09:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:38:00 [---] [rr_sim] doing sim: CPU sched 09:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987060122.94G/439.78G) 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244446.36 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 224444.64 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224444.636191 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2020001.73 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 202000.17 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426444.808764 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1818001.55 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 181800.16 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608244.964079 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636201.40 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 163620.14 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771865.103862 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472581.26 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 147258.13 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919123.229667 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325323.13 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 132532.31 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051655.542892 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192790.82 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 119279.08 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170934.624794 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073511.74 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 107351.17 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278285.798506 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966160.56 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 96616.06 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374901.854847 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869544.51 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 86954.45 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461856.305554 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782590.06 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 78259.01 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540115.311190 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704331.05 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 70433.11 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610548.416262 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633897.95 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 63389.79 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673938.210827 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570508.15 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 57050.82 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730989.025936 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513457.34 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 51345.73 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782334.759534 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462111.60 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 46211.16 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828545.919772 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415900.44 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 41590.04 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870135.963986 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374310.40 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 37431.04 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907567.003779 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336879.36 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 33687.94 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941254.939592 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303191.42 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 30319.14 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971574.081824 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272872.28 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 27287.23 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998861.309833 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245585.05 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 24558.51 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023419.815041 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221026.55 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 22102.65 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045522.469729 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198923.89 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 19892.39 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065414.858947 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179031.50 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 17903.15 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083318.009244 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161128.35 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 16112.84 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099430.844511 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145015.52 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 14501.55 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113932.396251 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130513.97 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 13051.40 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126983.792817 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117462.57 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 11746.26 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138730.049727 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105716.31 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 10571.63 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149301.680946 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95144.68 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 9514.47 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158816.149043 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85630.21 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 8563.02 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167379.170330 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77067.19 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 7706.72 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175085.889488 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69360.47 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 6936.05 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182021.936731 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62424.43 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 6242.44 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188264.379249 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56181.98 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 5618.20 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193882.577516 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50563.78 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 5056.38 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198938.955955 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45507.41 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 4550.74 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203489.696551 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40956.67 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 4095.67 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207585.363087 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36861.00 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3686.10 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211271.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33174.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214871.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29574.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218471.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25974.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222071.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22374.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225671.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18774.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229271.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15174.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232871.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11574.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236471.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7974.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240071.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4374.90 sec) 09:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243671.462970 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774.90 sec) 09:38:00 [GPUGRID] [rr_sim] 2244446.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987060122.94G/439.78G) 09:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852167.27 09:38:00 [---] pick_jobs_to_run() start 09:38:00 [---] pick_jobs_to_run() end 09:38:00 [---] [rr_sim] end 09:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:38:00 [---] [cpu_sched_debug] preliminary job list: 09:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:38:00 [---] [cpu_sched_debug] final job list: 09:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:38:00 [---] [cpu_sched_debug] enforce_run_list: end 09:38:00 [---] choose_project(): 34680.000000 09:38:00 [iThena.Measurements] doing RPC to get work 09:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400919.09 cpu 2146532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:38:00 [---] Got 0 tasks 09:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:39:00 [---] [rr_sim] doing sim: CPU sched 09:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987032532.58G/439.78G) 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244383.63 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 224438.36 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224438.362503 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019945.26 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 201994.53 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426432.888756 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817950.74 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 181795.07 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608227.962384 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636155.66 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 163615.57 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771843.528648 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472540.10 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 147254.01 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919097.538287 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325286.09 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 132528.61 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051626.146961 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192757.48 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 119275.75 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170901.894768 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073481.73 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 107348.17 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278250.067795 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966133.56 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 96613.36 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374863.423518 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869520.20 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 86952.02 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461815.443670 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782568.18 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 78256.82 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540072.261806 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704311.36 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 70431.14 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610503.398128 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633880.23 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 63388.02 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673891.420819 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570492.20 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 57049.22 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730940.641240 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513442.98 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 51344.30 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782284.939619 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462098.69 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 46209.87 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828494.808161 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415888.82 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 41588.88 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870083.689848 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374299.94 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 37429.99 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907513.683366 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336869.94 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 33686.99 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941200.677533 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303182.95 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 30318.29 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971518.972283 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272864.65 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 27286.47 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998805.437557 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245578.19 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 24557.82 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023363.256305 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221020.37 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 22102.04 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045465.293178 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198918.33 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 19891.83 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065357.126363 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179026.50 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 17902.65 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083259.776230 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161123.85 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 16112.38 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099372.161110 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145011.46 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 14501.15 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113873.307502 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130510.32 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 13051.03 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126924.339255 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117459.29 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 11745.93 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138670.267833 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105713.36 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 10571.34 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149241.603553 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95142.02 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 9514.20 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158755.805701 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85627.82 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 8562.78 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167318.587634 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77065.04 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 7706.50 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175025.091374 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69358.53 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 6935.85 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181960.944739 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62422.68 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 6242.27 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188203.212769 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56180.41 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 5618.04 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193821.253995 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50562.37 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 5056.24 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198877.491099 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45506.13 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 4550.61 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203428.104492 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40955.52 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 4095.55 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207523.656546 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36859.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3686.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211209.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33173.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214809.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29573.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218409.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25973.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222009.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22373.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225609.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18773.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229209.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15173.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232809.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11573.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236409.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7973.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240009.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4373.97 sec) 09:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243609.653394 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773.97 sec) 09:39:00 [GPUGRID] [rr_sim] 2244383.63: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987032532.58G/439.78G) 09:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852164.54 09:39:00 [---] pick_jobs_to_run() start 09:39:00 [---] pick_jobs_to_run() end 09:39:00 [---] [rr_sim] end 09:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:39:00 [---] [cpu_sched_debug] preliminary job list: 09:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:39:00 [---] [cpu_sched_debug] final job list: 09:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:39:00 [---] [cpu_sched_debug] enforce_run_list: end 09:39:00 [---] choose_project(): 34740.000000 09:39:00 [iThena.Measurements] doing RPC to get work 09:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400859.09 cpu 2146472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:39:00 [---] Got 0 tasks 09:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:40:00 [---] [rr_sim] doing sim: CPU sched 09:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (987004942.22G/439.78G) 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244320.89 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 224432.09 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224432.088815 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019888.80 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 201988.88 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426420.968748 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817899.92 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 181789.99 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608210.960689 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636109.93 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 163610.99 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771821.953435 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472498.93 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 147249.89 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919071.846906 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325249.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 132524.90 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051596.751030 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192724.14 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 119272.41 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170869.164742 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073451.72 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 107345.17 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278214.337083 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966106.55 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 96610.66 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374824.992190 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869495.90 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 86949.59 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461774.581786 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782546.31 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 78254.63 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1540029.212422 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704291.68 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 70429.17 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610458.379995 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633862.51 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 63386.25 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673844.630810 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570476.26 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 57047.63 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730892.256544 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513428.63 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 51342.86 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782235.119705 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462085.77 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 46208.58 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828443.696549 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415877.19 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 41587.72 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870031.415709 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374289.47 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 37428.95 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907460.362953 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336860.53 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 33686.05 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941146.415473 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303174.47 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 30317.45 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971463.862741 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272857.03 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 27285.70 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998749.565282 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245571.32 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 24557.13 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023306.697568 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221014.19 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 22101.42 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045408.116626 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198912.77 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 19891.28 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065299.393779 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179021.49 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 17902.15 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083201.543216 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161119.34 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 16111.93 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099313.477709 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145007.41 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 14500.74 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113814.218753 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130506.67 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 13050.67 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126864.885693 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117456.00 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 11745.60 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138610.485939 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105710.40 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 10571.04 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149181.526160 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95139.36 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 9513.94 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158695.462359 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85625.43 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 8562.54 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167258.004938 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77062.88 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 7706.29 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174964.293259 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69356.59 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 6935.66 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181899.952748 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62420.94 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 6242.09 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188142.046288 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56178.84 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 5617.88 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193759.930474 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50560.96 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 5056.10 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198816.026242 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45504.86 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 4550.49 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203366.512433 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40954.38 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 4095.44 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207461.950004 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36858.94 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3685.89 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211147.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33173.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214747.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29573.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218347.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25973.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221947.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22373.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225547.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18773.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229147.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15173.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232747.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11573.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236347.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7973.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239947.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4373.04 sec) 09:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243547.843819 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773.04 sec) 09:40:00 [GPUGRID] [rr_sim] 2244320.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (987004942.22G/439.78G) 09:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852161.80 09:40:00 [---] pick_jobs_to_run() start 09:40:00 [---] pick_jobs_to_run() end 09:40:00 [---] [rr_sim] end 09:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:40:00 [---] [cpu_sched_debug] preliminary job list: 09:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:40:00 [---] [cpu_sched_debug] final job list: 09:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:40:00 [---] [cpu_sched_debug] enforce_run_list: end 09:40:00 [---] choose_project(): 34800.000000 09:40:00 [iThena.Measurements] doing RPC to get work 09:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400799.09 cpu 2146412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:40:00 [---] Got 0 tasks 09:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:41:00 [---] [rr_sim] doing sim: CPU sched 09:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986977351.87G/439.78G) 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244258.15 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 224425.82 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224425.815127 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019832.34 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 201983.23 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426409.048741 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817849.10 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 181784.91 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608193.958993 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636064.19 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 163606.42 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771800.378221 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472457.77 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 147245.78 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919046.155526 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325212.00 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 132521.20 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051567.355100 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192690.80 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 119269.08 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170836.434716 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073421.72 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 107342.17 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278178.606372 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966079.54 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 96607.95 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374786.560861 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869471.59 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 86947.16 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461733.719902 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782524.43 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 78252.44 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539986.163038 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704271.99 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 70427.20 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610413.361861 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633844.79 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 63384.48 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673797.840802 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570460.31 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 57046.03 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730843.871848 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513414.28 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 51341.43 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782185.299790 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462072.85 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 46207.29 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828392.584938 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415865.57 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 41586.56 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869979.141571 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374279.01 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 37427.90 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907407.042541 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336851.11 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 33685.11 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941092.153413 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303166.00 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 30316.60 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971408.753199 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272849.40 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 27284.94 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998693.693006 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245564.46 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 24556.45 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023250.138832 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221008.01 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 22100.80 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045350.940075 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198907.21 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 19890.72 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065241.661195 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179016.49 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 17901.65 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083143.310202 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161114.84 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 16111.48 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099254.794308 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145003.36 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 14500.34 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113755.130004 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130503.02 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 13050.30 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126805.432131 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117452.72 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 11745.27 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138550.704044 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105707.45 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 10570.74 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149121.448767 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95136.70 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 9513.67 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158635.119017 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85623.03 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 8562.30 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167197.422242 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77060.73 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 7706.07 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174903.495144 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69354.66 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 6935.47 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181838.960757 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62419.19 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 6241.92 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188080.879808 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56177.27 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 5617.73 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193698.606954 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50559.54 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 5055.95 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198754.561385 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45503.59 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 4550.36 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203304.920373 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40953.23 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 4095.32 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207400.243463 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36857.91 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3685.79 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211086.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33172.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214686.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29572.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218286.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25972.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221886.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22372.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225486.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18772.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229086.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15172.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232686.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11572.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236286.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7972.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239886.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4372.12 sec) 09:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243486.034243 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772.12 sec) 09:41:00 [GPUGRID] [rr_sim] 2244258.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986977351.87G/439.78G) 09:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852159.06 09:41:00 [---] pick_jobs_to_run() start 09:41:00 [---] pick_jobs_to_run() end 09:41:00 [---] [rr_sim] end 09:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:41:00 [---] [cpu_sched_debug] preliminary job list: 09:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:41:00 [---] [cpu_sched_debug] final job list: 09:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:41:00 [---] [cpu_sched_debug] enforce_run_list: end 09:41:00 [---] choose_project(): 34860.000000 09:41:00 [iThena.Measurements] doing RPC to get work 09:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400739.09 cpu 2146352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:41:00 [---] Got 0 tasks 09:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:42:00 [---] [rr_sim] doing sim: CPU sched 09:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986949761.51G/439.78G) 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244195.41 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 224419.54 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224419.541439 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019775.87 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 201977.59 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426397.128733 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817798.29 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 181779.83 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608176.957298 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1636018.46 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 163601.85 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771778.803007 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472416.61 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 147241.66 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 919020.464145 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325174.95 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 132517.50 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051537.959169 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192657.46 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 119265.75 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170803.704691 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073391.71 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 107339.17 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278142.875660 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966052.54 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 96605.25 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374748.129533 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869447.28 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 86944.73 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461692.858018 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782502.56 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 78250.26 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539943.113655 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704252.30 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 70425.23 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610368.343728 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633827.07 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 63382.71 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673751.050793 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570444.36 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 57044.44 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730795.487153 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513399.93 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 51339.99 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782135.479876 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462059.93 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 46205.99 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828341.473327 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415853.94 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 41585.39 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869926.867433 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374268.55 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 37426.85 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907353.722128 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336841.69 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 33684.17 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941037.891354 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303157.52 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 30315.75 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971353.643657 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272841.77 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 27284.18 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998637.820730 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245557.59 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 24555.76 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023193.580095 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221001.83 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 22100.18 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045293.763524 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198901.65 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 19890.17 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065183.928610 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179011.49 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 17901.15 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083085.077188 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161110.34 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 16111.03 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099196.110908 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144999.30 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 14499.93 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113696.041255 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130499.37 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 13049.94 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126745.978568 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117449.44 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 11744.94 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138490.922150 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105704.49 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 10570.45 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149061.371374 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95134.04 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 9513.40 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158574.775675 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85620.64 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 8562.06 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167136.839546 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77058.57 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 7705.86 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174842.697030 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69352.72 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 6935.27 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181777.968765 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62417.45 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 6241.74 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188019.713327 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56175.70 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 5617.57 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193637.283433 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50558.13 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 5055.81 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198693.096528 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45502.32 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 4550.23 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203243.328314 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40952.09 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 4095.21 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207338.536921 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36856.88 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3685.69 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211024.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33171.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214624.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29571.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218224.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25971.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221824.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22371.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225424.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18771.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229024.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15171.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232624.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11571.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236224.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7971.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239824.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4371.19 sec) 09:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243424.224668 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771.19 sec) 09:42:00 [GPUGRID] [rr_sim] 2244195.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986949761.51G/439.78G) 09:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852156.33 09:42:00 [---] pick_jobs_to_run() start 09:42:00 [---] pick_jobs_to_run() end 09:42:00 [---] [rr_sim] end 09:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:42:00 [---] [cpu_sched_debug] preliminary job list: 09:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:42:00 [---] [cpu_sched_debug] final job list: 09:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:42:00 [---] [cpu_sched_debug] enforce_run_list: end 09:42:00 [---] choose_project(): 34920.000000 09:42:00 [iThena.Measurements] doing RPC to get work 09:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400679.09 cpu 2146292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:42:00 [---] Got 0 tasks 09:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:43:00 [---] [rr_sim] doing sim: CPU sched 09:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986922171.16G/439.78G) 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244132.68 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 224413.27 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224413.267750 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019719.41 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 201971.94 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426385.208726 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817747.47 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 181774.75 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608159.955603 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635972.72 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 163597.27 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771757.227793 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472375.45 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 147237.54 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918994.772764 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325137.90 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 132513.79 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051508.563238 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192624.11 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 119262.41 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170770.974665 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073361.70 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 107336.17 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278107.144949 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 966025.53 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 96602.55 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374709.698204 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869422.98 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 86942.30 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461651.996134 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782480.68 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 78248.07 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539900.064271 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704232.61 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 70423.26 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610323.325594 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633809.35 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 63380.94 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673704.260785 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570428.42 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 57042.84 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730747.102457 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513385.58 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 51338.56 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782085.659961 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462047.02 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 46204.70 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828290.361716 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415842.32 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 41584.23 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869874.593294 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374258.08 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 37425.81 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907300.401715 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336832.28 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 33683.23 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940983.629294 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303149.05 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 30314.90 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971298.534115 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272834.14 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 27283.41 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998581.948454 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245550.73 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 24555.07 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023137.021359 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220995.66 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 22099.57 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045236.586973 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198896.09 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 19889.61 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065126.196026 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179006.48 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 17900.65 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083026.844174 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161105.83 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 16110.58 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099137.427507 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144995.25 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 14499.52 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113636.952506 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130495.72 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 13049.57 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126686.525006 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117446.15 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 11744.62 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138431.140256 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105701.54 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 10570.15 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149001.293980 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95131.38 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 9513.14 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158514.432333 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85618.25 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 8561.82 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167076.256850 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77056.42 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 7705.64 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174781.898915 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69350.78 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 6935.08 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181716.976774 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62415.70 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 6241.57 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187958.546847 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56174.13 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 5617.41 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193575.959912 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50556.72 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 5055.67 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198631.631672 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45501.05 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 4550.10 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203181.736255 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40950.94 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 4095.09 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207276.830380 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36855.85 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3685.58 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210962.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33170.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214562.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29570.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218162.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25970.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221762.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22370.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225362.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18770.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228962.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15170.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232562.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11570.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236162.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7970.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239762.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4370.26 sec) 09:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243362.415092 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770.26 sec) 09:43:00 [GPUGRID] [rr_sim] 2244132.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986922171.16G/439.78G) 09:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852153.59 09:43:00 [---] pick_jobs_to_run() start 09:43:00 [---] pick_jobs_to_run() end 09:43:00 [---] [rr_sim] end 09:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:43:00 [---] [cpu_sched_debug] preliminary job list: 09:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:43:00 [---] [cpu_sched_debug] final job list: 09:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:43:00 [---] [cpu_sched_debug] enforce_run_list: end 09:43:00 [---] choose_project(): 34980.000000 09:43:00 [iThena.Measurements] doing RPC to get work 09:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400619.09 cpu 2146232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:43:00 [---] Got 0 tasks 09:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:44:00 [---] [rr_sim] doing sim: CPU sched 09:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986894580.80G/439.78G) 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244069.94 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 224406.99 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224406.994062 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019662.95 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 201966.29 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426373.288718 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817696.65 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 181769.67 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608142.953908 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635926.99 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 163592.70 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771735.652580 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472334.29 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 147233.43 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918969.081384 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325100.86 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 132510.09 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051479.167307 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192590.77 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 119259.08 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170738.244639 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073331.70 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 107333.17 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278071.414237 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965998.53 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 96599.85 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374671.266875 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869398.67 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 86939.87 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461611.134250 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782458.81 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 78245.88 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539857.014887 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704212.93 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 70421.29 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610278.307460 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633791.63 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 63379.16 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673657.470776 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570412.47 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 57041.25 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730698.717761 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513371.22 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 51337.12 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1782035.840047 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462034.10 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 46203.41 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828239.250104 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415830.69 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 41583.07 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869822.319156 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374247.62 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 37424.76 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907247.081302 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336822.86 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 33682.29 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940929.367234 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303140.57 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 30314.06 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971243.424573 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272826.52 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 27282.65 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998526.076178 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245543.86 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 24554.39 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023080.462622 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220989.48 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 22098.95 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045179.410422 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198890.53 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 19889.05 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065068.463442 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179001.48 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 17900.15 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082968.611160 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161101.33 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 16110.13 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099078.744106 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144991.20 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 14499.12 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113577.863757 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130492.08 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 13049.21 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126627.071444 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117442.87 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 11744.29 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138371.358361 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105698.58 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 10569.86 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148941.216587 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95128.72 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 9512.87 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158454.088991 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85615.85 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 8561.59 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167015.674154 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77054.27 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 7705.43 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174721.100800 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69348.84 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 6934.88 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181655.984783 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62413.96 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 6241.40 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187897.380366 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56172.56 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 5617.26 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193514.636392 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50555.30 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 5055.53 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198570.166815 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45499.77 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 4549.98 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203120.144195 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40949.80 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 4094.98 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207215.123838 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36854.82 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3685.48 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210900.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33169.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214500.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29569.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218100.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25969.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221700.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22369.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225300.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18769.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228900.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15169.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232500.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11569.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236100.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7969.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239700.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4369.34 sec) 09:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243300.605516 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769.34 sec) 09:44:00 [GPUGRID] [rr_sim] 2244069.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986894580.80G/439.78G) 09:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852150.85 09:44:00 [---] pick_jobs_to_run() start 09:44:00 [---] pick_jobs_to_run() end 09:44:00 [---] [rr_sim] end 09:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:44:00 [---] [cpu_sched_debug] preliminary job list: 09:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:44:00 [---] [cpu_sched_debug] final job list: 09:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:44:00 [---] [cpu_sched_debug] enforce_run_list: end 09:44:00 [---] choose_project(): 35040.000000 09:44:00 [iThena.Measurements] doing RPC to get work 09:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400559.09 cpu 2146172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:44:00 [---] Got 0 tasks 09:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:45:00 [---] [rr_sim] doing sim: CPU sched 09:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986866990.44G/439.78G) 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2244007.20 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 224400.72 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224400.720374 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019606.48 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 201960.65 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426361.368710 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817645.84 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 181764.58 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608125.952213 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635881.25 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 163588.13 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771714.077366 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472293.13 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 147229.31 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918943.390003 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325063.81 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 132506.38 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051449.771377 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192557.43 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 119255.74 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170705.514613 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073301.69 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 107330.17 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1278035.683525 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965971.52 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 96597.15 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374632.835547 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869374.37 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 86937.44 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461570.272366 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782436.93 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 78243.69 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539813.965503 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704193.24 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 70419.32 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610233.289327 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633773.91 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 63377.39 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673610.680768 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570396.52 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 57039.65 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730650.333065 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513356.87 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 51335.69 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781986.020132 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462021.18 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 46202.12 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828188.138493 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415819.07 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 41581.91 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869770.045018 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374237.16 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 37423.72 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907193.760890 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336813.44 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 33681.34 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940875.105175 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303132.10 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 30313.21 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971188.315031 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272818.89 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 27281.89 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998470.203902 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245537.00 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 24553.70 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023023.903886 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220983.30 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 22098.33 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045122.233871 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198884.97 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 19888.50 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065010.730858 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178996.47 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 17899.65 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082910.378146 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161096.83 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 16109.68 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099020.060705 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144987.14 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 14498.71 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113518.775008 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130488.43 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 13048.84 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126567.617881 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117439.59 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 11743.96 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138311.576467 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105695.63 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 10569.56 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148881.139194 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95126.06 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 9512.61 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158393.745649 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85613.46 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 8561.35 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166955.091458 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77052.11 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 7705.21 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174660.302686 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69346.90 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 6934.69 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181594.992791 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62412.21 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 6241.22 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187836.213886 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56170.99 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 5617.10 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193453.312871 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50553.89 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 5055.39 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198508.701958 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45498.50 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 4549.85 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203058.552136 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40948.65 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 4094.87 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207153.417296 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36853.79 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3685.38 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210838.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33168.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214438.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29568.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218038.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25968.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221638.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22368.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225238.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18768.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228838.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15168.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232438.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11568.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236038.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7968.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239638.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4368.41 sec) 09:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243238.795941 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768.41 sec) 09:45:00 [GPUGRID] [rr_sim] 2244007.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986866990.44G/439.78G) 09:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852148.12 09:45:00 [---] pick_jobs_to_run() start 09:45:00 [---] pick_jobs_to_run() end 09:45:00 [---] [rr_sim] end 09:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:45:00 [---] [cpu_sched_debug] preliminary job list: 09:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:45:00 [---] [cpu_sched_debug] final job list: 09:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:45:00 [---] [cpu_sched_debug] enforce_run_list: end 09:45:00 [---] choose_project(): 35100.000000 09:45:00 [iThena.Measurements] doing RPC to get work 09:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400499.09 cpu 2146112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:45:00 [---] Got 0 tasks 09:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:46:00 [---] [rr_sim] doing sim: CPU sched 09:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986839400.09G/439.78G) 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243944.47 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 224394.45 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224394.446686 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019550.02 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 201955.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426349.448703 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817595.02 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 181759.50 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608108.950518 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635835.52 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 163583.55 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771692.502152 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472251.96 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 147225.20 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918917.698622 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1325026.77 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 132502.68 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051420.375446 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192524.09 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 119252.41 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170672.784587 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073271.68 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 107327.17 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277999.952814 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965944.51 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 96594.45 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374594.404218 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869350.06 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 86935.01 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461529.410482 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782415.06 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 78241.51 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539770.916119 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704173.55 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 70417.36 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610188.271193 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633756.20 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 63375.62 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673563.890760 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570380.58 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 57038.06 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730601.948369 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513342.52 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 51334.25 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781936.200218 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462008.27 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 46200.83 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828137.026882 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415807.44 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 41580.74 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869717.770879 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374226.70 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 37422.67 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907140.440477 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336804.03 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 33680.40 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940820.843115 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303123.62 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 30312.36 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971133.205489 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272811.26 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 27281.13 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998414.331626 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245530.14 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 24553.01 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022967.345149 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220977.12 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 22097.71 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045065.057320 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198879.41 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 19887.94 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064952.998273 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178991.47 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 17899.15 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082852.145132 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161092.32 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 16109.23 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098961.377304 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144983.09 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 14498.31 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113459.686259 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130484.78 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 13048.48 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126508.164319 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117436.30 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 11743.63 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138251.794573 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105692.67 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 10569.27 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148821.061801 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95123.41 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 9512.34 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158333.402307 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85611.06 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 8561.11 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166894.508762 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77049.96 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 7705.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174599.504571 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69344.96 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 6934.50 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181534.000800 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62410.47 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 6241.05 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187775.047405 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56169.42 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 5616.94 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193391.989351 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50552.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 5055.25 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198447.237101 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45497.23 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 4549.72 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202996.960077 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40947.51 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 4094.75 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207091.710755 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36852.76 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3685.28 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210776.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33167.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214376.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29567.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217976.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25967.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221576.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22367.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225176.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18767.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228776.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15167.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232376.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11567.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235976.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7967.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239576.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4367.48 sec) 09:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243176.986365 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767.48 sec) 09:46:00 [GPUGRID] [rr_sim] 2243944.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986839400.09G/439.78G) 09:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852145.38 09:46:00 [---] pick_jobs_to_run() start 09:46:00 [---] pick_jobs_to_run() end 09:46:00 [---] [rr_sim] end 09:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:46:00 [---] [cpu_sched_debug] preliminary job list: 09:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:46:00 [---] [cpu_sched_debug] final job list: 09:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:46:00 [---] [cpu_sched_debug] enforce_run_list: end 09:46:00 [---] choose_project(): 35160.000000 09:46:00 [iThena.Measurements] doing RPC to get work 09:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400439.09 cpu 2146052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2146052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2146052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:46:00 [---] Got 0 tasks 09:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:47:00 [---] [rr_sim] doing sim: CPU sched 09:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986811809.73G/439.78G) 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243881.73 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 224388.17 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224388.172997 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019493.56 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 201949.36 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426337.528695 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817544.20 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 181754.42 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608091.948823 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635789.78 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 163578.98 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771670.926938 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472210.80 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 147221.08 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918892.007242 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324989.72 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 132498.97 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051390.979515 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192490.75 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 119249.08 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170640.054561 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073241.68 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 107324.17 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277964.222102 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965917.51 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 96591.75 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374555.972890 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869325.76 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 86932.58 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461488.548598 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782393.18 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 78239.32 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539727.866736 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704153.86 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 70415.39 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610143.253060 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633738.48 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 63373.85 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673517.100751 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570364.63 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 57036.46 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730553.563673 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513328.17 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 51332.82 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781886.380303 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461995.35 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 46199.53 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828085.915271 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415795.81 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 41579.58 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869665.496741 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374216.23 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 37421.62 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907087.120064 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336794.61 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 33679.46 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940766.581055 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303115.15 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 30311.51 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971078.095947 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272803.63 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 27280.36 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998358.459350 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245523.27 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 24552.33 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022910.786412 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220970.94 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 22097.09 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045007.880769 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198873.85 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 19887.38 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064895.265689 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178986.46 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 17898.65 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082793.912118 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161087.82 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 16108.78 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098902.693903 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144979.04 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 14497.90 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113400.597510 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130481.13 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 13048.11 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126448.710757 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117433.02 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 11743.30 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138192.012679 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105689.72 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 10568.97 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148760.984408 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95120.75 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 9512.07 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158273.058965 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85608.67 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 8560.87 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166833.926066 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77047.80 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 7704.78 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174538.706457 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69343.02 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 6934.30 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181473.008808 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62408.72 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 6240.87 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187713.880925 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56167.85 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 5616.78 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193330.665830 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50551.06 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 5055.11 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198385.772244 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45495.96 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 4549.60 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202935.368017 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40946.36 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 4094.64 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207030.004213 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36851.73 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3685.17 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210715.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33166.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214315.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29566.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217915.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25966.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221515.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22366.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225115.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18766.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228715.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15166.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232315.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11566.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235915.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7966.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239515.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4366.55 sec) 09:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243115.176789 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766.55 sec) 09:47:00 [GPUGRID] [rr_sim] 2243881.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986811809.73G/439.78G) 09:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852142.64 09:47:00 [---] pick_jobs_to_run() start 09:47:00 [---] pick_jobs_to_run() end 09:47:00 [---] [rr_sim] end 09:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:47:00 [---] [cpu_sched_debug] preliminary job list: 09:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:47:00 [---] [cpu_sched_debug] final job list: 09:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:47:00 [---] [cpu_sched_debug] enforce_run_list: end 09:47:00 [---] choose_project(): 35220.000000 09:47:00 [iThena.Measurements] doing RPC to get work 09:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400379.09 cpu 2145992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:47:00 [---] Got 0 tasks 09:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:48:00 [---] [rr_sim] doing sim: CPU sched 09:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986784219.37G/439.78G) 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243818.99 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 224381.90 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224381.899309 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019437.09 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 201943.71 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426325.608688 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817493.38 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 181749.34 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608074.947128 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635744.05 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 163574.40 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771649.351724 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472169.64 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 147216.96 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918866.315861 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324952.68 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 132495.27 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051361.583584 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192457.41 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 119245.74 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170607.324535 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073211.67 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 107321.17 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277928.491391 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965890.50 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 96589.05 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374517.541561 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869301.45 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 86930.15 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461447.686714 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782371.31 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 78237.13 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539684.817352 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704134.18 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 70413.42 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610098.234926 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633720.76 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 63372.08 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673470.310743 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570348.68 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 57034.87 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730505.178978 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513313.81 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 51331.38 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781836.560389 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461982.43 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 46198.24 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828034.803659 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415784.19 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 41578.42 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869613.222603 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374205.77 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 37420.58 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907033.799652 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336785.19 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 33678.52 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940712.318996 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303106.67 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 30310.67 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971022.986405 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272796.01 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 27279.60 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998302.587074 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245516.41 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 24551.64 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022854.227676 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220964.77 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 22096.48 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044950.704217 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198868.29 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 19886.83 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064837.533105 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178981.46 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 17898.15 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082735.679104 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161083.31 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 16108.33 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098844.010503 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144974.98 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 14497.50 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113341.508762 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130477.48 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 13047.75 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126389.257195 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117429.74 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 11742.97 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138132.230784 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105686.76 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 10568.68 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148700.907015 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95118.09 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 9511.81 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158212.715623 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85606.28 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 8560.63 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166773.343370 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77045.65 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 7704.56 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174477.908342 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69341.08 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 6934.11 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181412.016817 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62406.98 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 6240.70 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187652.714445 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56166.28 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 5616.63 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193269.342309 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50549.65 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 5054.97 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198324.307388 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45494.69 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 4549.47 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202873.775958 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40945.22 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 4094.52 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206968.297672 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36850.70 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3685.07 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210653.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33165.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214253.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29565.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217853.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25965.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221453.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22365.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225053.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18765.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228653.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15165.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232253.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11565.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235853.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7965.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239453.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4365.63 sec) 09:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2243053.367214 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765.63 sec) 09:48:00 [GPUGRID] [rr_sim] 2243818.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986784219.37G/439.78G) 09:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852139.91 09:48:00 [---] pick_jobs_to_run() start 09:48:00 [---] pick_jobs_to_run() end 09:48:00 [---] [rr_sim] end 09:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:48:00 [---] [cpu_sched_debug] preliminary job list: 09:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:48:00 [---] [cpu_sched_debug] final job list: 09:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:48:00 [---] [cpu_sched_debug] enforce_run_list: end 09:48:00 [---] choose_project(): 35280.000000 09:48:00 [iThena.Measurements] doing RPC to get work 09:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400319.09 cpu 2145932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:48:00 [---] Got 0 tasks 09:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:49:00 [---] [rr_sim] doing sim: CPU sched 09:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986756629.02G/439.78G) 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243756.26 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 224375.63 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224375.625621 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019380.63 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 201938.06 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426313.688680 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817442.57 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 181744.26 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608057.945433 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635698.31 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 163569.83 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771627.776511 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472128.48 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 147212.85 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918840.624481 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324915.63 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 132491.56 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051332.187654 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192424.07 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 119242.41 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170574.594509 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073181.66 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 107318.17 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277892.760679 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965863.50 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 96586.35 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374479.110232 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869277.15 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 86927.71 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461406.824830 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782349.43 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 78234.94 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539641.767968 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704114.49 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 70411.45 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610053.216792 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633703.04 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 63370.30 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673423.520734 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570332.74 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 57033.27 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730456.794282 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513299.46 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 51329.95 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781786.740475 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461969.52 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 46196.95 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827983.692048 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415772.56 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 41577.26 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869560.948464 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374195.31 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 37419.53 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906980.479239 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336775.78 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 33677.58 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940658.056936 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303098.20 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 30309.82 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970967.876863 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272788.38 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 27278.84 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998246.714798 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245509.54 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 24550.95 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022797.668939 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220958.59 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 22095.86 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044893.527666 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198862.73 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 19886.27 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064779.800521 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178976.46 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 17897.65 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082677.446090 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161078.81 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 16107.88 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098785.327102 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144970.93 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 14497.09 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113282.420013 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130473.84 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 13047.38 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126329.803632 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117426.45 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 11742.65 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138072.448890 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105683.81 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 10568.38 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148640.829622 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95115.43 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 9511.54 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158152.372281 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85603.88 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 8560.39 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166712.760674 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77043.50 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 7704.35 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174417.110227 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69339.15 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 6933.91 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181351.024826 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62405.23 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 6240.52 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187591.547964 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56164.71 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 5616.47 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193208.018789 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50548.24 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 5054.82 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198262.842531 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45493.41 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 4549.34 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202812.183899 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40944.07 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 4094.41 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206906.591130 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36849.67 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3684.97 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210591.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33164.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214191.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29564.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217791.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25964.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221391.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22364.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224991.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18764.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228591.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15164.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232191.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11564.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235791.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7964.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239391.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4364.70 sec) 09:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242991.557638 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764.70 sec) 09:49:00 [GPUGRID] [rr_sim] 2243756.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986756629.02G/439.78G) 09:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852137.17 09:49:00 [---] pick_jobs_to_run() start 09:49:00 [---] pick_jobs_to_run() end 09:49:00 [---] [rr_sim] end 09:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:49:00 [---] [cpu_sched_debug] preliminary job list: 09:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:49:00 [---] [cpu_sched_debug] final job list: 09:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:49:00 [---] [cpu_sched_debug] enforce_run_list: end 09:49:00 [---] choose_project(): 35340.000000 09:49:00 [iThena.Measurements] doing RPC to get work 09:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 573059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400259.09 cpu 2145872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:49:00 [---] Got 0 tasks 09:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:50:00 [---] [rr_sim] doing sim: CPU sched 09:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986729038.66G/439.78G) 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243693.52 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 224369.35 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224369.351933 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019324.17 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 201932.42 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426301.768672 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817391.75 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 181739.18 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608040.943738 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635652.58 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 163565.26 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771606.201297 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472087.32 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 147208.73 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918814.933100 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324878.59 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 132487.86 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051302.791723 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192390.73 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 119239.07 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170541.864483 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073151.65 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 107315.17 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277857.029968 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965836.49 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 96583.65 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374440.678904 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869252.84 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 86925.28 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461365.962946 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782327.56 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 78232.76 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539598.718584 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704094.80 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 70409.48 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1610008.198659 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633685.32 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 63368.53 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673376.730726 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570316.79 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 57031.68 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730408.409586 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513285.11 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 51328.51 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781736.920560 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461956.60 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 46195.66 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827932.580437 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415760.94 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 41576.09 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869508.674326 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374184.85 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 37418.48 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906927.158826 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336766.36 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 33676.64 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940603.794876 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303089.72 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 30308.97 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970912.767322 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272780.75 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 27278.08 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998190.842522 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245502.68 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 24550.27 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022741.110203 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220952.41 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 22095.24 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044836.351115 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198857.17 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 19885.72 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064722.067937 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178971.45 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 17897.15 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082619.213076 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161074.31 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 16107.43 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098726.643701 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144966.88 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 14496.69 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113223.331264 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130470.19 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 13047.02 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126270.350070 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117423.17 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 11742.32 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138012.666996 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105680.85 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 10568.09 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148580.752229 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95112.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 9511.28 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158092.028939 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85601.49 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 8560.15 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166652.177978 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77041.34 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 7704.13 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174356.312113 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69337.21 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 6933.72 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181290.032834 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62403.49 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 6240.35 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187530.381484 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56163.14 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 5616.31 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193146.695268 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50546.82 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 5054.68 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198201.377674 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45492.14 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 4549.21 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202750.591839 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40942.93 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 4094.29 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206844.884588 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36848.63 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3684.86 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210529.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33163.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214129.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29563.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217729.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25963.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221329.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22363.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224929.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18763.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228529.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15163.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232129.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11563.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235729.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7963.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239329.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4363.77 sec) 09:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242929.748062 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 763.77 sec) 09:50:00 [GPUGRID] [rr_sim] 2243693.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986729038.66G/439.78G) 09:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852134.43 09:50:00 [---] pick_jobs_to_run() start 09:50:00 [---] pick_jobs_to_run() end 09:50:00 [---] [rr_sim] end 09:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:50:00 [---] [cpu_sched_debug] preliminary job list: 09:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:50:00 [---] [cpu_sched_debug] final job list: 09:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:50:00 [---] [cpu_sched_debug] enforce_run_list: end 09:50:00 [---] choose_project(): 35400.000000 09:50:00 [iThena.Measurements] doing RPC to get work 09:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400199.09 cpu 2145812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:50:00 [---] Got 0 tasks 09:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:51:00 [---] [rr_sim] doing sim: CPU sched 09:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986701448.31G/439.78G) 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243630.78 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 224363.08 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224363.078245 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019267.70 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 201926.77 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426289.848665 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817340.93 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 181734.09 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608023.942043 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635606.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 163560.68 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771584.626083 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472046.16 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 147204.62 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918789.241719 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324841.54 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 132484.15 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051273.395792 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192357.39 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 119235.74 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170509.134457 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073121.65 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 107312.16 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277821.299256 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965809.48 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 96580.95 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374402.247575 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869228.53 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 86922.85 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461325.101062 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782305.68 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 78230.57 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539555.669201 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704075.11 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 70407.51 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609963.180525 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633667.60 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 63366.76 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673329.940717 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570300.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 57030.08 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730360.024890 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513270.76 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 51327.08 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781687.100646 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461943.68 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 46194.37 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827881.468826 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415749.31 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 41574.93 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869456.400188 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374174.38 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 37417.44 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906873.838413 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336756.94 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 33675.69 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940549.532817 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303081.25 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 30308.12 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970857.657780 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272773.12 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 27277.31 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998134.970246 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245495.81 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 24549.58 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022684.551466 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220946.23 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 22094.62 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044779.174564 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198851.61 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 19885.16 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064664.335352 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178966.45 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 17896.64 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082560.980062 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161069.80 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 16106.98 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098667.960300 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144962.82 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 14496.28 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113164.242515 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130466.54 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 13046.65 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126210.896508 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117419.89 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 11741.99 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137952.885102 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105677.90 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 10567.79 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148520.674836 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95110.11 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 9511.01 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158031.685597 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85599.10 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 8559.91 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166591.595282 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77039.19 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 7703.92 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174295.513998 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69335.27 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 6933.53 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181229.040843 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62401.74 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 6240.17 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187469.215003 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56161.57 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 5616.16 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193085.371748 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50545.41 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 5054.54 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198139.912817 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45490.87 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 4549.09 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202688.999780 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40941.78 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 4094.18 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206783.178047 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36847.60 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3684.76 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210467.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33162.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214067.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29562.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217667.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25962.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221267.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22362.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224867.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18762.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228467.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15162.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232067.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11562.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235667.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7962.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239267.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4362.84 sec) 09:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242867.938487 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 762.84 sec) 09:51:00 [GPUGRID] [rr_sim] 2243630.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986701448.31G/439.78G) 09:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852131.70 09:51:00 [---] pick_jobs_to_run() start 09:51:00 [---] pick_jobs_to_run() end 09:51:00 [---] [rr_sim] end 09:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:51:00 [---] [cpu_sched_debug] preliminary job list: 09:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:51:00 [---] [cpu_sched_debug] final job list: 09:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:51:00 [---] [cpu_sched_debug] enforce_run_list: end 09:51:00 [---] choose_project(): 35460.000000 09:51:00 [iThena.Measurements] doing RPC to get work 09:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400139.09 cpu 2145752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:51:00 [---] Got 0 tasks 09:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:52:00 [---] [rr_sim] doing sim: CPU sched 09:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986673857.95G/439.78G) 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243568.05 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 224356.80 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224356.804556 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019211.24 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 201921.12 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426277.928657 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817290.12 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 181729.01 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 608006.940348 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635561.11 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 163556.11 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771563.050869 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1472004.99 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 147200.50 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918763.550339 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324804.50 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 132480.45 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051243.999861 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192324.05 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 119232.40 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170476.404431 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073091.64 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 107309.16 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277785.568545 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965782.48 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 96578.25 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374363.816247 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869204.23 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 86920.42 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461284.239178 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782283.81 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 78228.38 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539512.619817 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704055.43 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 70405.54 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609918.162391 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633649.88 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 63364.99 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673283.150709 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570284.89 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 57028.49 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730311.640194 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513256.41 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 51325.64 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781637.280731 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461930.76 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 46193.08 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827830.357214 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415737.69 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 41573.77 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869404.126049 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374163.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 37416.39 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906820.518001 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336747.53 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 33674.75 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940495.270757 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303072.77 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 30307.28 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970802.548238 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272765.50 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 27276.55 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998079.097970 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245488.95 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 24548.89 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022627.992730 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220940.05 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 22094.01 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044721.998013 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198846.05 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 19884.60 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064606.602768 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178961.44 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 17896.14 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082502.747048 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161065.30 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 16106.53 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098609.276899 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144958.77 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 14495.88 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113105.153766 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130462.89 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 13046.29 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126151.442945 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117416.60 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 11741.66 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137893.103207 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105674.94 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 10567.49 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148460.597443 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95107.45 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 9510.74 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157971.342255 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85596.70 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 8559.67 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166531.012586 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77037.03 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 7703.70 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174234.715884 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69333.33 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 6933.33 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181168.048852 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62400.00 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 6240.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187408.048523 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56160.00 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 5616.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193024.048227 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50544.00 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 5054.40 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198078.447961 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45489.60 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 4548.96 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202627.407721 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40940.64 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 4094.06 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206721.471505 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36846.57 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3684.66 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210406.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33161.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214006.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29561.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217606.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25961.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221206.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22361.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224806.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18761.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228406.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15161.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232006.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11561.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235606.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7961.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239206.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4361.92 sec) 09:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242806.128911 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 761.92 sec) 09:52:00 [GPUGRID] [rr_sim] 2243568.05: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986673857.95G/439.78G) 09:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852128.96 09:52:00 [---] pick_jobs_to_run() start 09:52:00 [---] pick_jobs_to_run() end 09:52:00 [---] [rr_sim] end 09:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:52:00 [---] [cpu_sched_debug] preliminary job list: 09:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:52:00 [---] [cpu_sched_debug] final job list: 09:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:52:00 [---] [cpu_sched_debug] enforce_run_list: end 09:52:00 [---] choose_project(): 35520.000000 09:52:00 [iThena.Measurements] doing RPC to get work 09:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400079.09 cpu 2145692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:52:00 [---] Got 0 tasks 09:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:53:00 [---] [rr_sim] doing sim: CPU sched 09:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986646267.59G/439.78G) 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243505.31 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 224350.53 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224350.530868 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019154.78 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 201915.48 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426266.008649 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817239.30 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 181723.93 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607989.938653 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635515.37 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 163551.54 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771541.475656 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471963.83 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 147196.38 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918737.858958 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324767.45 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 132476.74 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051214.603930 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192290.70 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 119229.07 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170443.674406 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073061.63 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 107306.16 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277749.837833 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965755.47 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 96575.55 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374325.384918 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869179.92 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 86917.99 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461243.377294 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782261.93 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 78226.19 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539469.570433 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704035.74 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 70403.57 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609873.144258 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633632.16 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 63363.22 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673236.360700 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570268.95 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 57026.89 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730263.255498 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513242.05 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 51324.21 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781587.460817 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461917.85 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 46191.78 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827779.245603 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415726.06 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 41572.61 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869351.851911 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374153.46 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 37415.35 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906767.197588 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336738.11 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 33673.81 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940441.008697 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303064.30 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 30306.43 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970747.438696 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272757.87 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 27275.79 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998023.225694 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245482.08 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 24548.21 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022571.433993 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220933.87 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 22093.39 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044664.821462 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198840.49 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 19884.05 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064548.870184 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178956.44 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 17895.64 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082444.514034 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161060.79 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 16106.08 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098550.593498 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144954.72 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 14495.47 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113046.065017 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130459.24 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 13045.92 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126091.989383 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117413.32 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 11741.33 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137833.321313 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105671.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 10567.20 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148400.520050 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95104.79 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 9510.48 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157910.998913 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85594.31 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 8559.43 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166470.429890 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77034.88 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 7703.49 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174173.917769 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69331.39 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 6933.14 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181107.056860 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62398.25 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 6239.83 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187346.882042 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56158.43 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 5615.84 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192962.724706 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50542.58 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 5054.26 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198016.983104 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45488.33 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 4548.83 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202565.815662 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40939.49 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 4093.95 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206659.764964 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36845.54 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3684.55 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210344.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33160.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213944.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29560.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217544.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25960.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221144.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22360.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224744.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18760.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228344.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15160.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231944.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11560.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235544.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7960.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239144.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4360.99 sec) 09:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242744.319335 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 760.99 sec) 09:53:00 [GPUGRID] [rr_sim] 2243505.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986646267.59G/439.78G) 09:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852126.22 09:53:00 [---] pick_jobs_to_run() start 09:53:00 [---] pick_jobs_to_run() end 09:53:00 [---] [rr_sim] end 09:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:53:00 [---] [cpu_sched_debug] preliminary job list: 09:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:53:00 [---] [cpu_sched_debug] final job list: 09:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:53:00 [---] [cpu_sched_debug] enforce_run_list: end 09:53:00 [---] choose_project(): 35580.000000 09:53:00 [iThena.Measurements] doing RPC to get work 09:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 400019.09 cpu 2145632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:53:00 [---] Got 0 tasks 09:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:54:00 [---] [rr_sim] doing sim: CPU sched 09:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986618677.24G/439.78G) 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243442.57 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 224344.26 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224344.257180 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019098.31 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 201909.83 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426254.088642 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817188.48 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 181718.85 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607972.936958 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635469.63 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 163546.96 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771519.900442 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471922.67 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 147192.27 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918712.167578 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324730.40 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 132473.04 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051185.208000 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192257.36 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 119225.74 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170410.944380 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073031.63 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 107303.16 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277714.107122 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965728.46 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 96572.85 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374286.953589 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869155.62 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 86915.56 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461202.515410 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782240.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 78224.01 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539426.521049 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704016.05 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 70401.61 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609828.126124 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633614.45 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 63361.44 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673189.570692 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570253.00 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 57025.30 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730214.870803 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513227.70 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 51322.77 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781537.640902 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461904.93 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 46190.49 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827728.133992 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415714.44 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 41571.44 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869299.577773 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374142.99 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 37414.30 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906713.877175 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336728.69 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 33672.87 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940386.746638 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303055.83 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 30305.58 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970692.329154 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272750.24 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 27275.02 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997967.353418 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245475.22 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 24547.52 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022514.875256 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220927.70 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 22092.77 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044607.644911 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198834.93 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 19883.49 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064491.137600 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178951.43 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 17895.14 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082386.281020 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161056.29 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 16105.63 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098491.910098 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144950.66 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 14495.07 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112986.976268 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130455.60 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 13045.56 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126032.535821 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117410.04 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 11741.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137773.539419 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105669.03 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 10566.90 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148340.442657 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95102.13 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 9510.21 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157850.655571 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85591.92 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 8559.19 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166409.847194 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77032.72 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 7703.27 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174113.119654 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69329.45 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 6932.95 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181046.064869 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62396.51 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 6239.65 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187285.715562 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56156.86 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 5615.69 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192901.401186 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50541.17 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 5054.12 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197955.518247 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45487.05 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 4548.71 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202504.223602 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40938.35 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 4093.83 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206598.058422 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36844.51 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3684.45 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210282.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33160.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213882.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29560.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217482.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25960.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221082.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22360.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224682.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18760.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228282.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15160.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231882.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11560.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235482.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7960.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239082.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4360.06 sec) 09:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242682.509760 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 760.06 sec) 09:54:00 [GPUGRID] [rr_sim] 2243442.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986618677.24G/439.78G) 09:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852123.48 09:54:00 [---] pick_jobs_to_run() start 09:54:00 [---] pick_jobs_to_run() end 09:54:00 [---] [rr_sim] end 09:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:54:00 [---] [cpu_sched_debug] preliminary job list: 09:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:54:00 [---] [cpu_sched_debug] final job list: 09:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:54:00 [---] [cpu_sched_debug] enforce_run_list: end 09:54:00 [---] choose_project(): 35640.000000 09:54:00 [iThena.Measurements] doing RPC to get work 09:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399959.09 cpu 2145572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:54:00 [---] Got 0 tasks 09:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:55:00 [---] [rr_sim] doing sim: CPU sched 09:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986591086.88G/439.78G) 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243379.83 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 224337.98 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224337.983492 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2019041.85 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 201904.19 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426242.168634 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817137.67 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 181713.77 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607955.935263 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635423.90 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 163542.39 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771498.325228 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471881.51 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 147188.15 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918686.476197 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324693.36 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 132469.34 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051155.812069 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192224.02 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 119222.40 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170378.214354 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1073001.62 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 107300.16 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277678.376410 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965701.46 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 96570.15 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374248.522261 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869131.31 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 86913.13 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461161.653526 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782218.18 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 78221.82 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539383.471665 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703996.36 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 70399.64 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609783.107991 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633596.73 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 63359.67 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673142.780683 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570237.05 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 57023.71 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730166.486107 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513213.35 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 51321.33 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781487.820988 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461892.01 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 46189.20 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827677.022381 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415702.81 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 41570.28 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869247.303634 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374132.53 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 37413.25 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906660.556763 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336719.28 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 33671.93 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940332.484578 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303047.35 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 30304.74 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970637.219612 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272742.62 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 27274.26 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997911.481142 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245468.35 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 24546.84 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022458.316520 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220921.52 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 22092.15 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044550.468360 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198829.37 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 19882.94 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064433.405015 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178946.43 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 17894.64 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082328.048006 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161051.79 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 16105.18 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098433.226697 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144946.61 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 14494.66 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112927.887519 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130451.95 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 13045.19 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125973.082259 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117406.75 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 11740.68 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137713.757524 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105666.08 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 10566.61 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148280.365264 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95099.47 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 9509.95 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157790.312229 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85589.52 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 8558.95 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166349.264498 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77030.57 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 7703.06 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174052.321540 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69327.51 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 6932.75 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180985.072878 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62394.76 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 6239.48 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187224.549081 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56155.29 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 5615.53 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192840.077665 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50539.76 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 5053.98 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197894.053390 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45485.78 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 4548.58 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202442.631543 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40937.20 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 4093.72 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206536.351880 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36843.48 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3684.35 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210220.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33159.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213820.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29559.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217420.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25959.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221020.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22359.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224620.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18759.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228220.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15159.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231820.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11559.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235420.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7959.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239020.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4359.13 sec) 09:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242620.700184 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 759.13 sec) 09:55:00 [GPUGRID] [rr_sim] 2243379.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986591086.88G/439.78G) 09:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852120.75 09:55:00 [---] pick_jobs_to_run() start 09:55:00 [---] pick_jobs_to_run() end 09:55:00 [---] [rr_sim] end 09:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:55:00 [---] [cpu_sched_debug] preliminary job list: 09:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:55:00 [---] [cpu_sched_debug] final job list: 09:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:55:00 [---] [cpu_sched_debug] enforce_run_list: end 09:55:00 [---] choose_project(): 35700.000000 09:55:00 [iThena.Measurements] doing RPC to get work 09:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399899.09 cpu 2145512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:55:00 [---] Got 0 tasks 09:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:56:00 [---] [rr_sim] doing sim: CPU sched 09:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986563496.53G/439.78G) 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243317.10 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 224331.71 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224331.709803 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018985.39 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 201898.54 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426230.248627 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817086.85 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 181708.68 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607938.933567 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635378.16 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 163537.82 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771476.750014 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471840.35 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 147184.03 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918660.784816 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324656.31 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 132465.63 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051126.416138 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192190.68 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 119219.07 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170345.484328 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072971.61 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 107297.16 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277642.645699 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965674.45 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 96567.45 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374210.090932 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869107.01 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 86910.70 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461120.791642 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782196.31 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 78219.63 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539340.422282 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703976.68 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 70397.67 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609738.089857 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633579.01 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 63357.90 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673095.990675 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570221.11 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 57022.11 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730118.101411 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513199.00 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 51319.90 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781438.001073 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461879.10 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 46187.91 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827625.910769 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415691.19 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 41569.12 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869195.029496 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374122.07 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 37412.21 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906607.236350 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336709.86 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 33670.99 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940278.222518 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303038.88 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 30303.89 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970582.110070 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272734.99 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 27273.50 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997855.608867 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245461.49 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 24546.15 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022401.757783 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220915.34 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 22091.53 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044493.291809 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198823.81 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 19882.38 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064375.672431 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178941.43 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 17894.14 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082269.814992 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161047.28 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 16104.73 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098374.543296 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144942.55 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 14494.26 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112868.798770 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130448.30 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 13044.83 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125913.628696 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117403.47 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 11740.35 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137653.975630 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105663.12 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 10566.31 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148220.287871 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95096.81 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 9509.68 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157729.968887 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85587.13 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 8558.71 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166288.681802 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77028.42 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 7702.84 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173991.523425 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69325.57 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 6932.56 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180924.080886 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62393.02 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 6239.30 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187163.382601 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56153.72 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 5615.37 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192778.754144 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50538.34 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 5053.83 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197832.588533 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45484.51 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 4548.45 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202381.039484 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40936.06 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 4093.61 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206474.645339 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36842.45 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3684.25 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210158.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33158.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213758.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29558.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217358.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25958.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220958.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22358.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224558.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18758.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228158.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15158.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231758.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11558.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235358.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7958.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238958.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4358.21 sec) 09:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242558.890608 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 758.21 sec) 09:56:00 [GPUGRID] [rr_sim] 2243317.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986563496.53G/439.78G) 09:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852118.01 09:56:00 [---] pick_jobs_to_run() start 09:56:00 [---] pick_jobs_to_run() end 09:56:00 [---] [rr_sim] end 09:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:56:00 [---] [cpu_sched_debug] preliminary job list: 09:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:56:00 [---] [cpu_sched_debug] final job list: 09:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:56:00 [---] [cpu_sched_debug] enforce_run_list: end 09:56:00 [---] choose_project(): 35760.000000 09:56:00 [iThena.Measurements] doing RPC to get work 09:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399839.09 cpu 2145452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:56:00 [---] Got 0 tasks 09:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:57:00 [---] [rr_sim] doing sim: CPU sched 09:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986535906.17G/439.78G) 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243254.36 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 224325.44 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224325.436115 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018928.93 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 201892.89 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426218.328619 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1817036.03 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 181703.60 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607921.931872 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635332.43 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 163533.24 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771455.174800 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471799.19 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 147179.92 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918635.093436 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324619.27 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 132461.93 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051097.020207 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192157.34 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 119215.73 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170312.754302 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072941.61 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 107294.16 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277606.914987 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965647.45 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 96564.74 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374171.659604 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869082.70 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 86908.27 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461079.929759 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782174.43 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 78217.44 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539297.372898 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703956.99 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 70395.70 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609693.071723 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633561.29 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 63356.13 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673049.200666 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570205.16 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 57020.52 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730069.716715 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513184.64 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 51318.46 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781388.181159 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461866.18 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 46186.62 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827574.799158 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415679.56 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 41567.96 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869142.755358 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374111.61 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 37411.16 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906553.915937 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336700.45 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 33670.04 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940223.960459 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303030.40 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 30303.04 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970527.000528 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272727.36 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 27272.74 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997799.736591 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245454.62 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 24545.46 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022345.199047 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220909.16 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 22090.92 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044436.115257 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198818.25 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 19881.82 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064317.939847 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178936.42 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 17893.64 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082211.581978 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161042.78 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 16104.28 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098315.859895 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144938.50 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 14493.85 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112809.710021 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130444.65 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 13044.47 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125854.175134 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117400.19 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 11740.02 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137594.193736 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105660.17 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 10566.02 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148160.210478 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95094.15 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 9509.42 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157669.625545 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85584.74 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 8558.47 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166228.099106 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77026.26 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 7702.63 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173930.725311 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69323.64 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 6932.36 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180863.088895 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62391.27 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 6239.13 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187102.216121 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56152.15 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 5615.21 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192717.430624 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50536.93 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 5053.69 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197771.123677 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45483.24 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 4548.32 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202319.447424 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40934.91 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 4093.49 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206412.938797 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36841.42 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3684.14 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210097.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33157.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213697.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29557.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217297.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25957.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220897.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22357.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224497.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18757.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228097.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15157.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231697.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11557.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235297.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7957.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238897.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4357.28 sec) 09:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242497.081033 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 757.28 sec) 09:57:00 [GPUGRID] [rr_sim] 2243254.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986535906.17G/439.78G) 09:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852115.27 09:57:00 [---] pick_jobs_to_run() start 09:57:00 [---] pick_jobs_to_run() end 09:57:00 [---] [rr_sim] end 09:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:57:00 [---] [cpu_sched_debug] preliminary job list: 09:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:57:00 [---] [cpu_sched_debug] final job list: 09:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:57:00 [---] [cpu_sched_debug] enforce_run_list: end 09:57:00 [---] choose_project(): 35820.000000 09:57:00 [iThena.Measurements] doing RPC to get work 09:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399779.09 cpu 2145392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:57:00 [---] Got 0 tasks 09:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:58:00 [---] [rr_sim] doing sim: CPU sched 09:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986508315.81G/439.78G) 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243191.62 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 224319.16 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224319.162427 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018872.46 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 201887.25 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426206.408611 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816985.22 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 181698.52 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607904.930177 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635286.69 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 163528.67 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771433.599587 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471758.02 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 147175.80 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918609.402055 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324582.22 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 132458.22 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051067.624277 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192124.00 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 119212.40 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170280.024276 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072911.60 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 107291.16 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277571.184275 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965620.44 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 96562.04 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374133.228275 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869058.40 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 86905.84 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1461039.067875 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782152.56 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 78215.26 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539254.323514 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703937.30 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 70393.73 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609648.053590 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633543.57 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 63354.36 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1673002.410658 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570189.21 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 57018.92 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1730021.332019 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513170.29 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 51317.03 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781338.361244 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461853.26 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 46185.33 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827523.687547 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415667.94 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 41566.79 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869090.481219 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374101.14 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 37410.11 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906500.595524 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336691.03 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 33669.10 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940169.698399 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303021.93 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 30302.19 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970471.890986 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272719.73 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 27271.97 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997743.864315 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245447.76 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 24544.78 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022288.640310 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220902.98 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 22090.30 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044378.938706 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198812.69 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 19881.27 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064260.207263 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178931.42 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 17893.14 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082153.348964 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161038.28 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 16103.83 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098257.176494 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144934.45 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 14493.44 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112750.621272 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130441.00 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 13044.10 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125794.721572 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117396.90 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 11739.69 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137534.411842 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105657.21 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 10565.72 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148100.133085 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95091.49 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 9509.15 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157609.282203 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85582.34 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 8558.23 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166167.516410 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77024.11 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 7702.41 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173869.927196 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69321.70 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 6932.17 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180802.096903 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62389.53 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 6238.95 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187041.049640 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56150.57 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 5615.06 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192656.107103 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50535.52 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 5053.55 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197709.658820 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45481.97 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 4548.20 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202257.855365 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40933.77 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 4093.38 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206351.232256 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36840.39 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3684.04 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210035.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33156.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213635.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29556.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217235.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25956.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220835.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22356.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224435.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18756.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228035.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15156.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231635.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11556.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235235.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7956.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238835.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4356.35 sec) 09:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242435.271457 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 756.35 sec) 09:58:00 [GPUGRID] [rr_sim] 2243191.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986508315.81G/439.78G) 09:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852112.54 09:58:00 [---] pick_jobs_to_run() start 09:58:00 [---] pick_jobs_to_run() end 09:58:00 [---] [rr_sim] end 09:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:58:00 [---] [cpu_sched_debug] preliminary job list: 09:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:58:00 [---] [cpu_sched_debug] final job list: 09:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:58:00 [---] [cpu_sched_debug] enforce_run_list: end 09:58:00 [---] choose_project(): 35880.000000 09:58:00 [iThena.Measurements] doing RPC to get work 09:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399719.09 cpu 2145332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:58:00 [---] Got 0 tasks 09:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 09:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 09:59:00 [---] [rr_sim] doing sim: CPU sched 09:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986480725.46G/439.78G) 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243128.89 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 224312.89 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224312.888739 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018816.00 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 201881.60 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426194.488604 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816934.40 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 181693.44 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607887.928482 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635240.96 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 163524.10 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771412.024373 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471716.86 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 147171.69 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918583.710674 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324545.18 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 132454.52 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051038.228346 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192090.66 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 119209.07 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170247.294250 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072881.59 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 107288.16 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277535.453564 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965593.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 96559.34 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374094.796946 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869034.09 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 86903.41 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460998.205991 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782130.68 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 78213.07 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539211.274130 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703917.61 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 70391.76 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609603.035456 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633525.85 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 63352.59 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672955.620649 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570173.27 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 57017.33 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729972.947323 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513155.94 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 51315.59 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781288.541330 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461840.35 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 46184.03 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827472.575936 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415656.31 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 41565.63 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869038.207081 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374090.68 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 37409.07 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906447.275112 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336681.61 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 33668.16 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940115.436339 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303013.45 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 30301.35 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970416.781444 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272712.11 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 27271.21 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997687.992039 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245440.90 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 24544.09 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022232.081574 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220896.81 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 22089.68 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044321.762155 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198807.13 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 19880.71 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064202.474678 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178926.41 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 17892.64 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082095.115949 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161033.77 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 16103.38 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098198.493093 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144930.39 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 14493.04 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112691.532523 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130437.35 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 13043.74 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125735.268009 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117393.62 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 11739.36 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137474.629947 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105654.26 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 10565.43 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148040.055691 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95088.83 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 9508.88 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157548.938861 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85579.95 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 8557.99 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166106.933714 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77021.95 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 7702.20 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173809.129081 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69319.76 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 6931.98 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180741.104912 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62387.78 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 6238.78 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186979.883160 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56149.00 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 5614.90 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192594.783583 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50534.10 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 5053.41 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197648.193963 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45480.69 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 4548.07 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202196.263306 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40932.62 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 4093.26 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206289.525714 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36839.36 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3683.94 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209973.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33155.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213573.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29555.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217173.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25955.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220773.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22355.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224373.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18755.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227973.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15155.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231573.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11555.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235173.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7955.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238773.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4355.43 sec) 09:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242373.461881 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 755.43 sec) 09:59:00 [GPUGRID] [rr_sim] 2243128.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986480725.46G/439.78G) 09:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852109.80 09:59:00 [---] pick_jobs_to_run() start 09:59:00 [---] pick_jobs_to_run() end 09:59:00 [---] [rr_sim] end 09:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 09:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 09:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 09:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 09:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 09:59:00 [---] [cpu_sched_debug] preliminary job list: 09:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:59:00 [---] [cpu_sched_debug] final job list: 09:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 09:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 09:59:00 [---] [cpu_sched_debug] enforce_run_list: end 09:59:00 [---] choose_project(): 35940.000000 09:59:00 [iThena.Measurements] doing RPC to get work 09:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399659.09 cpu 2145272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 09:59:00 [---] Got 0 tasks 10:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:00:00 [---] [rr_sim] doing sim: CPU sched 10:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986453135.10G/439.78G) 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243066.15 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 224306.62 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224306.615051 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018759.54 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 201875.95 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426182.568596 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816883.58 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 181688.36 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607870.926787 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635195.22 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 163519.52 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771390.449159 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471675.70 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 147167.57 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918558.019294 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324508.13 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 132450.81 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1051008.832415 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192057.32 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 119205.73 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170214.564224 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072851.59 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 107285.16 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277499.722852 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965566.43 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 96556.64 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374056.365618 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 869009.78 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 86900.98 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460957.344107 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782108.81 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 78210.88 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539168.224747 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703897.93 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 70389.79 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609558.017323 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633508.13 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 63350.81 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672908.830641 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570157.32 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 57015.73 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729924.562627 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513141.59 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 51314.16 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781238.721415 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461827.43 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 46182.74 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827421.464324 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415644.69 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 41564.47 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868985.932943 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374080.22 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 37408.02 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906393.954699 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336672.20 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 33667.22 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940061.174280 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303004.98 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 30300.50 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970361.671902 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272704.48 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 27270.45 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997632.119763 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245434.03 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 24543.40 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022175.522837 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220890.63 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 22089.06 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044264.585604 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198801.56 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 19880.16 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064144.742094 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178921.41 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 17892.14 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082036.882935 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161029.27 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 16102.93 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098139.809693 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144926.34 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 14492.63 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112632.443774 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130433.71 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 13043.37 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125675.814447 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117390.34 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 11739.03 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137414.848053 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105651.30 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 10565.13 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147979.978298 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95086.17 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 9508.62 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157488.595519 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85577.55 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 8557.76 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166046.351018 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77019.80 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 7701.98 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173748.330967 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69317.82 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 6931.78 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180680.112921 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62386.04 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 6238.60 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186918.716679 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56147.43 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 5614.74 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192533.460062 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50532.69 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 5053.27 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197586.729106 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45479.42 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 4547.94 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202134.671246 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40931.48 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 4093.15 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206227.819172 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36838.33 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3683.83 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209911.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33154.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213511.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29554.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217111.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25954.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220711.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22354.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224311.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18754.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227911.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15154.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231511.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11554.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235111.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7954.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238711.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4354.50 sec) 10:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242311.652306 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 754.50 sec) 10:00:00 [GPUGRID] [rr_sim] 2243066.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986453135.10G/439.78G) 10:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852107.06 10:00:00 [---] pick_jobs_to_run() start 10:00:00 [---] pick_jobs_to_run() end 10:00:00 [---] [rr_sim] end 10:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:00:00 [---] [cpu_sched_debug] preliminary job list: 10:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:00:00 [---] [cpu_sched_debug] final job list: 10:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:00:00 [---] [cpu_sched_debug] enforce_run_list: end 10:00:00 [---] choose_project(): 36000.000000 10:00:00 [iThena.Measurements] doing RPC to get work 10:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399599.09 cpu 2145212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:00:00 [---] Got 0 tasks 10:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:01:00 [---] [rr_sim] doing sim: CPU sched 10:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986425544.74G/439.78G) 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2243003.41 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 224300.34 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224300.341362 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018703.07 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 201870.31 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426170.648589 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816832.77 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 181683.28 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607853.925092 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635149.49 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 163514.95 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771368.873945 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471634.54 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 147163.45 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918532.327913 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324471.09 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 132447.11 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050979.436484 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1192023.98 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 119202.40 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170181.834198 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072821.58 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 107282.16 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277463.992141 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965539.42 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 96553.94 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1374017.934289 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868985.48 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 86898.55 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460916.482223 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782086.93 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 78208.69 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539125.175363 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703878.24 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 70387.82 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609512.999189 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633490.41 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 63349.04 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672862.040632 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570141.37 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 57014.14 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729876.177932 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513127.24 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 51312.72 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781188.901501 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461814.51 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 46181.45 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827370.352713 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415633.06 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 41563.31 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868933.658804 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374069.75 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 37406.98 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906340.634286 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336662.78 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 33666.28 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940006.912220 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302996.50 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 30299.65 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970306.562360 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272696.85 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 27269.69 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997576.247487 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245427.17 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 24542.72 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022118.964101 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220884.45 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 22088.44 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044207.409053 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198796.00 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 19879.60 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064087.009510 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178916.40 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 17891.64 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081978.649921 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161024.76 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 16102.48 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098081.126292 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144922.29 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 14492.23 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112573.355025 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130430.06 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 13043.01 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125616.360885 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117387.05 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 11738.71 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137355.066159 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105648.35 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 10564.83 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147919.900905 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95083.51 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 9508.35 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157428.252177 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85575.16 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 8557.52 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165985.768322 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77017.65 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 7701.76 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173687.532852 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69315.88 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 6931.59 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180619.120929 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62384.29 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 6238.43 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186857.550199 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56145.86 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 5614.59 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192472.136541 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50531.28 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 5053.13 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197525.264250 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45478.15 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 4547.81 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202073.079187 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40930.33 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 4093.03 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206166.112631 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36837.30 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3683.73 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209849.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33153.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213449.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29553.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217049.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25953.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220649.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22353.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224249.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18753.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227849.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15153.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231449.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11553.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235049.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7953.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238649.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4353.57 sec) 10:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242249.842730 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 753.57 sec) 10:01:00 [GPUGRID] [rr_sim] 2243003.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986425544.74G/439.78G) 10:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852104.33 10:01:00 [---] pick_jobs_to_run() start 10:01:00 [---] pick_jobs_to_run() end 10:01:00 [---] [rr_sim] end 10:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:01:00 [---] [cpu_sched_debug] preliminary job list: 10:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:01:00 [---] [cpu_sched_debug] final job list: 10:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:01:00 [---] [cpu_sched_debug] enforce_run_list: end 10:01:00 [---] choose_project(): 36060.000000 10:01:00 [iThena.Measurements] doing RPC to get work 10:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399539.09 cpu 2145152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:01:00 [---] Got 0 tasks 10:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:02:00 [---] [rr_sim] doing sim: CPU sched 10:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986397954.39G/439.78G) 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242940.68 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 224294.07 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224294.067674 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018646.61 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 201864.66 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426158.728581 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816781.95 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 181678.19 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607836.923397 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635103.75 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 163510.38 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771347.298732 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471593.38 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 147159.34 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918506.636533 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324434.04 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 132443.40 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050950.040554 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191990.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 119199.06 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170149.104172 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072791.57 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 107279.16 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277428.261429 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965512.42 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 96551.24 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373979.502961 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868961.17 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 86896.12 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460875.620339 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782065.06 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 78206.51 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539082.125979 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703858.55 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 70385.86 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609467.981055 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633472.70 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 63347.27 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672815.250624 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570125.43 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 57012.54 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729827.793236 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513112.88 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 51311.29 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781139.081586 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461801.60 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 46180.16 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827319.241102 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415621.44 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 41562.14 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868881.384666 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374059.29 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 37405.93 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906287.313874 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336653.36 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 33665.34 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939952.650160 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302988.03 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 30298.80 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970251.452819 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272689.22 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 27268.92 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997520.375211 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245420.30 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 24542.03 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022062.405364 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220878.27 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 22087.83 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044150.232502 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198790.44 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 19879.04 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064029.276926 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178911.40 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 17891.14 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081920.416907 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161020.26 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 16102.03 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098022.442891 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144918.23 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 14491.82 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112514.266276 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130426.41 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 13042.64 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125556.907323 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117383.77 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 11738.38 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137295.284265 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105645.39 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 10564.54 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147859.823512 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95080.85 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 9508.09 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157367.908835 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85572.77 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 8557.28 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165925.185626 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77015.49 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 7701.55 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173626.734738 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69313.94 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 6931.39 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180558.128938 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62382.55 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 6238.25 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186796.383718 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56144.29 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 5614.43 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192410.813021 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50529.86 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 5052.99 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197463.799393 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45476.88 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 4547.69 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202011.487128 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40929.19 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 4092.92 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206104.406089 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36836.27 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3683.63 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209788.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33152.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213388.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29552.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216988.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25952.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220588.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22352.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224188.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18752.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227788.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15152.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231388.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11552.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234988.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7952.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238588.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4352.64 sec) 10:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242188.033154 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 752.64 sec) 10:02:00 [GPUGRID] [rr_sim] 2242940.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986397954.39G/439.78G) 10:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852101.59 10:02:00 [---] pick_jobs_to_run() start 10:02:00 [---] pick_jobs_to_run() end 10:02:00 [---] [rr_sim] end 10:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:02:00 [---] [cpu_sched_debug] preliminary job list: 10:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:02:00 [---] [cpu_sched_debug] final job list: 10:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:02:00 [---] [cpu_sched_debug] enforce_run_list: end 10:02:00 [---] choose_project(): 36120.000000 10:02:00 [iThena.Measurements] doing RPC to get work 10:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399479.09 cpu 2145092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:02:00 [---] Got 0 tasks 10:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:03:00 [---] [rr_sim] doing sim: CPU sched 10:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986370364.03G/439.78G) 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242877.94 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 224287.79 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224287.793986 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018590.15 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 201859.01 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426146.808573 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816731.13 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 181673.11 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607819.921702 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635058.02 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 163505.80 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771325.723518 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471552.22 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 147155.22 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918480.945152 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324396.99 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 132439.70 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050920.644623 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191957.30 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 119195.73 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170116.374146 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072761.57 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 107276.16 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277392.530718 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965485.41 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 96548.54 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373941.071632 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868936.87 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 86893.69 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460834.758455 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782043.18 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 78204.32 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1539039.076595 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703838.86 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 70383.89 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609422.962922 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633454.98 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 63345.50 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672768.460616 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570109.48 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 57010.95 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729779.408540 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513098.53 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 51309.85 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781089.261672 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461788.68 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 46178.87 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827268.129491 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415609.81 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 41560.98 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868829.110528 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374048.83 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 37404.88 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906233.993461 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336643.95 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 33664.39 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939898.388101 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302979.55 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 30297.96 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970196.343277 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272681.60 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 27268.16 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997464.502935 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245413.44 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 24541.34 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022005.846627 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220872.09 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 22087.21 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044093.055951 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198784.88 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 19878.49 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063971.544342 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178906.40 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 17890.64 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081862.183893 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161015.76 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 16101.58 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097963.759490 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144914.18 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 14491.42 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112455.177527 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130422.76 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 13042.28 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125497.453760 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117380.49 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 11738.05 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137235.502370 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105642.44 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 10564.24 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147799.746119 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95078.19 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 9507.82 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157307.565493 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85570.37 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 8557.04 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165864.602930 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77013.34 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 7701.33 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173565.936623 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69312.00 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 6931.20 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180497.136947 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62380.80 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 6238.08 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186735.217238 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56142.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 5614.27 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192349.489500 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50528.45 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 5052.85 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197402.334536 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45475.61 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 4547.56 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201949.895068 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40928.04 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 4092.80 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206042.699548 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36835.24 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3683.52 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209726.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33151.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213326.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29551.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216926.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25951.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220526.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22351.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224126.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18751.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227726.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15151.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231326.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11551.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234926.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7951.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238526.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4351.72 sec) 10:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242126.223579 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 751.72 sec) 10:03:00 [GPUGRID] [rr_sim] 2242877.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986370364.03G/439.78G) 10:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852098.85 10:03:00 [---] pick_jobs_to_run() start 10:03:00 [---] pick_jobs_to_run() end 10:03:00 [---] [rr_sim] end 10:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:03:00 [---] [cpu_sched_debug] preliminary job list: 10:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:03:00 [---] [cpu_sched_debug] final job list: 10:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:03:00 [---] [cpu_sched_debug] enforce_run_list: end 10:03:00 [---] choose_project(): 36180.000000 10:03:00 [iThena.Measurements] doing RPC to get work 10:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399419.09 cpu 2145032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2145032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2145032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:03:00 [---] Got 0 tasks 10:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:04:00 [---] [rr_sim] doing sim: CPU sched 10:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986342773.68G/439.78G) 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242815.20 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 224281.52 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224281.520298 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018533.68 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 201853.37 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426134.888566 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816680.31 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 181668.03 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607802.920007 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1635012.28 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 163501.23 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771304.148304 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471511.05 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 147151.11 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918455.253771 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324359.95 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 132435.99 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050891.248692 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191923.95 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 119192.40 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170083.644121 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072731.56 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 107273.16 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277356.800006 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965458.40 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 96545.84 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373902.640303 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868912.56 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 86891.26 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460793.896571 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 782021.31 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 78202.13 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538996.027212 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703819.18 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 70381.92 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609377.944788 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633437.26 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 63343.73 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672721.670607 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570093.53 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 57009.35 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729731.023844 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513084.18 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 51308.42 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1781039.441757 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461775.76 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 46177.58 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827217.017879 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415598.19 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 41559.82 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868776.836389 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374038.37 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 37403.84 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906180.673048 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336634.53 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 33663.45 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939844.126041 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302971.08 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 30297.11 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970141.233735 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272673.97 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 27267.40 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997408.630659 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245406.57 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 24540.66 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021949.287891 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220865.92 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 22086.59 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044035.879400 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198779.32 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 19877.93 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063913.811757 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178901.39 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 17890.14 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081803.950879 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161011.25 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 16101.13 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097905.076089 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144910.13 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 14491.01 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112396.088778 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130419.11 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 13041.91 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125438.000198 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117377.20 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 11737.72 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137175.720476 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105639.48 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 10563.95 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147739.668726 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95075.53 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 9507.55 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157247.222151 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85567.98 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 8556.80 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165804.020234 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77011.18 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 7701.12 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173505.138508 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69310.06 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 6931.01 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180436.144955 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62379.06 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 6237.91 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186674.050757 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56141.15 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 5614.12 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192288.165979 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50527.04 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 5052.70 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197340.869679 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45474.33 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 4547.43 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201888.303009 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40926.90 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 4092.69 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205980.993006 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36834.21 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3683.42 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209664.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33150.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213264.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29550.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216864.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25950.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220464.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22350.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224064.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18750.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227664.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15150.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231264.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11550.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234864.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7950.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238464.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4350.79 sec) 10:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242064.414003 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 750.79 sec) 10:04:00 [GPUGRID] [rr_sim] 2242815.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986342773.68G/439.78G) 10:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852096.12 10:04:00 [---] pick_jobs_to_run() start 10:04:00 [---] pick_jobs_to_run() end 10:04:00 [---] [rr_sim] end 10:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:04:00 [---] [cpu_sched_debug] preliminary job list: 10:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:04:00 [---] [cpu_sched_debug] final job list: 10:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:04:00 [---] [cpu_sched_debug] enforce_run_list: end 10:04:00 [---] choose_project(): 36240.000000 10:04:00 [iThena.Measurements] doing RPC to get work 10:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399359.09 cpu 2144972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:04:00 [---] Got 0 tasks 10:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:05:00 [---] [rr_sim] doing sim: CPU sched 10:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986315183.32G/439.78G) 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242752.47 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 224275.25 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224275.246610 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018477.22 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 201847.72 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426122.968558 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816629.50 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 181662.95 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607785.918312 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634966.55 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 163496.65 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771282.573090 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471469.89 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 147146.99 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918429.562391 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324322.90 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 132432.29 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050861.852761 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191890.61 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 119189.06 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170050.914095 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072701.55 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 107270.16 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277321.069295 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965431.40 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 96543.14 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373864.208975 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868888.26 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 86888.83 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460753.034687 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781999.43 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 78199.94 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538952.977828 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703799.49 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 70379.95 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609332.926655 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633419.54 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 63341.95 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672674.880599 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570077.59 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 57007.76 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729682.639148 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513069.83 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 51306.98 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780989.621843 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461762.84 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 46176.28 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827165.906268 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415586.56 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 41558.66 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868724.562251 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374027.90 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 37402.79 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906127.352635 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336625.11 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 33662.51 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939789.863981 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302962.60 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 30296.26 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970086.124193 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272666.34 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 27266.63 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997352.758383 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245399.71 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 24539.97 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021892.729154 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220859.74 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 22085.97 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043978.702848 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198773.76 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 19877.38 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063856.079173 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178896.39 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 17889.64 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081745.717865 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161006.75 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 16100.67 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097846.392688 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144906.07 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 14490.61 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112337.000029 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130415.47 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 13041.55 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125378.546636 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117373.92 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 11737.39 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137115.938582 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105636.53 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 10563.65 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147679.591333 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95072.87 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 9507.29 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157186.878809 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85565.59 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 8556.56 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165743.437538 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77009.03 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 7700.90 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173444.340394 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69308.13 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 6930.81 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180375.152964 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62377.31 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 6237.73 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186612.884277 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56139.58 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 5613.96 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192226.842459 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50525.62 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 5052.56 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197279.404823 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45473.06 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 4547.31 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201826.710950 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40925.76 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 4092.58 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205919.286464 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36833.18 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3683.32 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209602.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33149.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213202.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29549.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216802.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25949.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220402.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22349.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224002.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18749.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227602.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15149.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231202.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11549.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234802.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7949.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238402.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4349.86 sec) 10:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2242002.604427 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 749.86 sec) 10:05:00 [GPUGRID] [rr_sim] 2242752.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986315183.32G/439.78G) 10:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852093.38 10:05:00 [---] pick_jobs_to_run() start 10:05:00 [---] pick_jobs_to_run() end 10:05:00 [---] [rr_sim] end 10:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:05:00 [---] [cpu_sched_debug] preliminary job list: 10:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:05:00 [---] [cpu_sched_debug] final job list: 10:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:05:00 [---] [cpu_sched_debug] enforce_run_list: end 10:05:00 [---] choose_project(): 36300.000000 10:05:00 [iThena.Measurements] doing RPC to get work 10:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399299.09 cpu 2144912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:05:00 [---] Got 0 tasks 10:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:06:00 [---] [rr_sim] doing sim: CPU sched 10:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986287592.96G/439.78G) 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242689.73 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 224268.97 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224268.972921 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018420.76 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 201842.08 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426111.048551 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816578.68 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 181657.87 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607768.916617 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634920.81 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 163492.08 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771260.997876 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471428.73 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 147142.87 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918403.871010 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324285.86 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 132428.59 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050832.456830 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191857.27 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 119185.73 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1170018.184069 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072671.55 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 107267.15 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277285.338583 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965404.39 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 96540.44 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373825.777646 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868863.95 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 86886.40 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460712.172803 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781977.56 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 78197.76 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538909.928444 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703779.80 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 70377.98 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609287.908521 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633401.82 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 63340.18 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672628.090590 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570061.64 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 57006.16 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729634.254452 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513055.47 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 51305.55 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780939.801929 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461749.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 46174.99 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827114.794657 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415574.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 41557.49 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868672.288113 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374017.44 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 37401.74 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906074.032223 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336615.70 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 33661.57 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939735.601922 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302954.13 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 30295.41 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970031.014651 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272658.71 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 27265.87 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997296.886107 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245392.84 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 24539.28 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021836.170418 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220853.56 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 22085.36 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043921.526297 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198768.20 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 19876.82 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063798.346589 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178891.38 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 17889.14 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081687.484851 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161002.24 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 16100.22 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097787.709288 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144902.02 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 14490.20 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112277.911280 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130411.82 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 13041.18 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125319.093073 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117370.64 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 11737.06 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137056.156687 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105633.57 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 10563.36 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147619.513940 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95070.22 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 9507.02 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157126.535467 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85563.19 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 8556.32 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165682.854842 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77006.87 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 7700.69 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173383.542279 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69306.19 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 6930.62 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180314.160973 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62375.57 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 6237.56 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186551.717797 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56138.01 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 5613.80 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192165.518938 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50524.21 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 5052.42 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197217.939966 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45471.79 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 4547.18 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201765.118891 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40924.61 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 4092.46 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205857.579923 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36832.15 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3683.21 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209540.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33148.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213140.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29548.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216740.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25948.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220340.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22348.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223940.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18748.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227540.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15148.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231140.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11548.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234740.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7948.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238340.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4348.93 sec) 10:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241940.794852 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 748.93 sec) 10:06:00 [GPUGRID] [rr_sim] 2242689.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986287592.96G/439.78G) 10:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852090.64 10:06:00 [---] pick_jobs_to_run() start 10:06:00 [---] pick_jobs_to_run() end 10:06:00 [---] [rr_sim] end 10:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:06:00 [---] [cpu_sched_debug] preliminary job list: 10:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:06:00 [---] [cpu_sched_debug] final job list: 10:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:06:00 [---] [cpu_sched_debug] enforce_run_list: end 10:06:00 [---] choose_project(): 36360.000000 10:06:00 [iThena.Measurements] doing RPC to get work 10:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 572039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399239.09 cpu 2144852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:06:00 [---] Got 0 tasks 10:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:07:00 [---] [rr_sim] doing sim: CPU sched 10:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986260002.61G/439.78G) 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242626.99 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 224262.70 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224262.699233 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018364.29 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 201836.43 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426099.128543 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816527.86 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 181652.79 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607751.914922 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634875.08 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 163487.51 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771239.422663 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471387.57 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 147138.76 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918378.179630 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324248.81 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 132424.88 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050803.060900 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191823.93 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 119182.39 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169985.454043 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072641.54 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 107264.15 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277249.607872 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965377.38 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 96537.74 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373787.346318 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868839.65 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 86883.96 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460671.310919 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781955.68 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 78195.57 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538866.879060 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703760.11 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 70376.01 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609242.890387 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633384.10 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 63338.41 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672581.300582 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570045.69 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 57004.57 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729585.869757 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513041.12 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 51304.11 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780889.982014 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461737.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 46173.70 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827063.683046 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415563.31 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 41556.33 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868620.013974 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374006.98 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 37400.70 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906020.711810 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336606.28 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 33660.63 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939681.339862 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302945.65 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 30294.57 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969975.905109 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272651.09 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 27265.11 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997241.013831 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245385.98 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 24538.60 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021779.611681 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220847.38 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 22084.74 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043864.349746 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198762.64 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 19876.26 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063740.614005 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178886.38 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 17888.64 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081629.251837 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160997.74 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 16099.77 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097729.025887 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144897.97 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 14489.80 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112218.822531 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130408.17 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 13040.82 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125259.639511 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117367.35 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 11736.74 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136996.374793 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105630.62 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 10563.06 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147559.436547 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95067.56 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 9506.76 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157066.192125 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85560.80 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 8556.08 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165622.272146 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77004.72 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 7700.47 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173322.744164 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69304.25 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 6930.42 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180253.168981 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62373.82 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 6237.38 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186490.551316 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56136.44 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 5613.64 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192104.195418 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50522.80 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 5052.28 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197156.475109 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45470.52 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 4547.05 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201703.526831 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40923.47 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 4092.35 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205795.873381 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36831.12 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3683.11 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209478.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33148.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213078.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29548.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216678.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25948.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220278.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22348.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223878.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18748.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227478.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15148.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231078.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11548.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234678.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7948.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238278.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4348.01 sec) 10:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241878.985276 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 748.01 sec) 10:07:00 [GPUGRID] [rr_sim] 2242626.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986260002.61G/439.78G) 10:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852087.91 10:07:00 [---] pick_jobs_to_run() start 10:07:00 [---] pick_jobs_to_run() end 10:07:00 [---] [rr_sim] end 10:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:07:00 [---] [cpu_sched_debug] preliminary job list: 10:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:07:00 [---] [cpu_sched_debug] final job list: 10:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:07:00 [---] [cpu_sched_debug] enforce_run_list: end 10:07:00 [---] choose_project(): 36420.000000 10:07:00 [iThena.Measurements] doing RPC to get work 10:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399179.09 cpu 2144792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:07:00 [---] Got 0 tasks 10:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:08:00 [---] [rr_sim] doing sim: CPU sched 10:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986232412.25G/439.78G) 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242564.26 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 224256.43 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224256.425545 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018307.83 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 201830.78 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426087.208535 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816477.05 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 181647.70 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607734.913227 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634829.34 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 163482.93 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771217.847449 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471346.41 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 147134.64 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918352.488249 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324211.77 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 132421.18 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050773.664969 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191790.59 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 119179.06 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169952.724017 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072611.53 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 107261.15 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277213.877160 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965350.38 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 96535.04 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373748.914989 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868815.34 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 86881.53 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460630.449035 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781933.81 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 78193.38 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538823.829676 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703740.43 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 70374.04 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609197.872254 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633366.38 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 63336.64 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672534.510573 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570029.74 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 57002.97 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729537.485061 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513026.77 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 51302.68 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780840.162100 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461724.09 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 46172.41 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827012.571435 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415551.68 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 41555.17 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868567.739836 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373996.52 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 37399.65 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905967.391397 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336596.86 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 33659.69 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939627.077802 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302937.18 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 30293.72 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969920.795567 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272643.46 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 27264.35 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997185.141555 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245379.11 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 24537.91 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021723.052945 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220841.20 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 22084.12 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043807.173195 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198757.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 19875.71 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063682.881420 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178881.37 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 17888.14 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081571.018823 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160993.24 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 16099.32 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097670.342486 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144893.91 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 14489.39 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112159.733782 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130404.52 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 13040.45 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125200.185949 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117364.07 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 11736.41 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136936.592899 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105627.66 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 10562.77 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147499.359154 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95064.90 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 9506.49 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157005.848783 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85558.41 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 8555.84 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165561.689450 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77002.57 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 7700.26 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173261.946050 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69302.31 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 6930.23 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180192.176990 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62372.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 6237.21 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186429.384836 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56134.87 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 5613.49 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192042.871897 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50521.38 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 5052.14 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197095.010252 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45469.25 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 4546.92 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201641.934772 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40922.32 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 4092.23 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205734.166840 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36830.09 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3683.01 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209417.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33147.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213017.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29547.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216617.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25947.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220217.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22347.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223817.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18747.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227417.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15147.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231017.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11547.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234617.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7947.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238217.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4347.08 sec) 10:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241817.175701 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 747.08 sec) 10:08:00 [GPUGRID] [rr_sim] 2242564.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986232412.25G/439.78G) 10:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852085.17 10:08:00 [---] pick_jobs_to_run() start 10:08:00 [---] pick_jobs_to_run() end 10:08:00 [---] [rr_sim] end 10:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:08:00 [---] [cpu_sched_debug] preliminary job list: 10:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:08:00 [---] [cpu_sched_debug] final job list: 10:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:08:00 [---] [cpu_sched_debug] enforce_run_list: end 10:08:00 [---] choose_project(): 36480.000000 10:08:00 [iThena.Measurements] doing RPC to get work 10:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399119.09 cpu 2144732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:08:00 [---] Got 0 tasks 10:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:09:00 [---] [rr_sim] doing sim: CPU sched 10:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986232412.25G/439.78G) 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242564.26 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 224256.43 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224256.425545 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018307.83 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 201830.78 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426087.208535 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816477.05 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 181647.70 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607734.913227 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634829.34 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 163482.93 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771217.847449 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471346.41 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 147134.64 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918352.488249 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324211.77 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 132421.18 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050773.664969 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191790.59 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 119179.06 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169952.724017 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072611.53 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 107261.15 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277213.877160 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965350.38 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 96535.04 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373748.914989 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868815.34 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 86881.53 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460630.449035 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781933.81 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 78193.38 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538823.829676 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703740.43 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 70374.04 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609197.872254 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633366.38 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 63336.64 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672534.510573 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570029.74 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 57002.97 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729537.485061 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513026.77 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 51302.68 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780840.162100 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461724.09 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 46172.41 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827012.571435 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415551.68 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 41555.17 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868567.739836 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373996.52 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 37399.65 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905967.391397 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336596.86 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 33659.69 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939627.077802 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302937.18 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 30293.72 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969920.795567 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272643.46 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 27264.35 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997185.141555 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245379.11 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 24537.91 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021723.052945 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220841.20 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 22084.12 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043807.173195 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198757.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 19875.71 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063682.881420 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178881.37 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 17888.14 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081571.018823 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160993.24 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 16099.32 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097670.342486 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144893.91 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 14489.39 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112159.733782 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130404.52 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 13040.45 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125200.185949 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117364.07 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 11736.41 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136936.592899 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105627.66 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 10562.77 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147499.359154 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95064.90 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 9506.49 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157005.848783 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85558.41 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 8555.84 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165561.689450 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77002.57 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 7700.26 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173261.946050 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69302.31 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 6930.23 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180192.176990 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62372.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 6237.21 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186429.384836 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56134.87 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 5613.49 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192042.871897 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50521.38 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 5052.14 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197095.010252 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45469.25 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 4546.92 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201641.934772 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40922.32 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 4092.23 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205734.166840 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36830.09 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3683.01 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209417.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33147.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213017.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29547.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216617.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25947.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220217.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22347.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223817.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18747.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227417.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15147.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231017.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11547.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234617.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7947.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238217.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4347.08 sec) 10:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241817.175701 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 747.08 sec) 10:09:00 [GPUGRID] [rr_sim] 2242564.26: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986232412.25G/439.78G) 10:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852145.17 10:09:00 [---] pick_jobs_to_run() start 10:09:00 [---] pick_jobs_to_run() end 10:09:00 [---] [rr_sim] end 10:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:09:00 [---] [cpu_sched_debug] preliminary job list: 10:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:09:00 [---] [cpu_sched_debug] final job list: 10:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:09:00 [---] [cpu_sched_debug] enforce_run_list: end 10:09:00 [---] choose_project(): 36540.000000 10:09:00 [iThena.Measurements] doing RPC to get work 10:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 399059.09 cpu 2144732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:09:00 [---] Got 0 tasks 10:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:10:00 [---] [rr_sim] doing sim: CPU sched 10:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986204821.89G/439.78G) 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242501.52 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 224250.15 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224250.151857 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018251.37 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 201825.14 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426075.288528 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816426.23 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 181642.62 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607717.911532 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634783.61 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 163478.36 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771196.272235 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471305.25 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 147130.52 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918326.796868 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324174.72 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 132417.47 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050744.269038 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191757.25 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 119175.72 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169919.993991 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072581.52 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 107258.15 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277178.146449 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965323.37 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 96532.34 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373710.483660 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868791.03 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 86879.10 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460589.587151 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781911.93 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 78191.19 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538780.780293 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703720.74 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 70372.07 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609152.854120 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633348.66 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 63334.87 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672487.720565 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570013.80 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 57001.38 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729489.100365 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513012.42 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 51301.24 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780790.342185 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461711.18 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 46171.12 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826961.459823 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415540.06 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 41554.01 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868515.465698 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373986.05 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 37398.61 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905914.070985 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336587.45 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 33658.74 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939572.815743 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302928.70 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 30292.87 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969865.686025 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272635.83 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 27263.58 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997129.269279 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245372.25 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 24537.22 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021666.494208 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220835.02 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 22083.50 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043749.996644 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198751.52 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 19875.15 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063625.148836 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178876.37 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 17887.64 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081512.785809 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160988.73 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 16098.87 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097611.659085 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144889.86 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 14488.99 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112100.645033 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130400.87 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 13040.09 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125140.732387 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117360.79 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 11736.08 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136876.811005 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105624.71 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 10562.47 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147439.281761 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95062.24 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 9506.22 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156945.505441 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85556.01 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 8555.60 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165501.106754 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77000.41 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 7700.04 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173201.147935 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69300.37 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 6930.04 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180131.184998 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62370.33 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 6237.03 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186368.218355 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56133.30 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 5613.33 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191981.548376 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50519.97 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 5052.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197033.545395 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45467.97 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 4546.80 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201580.342713 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40921.18 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 4092.12 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205672.460298 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36829.06 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3682.91 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209355.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33146.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212955.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29546.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216555.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25946.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220155.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22346.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223755.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18746.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227355.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15146.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230955.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11546.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234555.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7946.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238155.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4346.15 sec) 10:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241755.366125 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 746.15 sec) 10:10:00 [GPUGRID] [rr_sim] 2242501.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986204821.89G/439.78G) 10:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852142.43 10:10:00 [---] pick_jobs_to_run() start 10:10:00 [---] pick_jobs_to_run() end 10:10:00 [---] [rr_sim] end 10:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:10:00 [---] [cpu_sched_debug] preliminary job list: 10:10:00 [---] [cpu_sched_debug] final job list: 10:10:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 10:10:00 [---] [work_fetch] Request work fetch: CPUs idle 10:10:00 [---] [cpu_sched_debug] enforce_run_list: end 10:10:00 [---] choose_project(): 36600.000000 10:10:00 [iThena.Measurements] doing RPC to get work 10:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398999.09 cpu 2144672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:10:00 [---] Got 0 tasks 10:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:11:00 [---] [rr_sim] doing sim: CPU sched 10:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986204821.89G/439.78G) 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242501.52 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 224250.15 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224250.151857 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018251.37 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 201825.14 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426075.288528 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816426.23 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 181642.62 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607717.911532 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634783.61 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 163478.36 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771196.272235 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471305.25 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 147130.52 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918326.796868 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324174.72 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 132417.47 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050744.269038 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191757.25 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 119175.72 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169919.993991 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072581.52 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 107258.15 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277178.146449 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965323.37 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 96532.34 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373710.483660 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868791.03 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 86879.10 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460589.587151 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781911.93 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 78191.19 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538780.780293 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703720.74 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 70372.07 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609152.854120 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633348.66 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 63334.87 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672487.720565 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570013.80 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 57001.38 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729489.100365 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513012.42 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 51301.24 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780790.342185 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461711.18 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 46171.12 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826961.459823 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415540.06 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 41554.01 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868515.465698 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373986.05 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 37398.61 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905914.070985 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336587.45 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 33658.74 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939572.815743 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302928.70 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 30292.87 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969865.686025 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272635.83 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 27263.58 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997129.269279 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245372.25 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 24537.22 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021666.494208 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220835.02 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 22083.50 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043749.996644 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198751.52 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 19875.15 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063625.148836 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178876.37 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 17887.64 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081512.785809 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160988.73 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 16098.87 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097611.659085 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144889.86 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 14488.99 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112100.645033 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130400.87 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 13040.09 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125140.732387 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117360.79 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 11736.08 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136876.811005 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105624.71 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 10562.47 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147439.281761 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95062.24 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 9506.22 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156945.505441 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85556.01 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 8555.60 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165501.106754 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77000.41 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 7700.04 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173201.147935 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69300.37 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 6930.04 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180131.184998 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62370.33 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 6237.03 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186368.218355 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56133.30 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 5613.33 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191981.548376 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50519.97 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 5052.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197033.545395 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45467.97 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 4546.80 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201580.342713 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40921.18 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 4092.12 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205672.460298 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36829.06 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3682.91 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209355.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33146.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212955.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29546.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216555.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25946.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220155.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22346.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223755.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18746.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227355.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15146.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230955.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11546.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234555.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7946.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238155.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4346.15 sec) 10:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241755.366125 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 746.15 sec) 10:11:00 [GPUGRID] [rr_sim] 2242501.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986204821.89G/439.78G) 10:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852202.43 10:11:00 [---] pick_jobs_to_run() start 10:11:00 [---] pick_jobs_to_run() end 10:11:00 [---] [rr_sim] end 10:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:11:00 [---] [cpu_sched_debug] preliminary job list: 10:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:11:00 [---] [cpu_sched_debug] final job list: 10:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:11:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 986204821.89G 10:11:00 [---] [cpu_sched_debug] enforce_run_list: end 10:11:00 [---] choose_project(): 36660.000000 10:11:00 [iThena.Measurements] doing RPC to get work 10:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398939.09 cpu 2144672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:11:00 [---] Got 0 tasks 10:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:12:00 [---] [rr_sim] doing sim: CPU sched 10:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986177231.54G/439.78G) 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242438.78 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 224243.88 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224243.878168 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018194.90 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 201819.49 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426063.368520 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816375.41 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 181637.54 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607700.909837 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634737.87 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 163473.79 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771174.697021 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471264.08 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 147126.41 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918301.105488 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324137.68 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 132413.77 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050714.873107 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191723.91 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 119172.39 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169887.263965 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072551.52 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 107255.15 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277142.415737 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965296.37 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 96529.64 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373672.052332 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868766.73 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 86876.67 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460548.725267 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781890.06 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 78189.01 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538737.730909 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703701.05 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 70370.11 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609107.835986 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633330.95 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 63333.09 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672440.930556 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569997.85 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 56999.79 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729440.715669 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512998.07 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 51299.81 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780740.522271 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461698.26 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 46169.83 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826910.348212 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415528.43 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 41552.84 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868463.191559 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373975.59 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 37397.56 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905860.750572 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336578.03 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 33657.80 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939518.553683 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302920.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 30292.02 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969810.576483 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272628.21 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 27262.82 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997073.397003 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245365.38 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 24536.54 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021609.935472 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220828.85 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 22082.88 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043692.820093 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198745.96 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 19874.60 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063567.416252 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178871.37 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 17887.14 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081454.552795 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160984.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 16098.42 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097552.975684 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144885.81 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 14488.58 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112041.556284 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130397.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 13039.72 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125081.278824 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117357.50 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 11735.75 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136817.029110 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105621.75 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 10562.18 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147379.204368 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95059.58 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 9505.96 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156885.162099 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85553.62 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 8555.36 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165440.524058 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76998.26 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 7699.83 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173140.349821 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69298.43 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 6929.84 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180070.193007 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62368.59 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 6236.86 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186307.051875 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56131.73 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 5613.17 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191920.224856 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50518.56 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 5051.86 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196972.080539 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45466.70 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 4546.67 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201518.750653 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40920.03 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 4092.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205610.753756 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36828.03 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3682.80 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209293.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33145.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212893.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29545.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216493.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25945.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220093.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22345.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223693.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18745.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227293.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15145.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230893.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11545.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234493.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7945.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238093.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4345.23 sec) 10:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241693.556549 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 745.23 sec) 10:12:00 [GPUGRID] [rr_sim] 2242438.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986177231.54G/439.78G) 10:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852199.69 10:12:00 [---] pick_jobs_to_run() start 10:12:00 [---] pick_jobs_to_run() end 10:12:00 [---] [rr_sim] end 10:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:12:00 [---] [cpu_sched_debug] preliminary job list: 10:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 10:12:00 [---] [cpu_sched_debug] final job list: 10:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:12:00 [---] [cpu_sched_debug] enforce_run_list: end 10:12:00 [---] choose_project(): 36720.000000 10:12:00 [iThena.Measurements] doing RPC to get work 10:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398879.09 cpu 2144612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:12:00 [---] Got 0 tasks 10:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:13:00 [---] [rr_sim] doing sim: CPU sched 10:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986149641.18G/439.78G) 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242376.04 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 224237.60 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224237.604480 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018138.44 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 201813.84 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426051.448512 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816324.60 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 181632.46 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607683.908141 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634692.14 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 163469.21 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771153.121808 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471222.92 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 147122.29 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918275.414107 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324100.63 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 132410.06 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050685.477177 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191690.57 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 119169.06 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169854.533939 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072521.51 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 107252.15 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277106.685026 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965269.36 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 96526.94 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373633.621003 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868742.42 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 86874.24 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460507.863383 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781868.18 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 78186.82 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538694.681525 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703681.36 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 70368.14 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609062.817853 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633313.23 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 63331.32 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672394.140548 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569981.90 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 56998.19 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729392.330973 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512983.71 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 51298.37 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780690.702356 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461685.34 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 46168.53 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826859.236601 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415516.81 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 41551.68 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868410.917421 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373965.13 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 37396.51 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905807.430159 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336568.61 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 33656.86 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939464.291623 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302911.75 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 30291.18 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969755.466941 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272620.58 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 27262.06 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997017.524727 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245358.52 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 24535.85 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021553.376735 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220822.67 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 22082.27 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043635.643542 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198740.40 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 19874.04 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063509.683668 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178866.36 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 17886.64 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081396.319781 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160979.73 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 16097.97 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097494.292283 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144881.75 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 14488.18 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111982.467535 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130393.58 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 13039.36 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125021.825262 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117354.22 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 11735.42 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136757.247216 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105618.80 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 10561.88 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147319.126975 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95056.92 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 9505.69 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156824.818757 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85551.23 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 8555.12 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165379.941362 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76996.10 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 7699.61 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173079.551706 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69296.49 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 6929.65 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180009.201016 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62366.84 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 6236.68 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186245.885394 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56130.16 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 5613.02 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191858.901335 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50517.14 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 5051.71 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196910.615682 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45465.43 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 4546.54 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201457.158594 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40918.89 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 4091.89 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205549.047215 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36827.00 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3682.70 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209231.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33144.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212831.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29544.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216431.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25944.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220031.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22344.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223631.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18744.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227231.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15144.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230831.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11544.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234431.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7944.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238031.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4344.30 sec) 10:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241631.746974 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 744.30 sec) 10:13:00 [GPUGRID] [rr_sim] 2242376.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986149641.18G/439.78G) 10:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852196.96 10:13:00 [---] pick_jobs_to_run() start 10:13:00 [---] pick_jobs_to_run() end 10:13:00 [---] [rr_sim] end 10:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:13:00 [---] [cpu_sched_debug] preliminary job list: 10:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:13:00 [---] [cpu_sched_debug] final job list: 10:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:13:00 [---] [cpu_sched_debug] enforce_run_list: end 10:13:00 [---] choose_project(): 36780.000000 10:13:00 [iThena.Measurements] doing RPC to get work 10:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398819.09 cpu 2144552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:13:00 [---] Got 0 tasks 10:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:14:00 [---] [rr_sim] doing sim: CPU sched 10:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986122050.83G/439.78G) 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242313.31 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 224231.33 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224231.330792 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018081.98 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 201808.20 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426039.528505 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816273.78 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 181627.38 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607666.906446 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634646.40 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 163464.64 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771131.546594 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471181.76 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 147118.18 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918249.722726 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324063.59 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 132406.36 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050656.081246 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191657.23 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 119165.72 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169821.803913 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072491.50 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 107249.15 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277070.954314 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965242.35 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 96524.24 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373595.189675 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868718.12 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 86871.81 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460467.001499 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781846.31 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 78184.63 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538651.632141 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703661.68 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 70366.17 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1609017.799719 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633295.51 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 63329.55 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672347.350539 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569965.96 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 56996.60 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729343.946277 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512969.36 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 51296.94 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780640.882442 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461672.43 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 46167.24 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826808.124990 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415505.18 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 41550.52 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868358.643283 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373954.66 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 37395.47 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905754.109746 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336559.20 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 33655.92 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939410.029564 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302903.28 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 30290.33 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969700.357399 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272612.95 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 27261.30 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996961.652452 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245351.66 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 24535.17 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021496.817998 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220816.49 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 22081.65 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043578.466991 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198734.84 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 19873.48 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063451.951084 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178861.36 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 17886.14 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081338.086767 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160975.22 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 16097.52 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097435.608883 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144877.70 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 14487.77 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111923.378786 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130389.93 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 13038.99 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124962.371700 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117350.94 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 11735.09 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136697.465322 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105615.84 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 10561.58 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147259.049582 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95054.26 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 9505.43 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156764.475416 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85548.83 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 8554.88 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165319.358666 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76993.95 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 7699.39 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173018.753591 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69294.55 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 6929.46 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179948.209024 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62365.10 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 6236.51 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186184.718914 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56128.59 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 5612.86 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191797.577815 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50515.73 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 5051.57 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196849.150825 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45464.16 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 4546.42 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201395.566535 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40917.74 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 4091.77 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205487.340673 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36825.97 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3682.60 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209169.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33143.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212769.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29543.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216369.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25943.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219969.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22343.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223569.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18743.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227169.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15143.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230769.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11543.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234369.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7943.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237969.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4343.37 sec) 10:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241569.937398 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 743.37 sec) 10:14:00 [GPUGRID] [rr_sim] 2242313.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986122050.83G/439.78G) 10:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852194.22 10:14:00 [---] pick_jobs_to_run() start 10:14:00 [---] pick_jobs_to_run() end 10:14:00 [---] [rr_sim] end 10:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:14:00 [---] [cpu_sched_debug] preliminary job list: 10:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:14:00 [---] [cpu_sched_debug] final job list: 10:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:14:00 [---] [cpu_sched_debug] enforce_run_list: end 10:14:00 [---] choose_project(): 36840.000000 10:14:00 [iThena.Measurements] doing RPC to get work 10:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398759.09 cpu 2144492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:14:00 [---] Got 0 tasks 10:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:15:00 [---] [rr_sim] doing sim: CPU sched 10:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986094460.47G/439.78G) 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242250.57 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 224225.06 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224225.057104 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2018025.51 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 201802.55 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426027.608497 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816222.96 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 181622.30 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607649.904751 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634600.67 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 163460.07 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771109.971380 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471140.60 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 147114.06 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918224.031346 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1324026.54 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 132402.65 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050626.685315 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191623.89 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 119162.39 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169789.073887 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072461.50 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 107246.15 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1277035.223602 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965215.35 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 96521.53 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373556.758346 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868693.81 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 86869.38 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460426.139615 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781824.43 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 78182.44 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538608.582758 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703641.99 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 70364.20 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608972.781586 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633277.79 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 63327.78 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672300.560531 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569950.01 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 56995.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729295.561582 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512955.01 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 51295.50 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780591.062527 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461659.51 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 46165.95 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826757.013378 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415493.56 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 41549.36 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868306.369144 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373944.20 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 37394.42 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905700.789334 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336549.78 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 33654.98 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939355.767504 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302894.80 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 30289.48 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969645.247858 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272605.32 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 27260.53 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996905.780176 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245344.79 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 24534.48 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021440.259262 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220810.31 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 22081.03 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043521.290439 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198729.28 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 19872.93 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063394.218499 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178856.35 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 17885.64 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081279.853753 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160970.72 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 16097.07 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097376.925482 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144873.65 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 14487.36 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111864.290037 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130386.28 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 13038.63 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124902.918137 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117347.65 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 11734.77 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136637.683428 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105612.89 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 10561.29 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147198.972189 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95051.60 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 9505.16 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156704.132074 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85546.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 8554.64 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165258.775970 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76991.80 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 7699.18 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172957.955477 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69292.62 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 6929.26 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179887.217033 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62363.35 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 6236.34 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186123.552433 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56127.02 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 5612.70 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191736.254294 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50514.32 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 5051.43 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196787.685968 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45462.89 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 4546.29 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201333.974475 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40916.60 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 4091.66 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205425.634132 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36824.94 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3682.49 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209108.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33142.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212708.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29542.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216308.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25942.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219908.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22342.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223508.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18742.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227108.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15142.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230708.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11542.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234308.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7942.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237908.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4342.44 sec) 10:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241508.127822 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 742.44 sec) 10:15:00 [GPUGRID] [rr_sim] 2242250.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986094460.47G/439.78G) 10:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852191.48 10:15:00 [---] pick_jobs_to_run() start 10:15:00 [---] pick_jobs_to_run() end 10:15:00 [---] [rr_sim] end 10:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:15:00 [---] [cpu_sched_debug] preliminary job list: 10:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:15:00 [---] [cpu_sched_debug] final job list: 10:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:15:00 [---] [cpu_sched_debug] enforce_run_list: end 10:15:00 [---] choose_project(): 36900.000000 10:15:00 [iThena.Measurements] doing RPC to get work 10:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398699.09 cpu 2144432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:15:00 [---] Got 0 tasks 10:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:16:00 [---] [rr_sim] doing sim: CPU sched 10:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986066870.11G/439.78G) 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242187.83 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 224218.78 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224218.783416 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017969.05 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 201796.91 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426015.688490 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816172.15 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 181617.21 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607632.903056 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634554.93 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 163455.49 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771088.396166 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471099.44 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 147109.94 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918198.339965 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323989.49 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 132398.95 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050597.289384 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191590.54 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 119159.05 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169756.343861 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072431.49 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 107243.15 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276999.492891 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965188.34 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 96518.83 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373518.327017 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868669.51 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 86866.95 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460385.277731 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781802.56 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 78180.26 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538565.533374 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703622.30 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 70362.23 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608927.763452 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633260.07 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 63326.01 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672253.770522 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569934.06 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 56993.41 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729247.176886 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512940.66 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 51294.07 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780541.242613 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461646.59 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 46164.66 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826705.901767 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415481.93 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 41548.19 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868254.095006 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373933.74 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 37393.37 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905647.468921 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336540.37 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 33654.04 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939301.505444 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302886.33 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 30288.63 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969590.138316 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272597.70 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 27259.77 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996849.907900 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245337.93 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 24533.79 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021383.700525 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220804.13 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 22080.41 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043464.113888 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198723.72 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 19872.37 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063336.485915 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178851.35 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 17885.13 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081221.620739 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160966.21 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 16096.62 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097318.242081 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144869.59 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 14486.96 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111805.201288 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130382.63 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 13038.26 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124843.464575 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117344.37 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 11734.44 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136577.901533 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105609.93 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 10560.99 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147138.894796 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95048.94 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 9504.89 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156643.788732 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85544.05 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 8554.40 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165198.193274 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76989.64 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 7698.96 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172897.157362 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69290.68 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 6929.07 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179826.225042 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62361.61 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 6236.16 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186062.385953 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56125.45 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 5612.54 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191674.930773 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50512.90 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 5051.29 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196726.221112 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45461.61 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 4546.16 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201272.382416 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40915.45 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 4091.55 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205363.927590 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36823.91 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3682.39 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209046.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33141.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212646.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29541.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216246.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25941.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219846.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22341.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223446.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18741.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227046.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15141.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230646.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11541.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234246.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7941.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237846.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4341.52 sec) 10:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241446.318247 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 741.52 sec) 10:16:00 [GPUGRID] [rr_sim] 2242187.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986066870.11G/439.78G) 10:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852188.75 10:16:00 [---] pick_jobs_to_run() start 10:16:00 [---] pick_jobs_to_run() end 10:16:00 [---] [rr_sim] end 10:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:16:00 [---] [cpu_sched_debug] preliminary job list: 10:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:16:00 [---] [cpu_sched_debug] final job list: 10:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:16:00 [---] [cpu_sched_debug] enforce_run_list: end 10:16:00 [---] choose_project(): 36960.000000 10:16:00 [iThena.Measurements] doing RPC to get work 10:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398639.09 cpu 2144372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:16:00 [---] Got 0 tasks 10:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:17:00 [---] [rr_sim] doing sim: CPU sched 10:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986039279.76G/439.78G) 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242125.10 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 224212.51 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224212.509727 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017912.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 201791.26 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 426003.768482 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816121.33 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 181612.13 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607615.901361 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634509.20 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 163450.92 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771066.820952 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471058.28 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 147105.83 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918172.648585 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323952.45 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 132395.24 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050567.893454 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191557.20 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 119155.72 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169723.613836 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072401.48 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 107240.15 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276963.762179 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965161.34 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 96516.13 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373479.895689 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868645.20 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 86864.52 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460344.415847 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781780.68 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 78178.07 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538522.483990 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703602.61 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 70360.26 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608882.745318 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633242.35 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 63324.24 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672206.980514 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569918.12 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 56991.81 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729198.792190 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512926.31 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 51292.63 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780491.422698 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461633.67 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 46163.37 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826654.790156 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415470.31 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 41547.03 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868201.820868 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373923.28 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 37392.33 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905594.148508 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336530.95 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 33653.09 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939247.243385 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302877.85 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 30287.79 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969535.028774 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272590.07 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 27259.01 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996794.035624 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245331.06 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 24533.11 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021327.141789 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220797.96 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 22079.80 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043406.937337 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198718.16 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 19871.82 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063278.753331 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178846.34 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 17884.63 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081163.387725 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160961.71 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 16096.17 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097259.558680 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144865.54 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 14486.55 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111746.112539 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130378.98 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 13037.90 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124784.011013 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117341.09 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 11734.11 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136518.119639 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105606.98 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 10560.70 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147078.817402 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95046.28 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 9504.63 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156583.445390 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85541.65 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 8554.17 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165137.610578 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76987.49 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 7698.75 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172836.359248 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69288.74 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 6928.87 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179765.233050 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62359.86 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 6235.99 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186001.219473 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56123.88 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 5612.39 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191613.607253 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50511.49 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 5051.15 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196664.756255 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45460.34 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 4546.03 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201210.790357 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40914.31 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 4091.43 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205302.221048 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36822.88 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3682.29 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208984.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33140.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212584.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29540.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216184.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25940.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219784.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22340.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223384.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18740.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226984.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15140.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230584.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11540.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234184.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7940.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237784.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4340.59 sec) 10:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241384.508671 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 740.59 sec) 10:17:00 [GPUGRID] [rr_sim] 2242125.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986039279.76G/439.78G) 10:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852186.01 10:17:00 [---] pick_jobs_to_run() start 10:17:00 [---] pick_jobs_to_run() end 10:17:00 [---] [rr_sim] end 10:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:17:00 [---] [cpu_sched_debug] preliminary job list: 10:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:17:00 [---] [cpu_sched_debug] final job list: 10:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:17:00 [---] [cpu_sched_debug] enforce_run_list: end 10:17:00 [---] choose_project(): 37020.000000 10:17:00 [iThena.Measurements] doing RPC to get work 10:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398579.09 cpu 2144312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:17:00 [---] Got 0 tasks 10:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:18:00 [---] [rr_sim] doing sim: CPU sched 10:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (986011689.40G/439.78G) 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2242062.36 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 224206.24 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224206.236039 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017856.12 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 201785.61 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425991.848474 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816070.51 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 181607.05 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607598.899666 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634463.46 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 163446.35 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771045.245739 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1471017.11 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 147101.71 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918146.957204 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323915.40 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 132391.54 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050538.497523 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191523.86 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 119152.39 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169690.883810 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072371.48 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 107237.15 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276928.031468 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965134.33 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 96513.43 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373441.464360 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868620.90 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 86862.09 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460303.553963 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781758.81 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 78175.88 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538479.434606 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703582.93 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 70358.29 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608837.727185 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633224.63 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 63322.46 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672160.190505 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569902.17 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 56990.22 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729150.407494 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512911.95 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 51291.20 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780441.602784 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461620.76 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 46162.08 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826603.678545 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415458.68 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 41545.87 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868149.546729 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373912.81 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 37391.28 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905540.828096 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336521.53 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 33652.15 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939192.981325 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302869.38 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 30286.94 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969479.919232 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272582.44 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 27258.24 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996738.163348 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245324.20 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 24532.42 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021270.583052 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220791.78 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 22079.18 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043349.760786 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198712.60 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 19871.26 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063221.020747 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178841.34 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 17884.13 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081105.154711 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160957.21 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 16095.72 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097200.875279 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144861.49 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 14486.15 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111687.023790 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130375.34 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 13037.53 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124724.557451 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117337.80 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 11733.78 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136458.337745 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105604.02 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 10560.40 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147018.740009 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95043.62 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 9504.36 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156523.102048 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85539.26 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 8553.93 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165077.027882 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76985.33 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 7698.53 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172775.561133 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69286.80 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 6928.68 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179704.241059 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62358.12 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 6235.81 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185940.052992 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56122.31 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 5612.23 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191552.283732 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50510.08 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 5051.01 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196603.291398 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45459.07 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 4545.91 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201149.198297 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40913.16 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 4091.32 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205240.514507 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36821.85 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3682.18 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208922.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33139.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212522.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29539.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216122.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25939.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219722.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22339.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223322.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18739.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226922.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15139.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230522.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11539.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234122.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7939.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237722.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4339.66 sec) 10:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241322.699095 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 739.66 sec) 10:18:00 [GPUGRID] [rr_sim] 2242062.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (986011689.40G/439.78G) 10:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852183.27 10:18:00 [---] pick_jobs_to_run() start 10:18:00 [---] pick_jobs_to_run() end 10:18:00 [---] [rr_sim] end 10:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:18:00 [---] [cpu_sched_debug] preliminary job list: 10:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:18:00 [---] [cpu_sched_debug] final job list: 10:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:18:00 [---] [cpu_sched_debug] enforce_run_list: end 10:18:00 [---] choose_project(): 37080.000000 10:18:00 [iThena.Measurements] doing RPC to get work 10:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398519.09 cpu 2144252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:18:00 [---] Got 0 tasks 10:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:19:00 [---] [rr_sim] doing sim: CPU sched 10:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985984099.04G/439.78G) 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241999.62 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 224199.96 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224199.962351 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017799.66 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 201779.97 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425979.928467 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1816019.70 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 181601.97 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607581.897971 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634417.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 163441.77 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771023.670525 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470975.95 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 147097.60 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918121.265823 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323878.36 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 132387.84 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050509.101592 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191490.52 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 119149.05 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169658.153784 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072341.47 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 107234.15 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276892.300756 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965107.32 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 96510.73 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373403.033032 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868596.59 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 86859.66 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460262.692079 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781736.93 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 78173.69 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538436.385222 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703563.24 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 70356.32 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608792.709051 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633206.91 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 63320.69 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672113.400497 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569886.22 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 56988.62 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729102.022798 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512897.60 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 51289.76 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780391.782869 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461607.84 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 46160.78 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826552.566933 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415447.06 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 41544.71 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868097.272591 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373902.35 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 37390.24 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905487.507683 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336512.12 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 33651.21 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939138.719265 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302860.90 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 30286.09 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969424.809690 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272574.81 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 27257.48 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996682.291072 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245317.33 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 24531.73 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021214.024316 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220785.60 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 22078.56 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043292.584235 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198707.04 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 19870.70 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063163.288162 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178836.34 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 17883.63 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081046.921697 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160952.70 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 16095.27 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097142.191878 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144857.43 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 14485.74 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111627.935041 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130371.69 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 13037.17 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124665.103888 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117334.52 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 11733.45 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136398.555850 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105601.07 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 10560.11 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146958.662616 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95040.96 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 9504.10 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156462.758706 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85536.86 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 8553.69 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165016.445186 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76983.18 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 7698.32 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172714.763018 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69284.86 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 6928.49 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179643.249067 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62356.37 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 6235.64 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185878.886512 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56120.74 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 5612.07 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191490.960211 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50508.66 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 5050.87 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196541.826541 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45457.80 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 4545.78 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201087.606238 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40912.02 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 4091.20 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205178.807965 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36820.82 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3682.08 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208860.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33138.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212460.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29538.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216060.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25938.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219660.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22338.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223260.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18738.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226860.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15138.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230460.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11538.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234060.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7938.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237660.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4338.73 sec) 10:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241260.889520 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 738.73 sec) 10:19:00 [GPUGRID] [rr_sim] 2241999.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985984099.04G/439.78G) 10:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852180.54 10:19:00 [---] pick_jobs_to_run() start 10:19:00 [---] pick_jobs_to_run() end 10:19:00 [---] [rr_sim] end 10:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:19:00 [---] [cpu_sched_debug] preliminary job list: 10:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:19:00 [---] [cpu_sched_debug] final job list: 10:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:19:00 [---] [cpu_sched_debug] enforce_run_list: end 10:19:00 [---] choose_project(): 37140.000000 10:19:00 [iThena.Measurements] doing RPC to get work 10:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398459.09 cpu 2144192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:19:00 [---] Got 0 tasks 10:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:20:00 [---] [rr_sim] doing sim: CPU sched 10:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985956508.69G/439.78G) 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241936.89 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 224193.69 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224193.688663 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017743.20 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 201774.32 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425968.008459 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815968.88 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 181596.89 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607564.896276 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634371.99 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 163437.20 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 771002.095311 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470934.79 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 147093.48 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918095.574443 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323841.31 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 132384.13 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050479.705661 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191457.18 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 119145.72 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169625.423758 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072311.46 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 107231.15 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276856.570045 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965080.32 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 96508.03 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373364.601703 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868572.28 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 86857.23 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460221.830195 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781715.06 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 78171.51 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538393.335839 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703543.55 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 70354.36 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608747.690918 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633189.20 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 63318.92 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672066.610488 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569870.28 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 56987.03 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729053.638102 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512883.25 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 51288.32 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780341.962955 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461594.92 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 46159.49 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826501.455322 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415435.43 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 41543.54 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868044.998453 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373891.89 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 37389.19 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905434.187270 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336502.70 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 33650.27 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939084.457206 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302852.43 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 30285.24 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969369.700148 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272567.19 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 27256.72 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996626.418796 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245310.47 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 24531.05 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021157.465579 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220779.42 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 22077.94 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043235.407684 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198701.48 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 19870.15 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063105.555578 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178831.33 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 17883.13 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080988.688683 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160948.20 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 16094.82 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097083.508478 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144853.38 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 14485.34 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111568.846293 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130368.04 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 13036.80 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124605.650326 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117331.24 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 11733.12 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136338.773956 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105598.11 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 10559.81 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146898.585223 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95038.30 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 9503.83 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156402.415364 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85534.47 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 8553.45 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164955.862490 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76981.02 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 7698.10 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172653.964904 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69282.92 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 6928.29 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179582.257076 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62354.63 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 6235.46 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185817.720031 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56119.17 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 5611.92 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191429.636691 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50507.25 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 5050.72 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196480.361684 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45456.52 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 4545.65 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201026.014179 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40910.87 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 4091.09 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205117.101424 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36819.79 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3681.98 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208799.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33137.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212399.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29537.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215999.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25937.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219599.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22337.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223199.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18737.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226799.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15137.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230399.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11537.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233999.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7937.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237599.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4337.81 sec) 10:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241199.079944 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 737.81 sec) 10:20:00 [GPUGRID] [rr_sim] 2241936.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985956508.69G/439.78G) 10:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852177.80 10:20:00 [---] pick_jobs_to_run() start 10:20:00 [---] pick_jobs_to_run() end 10:20:00 [---] [rr_sim] end 10:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:20:00 [---] [cpu_sched_debug] preliminary job list: 10:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:20:00 [---] [cpu_sched_debug] final job list: 10:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:20:00 [---] [cpu_sched_debug] enforce_run_list: end 10:20:00 [---] choose_project(): 37200.000000 10:20:00 [iThena.Measurements] doing RPC to get work 10:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398399.09 cpu 2144132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:20:00 [---] Got 0 tasks 10:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:21:00 [---] [rr_sim] doing sim: CPU sched 10:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985928918.33G/439.78G) 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241874.15 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 224187.41 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224187.414975 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017686.73 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 201768.67 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425956.088452 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815918.06 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 181591.81 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607547.894581 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634326.26 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 163432.63 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770980.520097 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470893.63 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 147089.36 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918069.883062 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323804.27 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 132380.43 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050450.309730 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191423.84 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 119142.38 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169592.693732 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072281.46 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 107228.15 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276820.839333 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965053.31 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 96505.33 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373326.170374 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868547.98 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 86854.80 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460180.968312 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781693.18 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 78169.32 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538350.286455 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703523.86 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 70352.39 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608702.672784 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633171.48 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 63317.15 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1672019.820480 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569854.33 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 56985.43 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1729005.253407 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512868.90 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 51286.89 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780292.143040 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461582.01 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 46158.20 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826450.343711 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415423.81 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 41542.38 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867992.724314 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373881.43 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 37388.14 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905380.866857 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336493.28 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 33649.33 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939030.195146 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302843.95 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 30284.40 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969314.590606 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272559.56 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 27255.96 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996570.546520 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245303.60 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 24530.36 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021100.906842 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220773.24 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 22077.32 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043178.231133 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198695.92 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 19869.59 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063047.822994 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178826.33 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 17882.63 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080930.455669 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160943.69 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 16094.37 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097024.825077 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144849.32 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 14484.93 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111509.757544 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130364.39 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 13036.44 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124546.196764 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117327.95 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 11732.80 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136278.992062 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105595.16 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 10559.52 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146838.507830 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95035.64 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 9503.56 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156342.072022 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85532.08 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 8553.21 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164895.279794 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76978.87 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 7697.89 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172593.166789 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69280.98 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 6928.10 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179521.265085 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62352.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 6235.29 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185756.553551 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56117.60 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 5611.76 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191368.313170 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50505.84 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 5050.58 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196418.896828 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45455.25 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 4545.53 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200964.422119 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40909.73 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 4090.97 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205055.394882 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36818.75 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3681.88 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208737.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33136.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212337.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29536.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215937.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25936.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219537.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22336.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223137.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18736.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226737.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15136.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230337.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11536.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233937.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7936.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237537.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4336.88 sec) 10:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241137.270368 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 736.88 sec) 10:21:00 [GPUGRID] [rr_sim] 2241874.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985928918.33G/439.78G) 10:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852175.06 10:21:00 [---] pick_jobs_to_run() start 10:21:00 [---] pick_jobs_to_run() end 10:21:00 [---] [rr_sim] end 10:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:21:00 [---] [cpu_sched_debug] preliminary job list: 10:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:21:00 [---] [cpu_sched_debug] final job list: 10:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:21:00 [---] [cpu_sched_debug] enforce_run_list: end 10:21:00 [---] choose_project(): 37260.000000 10:21:00 [iThena.Measurements] doing RPC to get work 10:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398339.09 cpu 2144072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:21:00 [---] Got 0 tasks 10:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:23:00 [---] [rr_sim] doing sim: CPU sched 10:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985901327.98G/439.78G) 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241811.41 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 224181.14 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224181.141286 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017630.27 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 201763.03 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425944.168444 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815867.24 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 181586.72 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607530.892886 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634280.52 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 163428.05 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770958.944884 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470852.47 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 147085.25 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918044.191682 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323767.22 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 132376.72 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050420.913800 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191390.50 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 119139.05 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169559.963706 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072251.45 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 107225.14 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276785.108622 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 965026.30 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 96502.63 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373287.739046 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868523.67 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 86852.37 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460140.106428 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781671.31 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 78167.13 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538307.237071 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703504.18 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 70350.42 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608657.654650 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633153.76 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 63315.38 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671973.030472 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569838.38 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 56983.84 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728956.868711 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512854.54 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 51285.45 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780242.323126 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461569.09 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 46156.91 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826399.232100 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415412.18 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 41541.22 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867940.450176 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373870.96 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 37387.10 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905327.546445 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336483.87 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 33648.39 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938975.933087 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302835.48 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 30283.55 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969259.481064 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272551.93 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 27255.19 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996514.674244 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245296.74 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 24529.67 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021044.348106 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220767.06 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 22076.71 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043121.054582 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198690.36 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 19869.04 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062990.090410 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178821.32 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 17882.13 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080872.222655 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160939.19 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 16093.92 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096966.141676 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144845.27 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 14484.53 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111450.668795 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130360.74 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 13036.07 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124486.743201 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117324.67 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 11732.47 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136219.210168 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105592.20 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 10559.22 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146778.430437 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95032.98 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 9503.30 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156281.728680 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85529.68 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 8552.97 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164834.697098 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76976.72 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 7697.67 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172532.368675 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69279.04 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 6927.90 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179460.273093 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62351.14 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 6235.11 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185695.387070 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56116.03 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 5611.60 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191306.989650 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50504.42 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 5050.44 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196357.431971 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45453.98 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 4545.40 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200902.830060 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40908.58 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 4090.86 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204993.688340 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36817.72 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3681.77 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208675.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33135.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212275.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29535.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215875.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25935.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219475.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22335.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223075.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18735.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226675.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15135.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230275.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11535.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233875.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7935.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237475.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4335.95 sec) 10:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241075.460793 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 735.95 sec) 10:23:00 [GPUGRID] [rr_sim] 2241811.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985901327.98G/439.78G) 10:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852232.33 10:23:00 [---] pick_jobs_to_run() start 10:23:00 [---] pick_jobs_to_run() end 10:23:00 [---] [rr_sim] end 10:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:23:00 [---] [cpu_sched_debug] preliminary job list: 10:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:23:00 [---] [cpu_sched_debug] final job list: 10:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:23:00 [---] [cpu_sched_debug] enforce_run_list: end 10:23:00 [---] choose_project(): 37380.000000 10:23:00 [iThena.Measurements] doing RPC to get work 10:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 571019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398219.09 cpu 2144012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2144012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2144012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:23:00 [---] Got 0 tasks 10:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:24:00 [---] [rr_sim] doing sim: CPU sched 10:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985873737.62G/439.78G) 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241748.68 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 224174.87 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224174.867598 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017573.81 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 201757.38 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425932.248436 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815816.43 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 181581.64 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607513.891191 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634234.78 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 163423.48 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770937.369670 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470811.31 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 147081.13 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 918018.500301 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323730.18 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 132373.02 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050391.517869 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191357.16 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 119135.72 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169527.233680 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072221.44 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 107222.14 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276749.377910 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964999.30 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 96499.93 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373249.307717 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868499.37 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 86849.94 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460099.244544 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781649.43 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 78164.94 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538264.187687 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703484.49 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 70348.45 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608612.636517 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633136.04 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 63313.60 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671926.240463 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569822.44 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 56982.24 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728908.484015 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512840.19 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 51284.02 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780192.503211 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461556.17 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 46155.62 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826348.120488 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415400.56 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 41540.06 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867888.176038 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373860.50 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 37386.05 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905274.226032 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336474.45 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 33647.44 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938921.671027 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302827.00 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 30282.70 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969204.371522 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272544.30 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 27254.43 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996458.801968 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245289.87 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 24528.99 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020987.789369 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220760.89 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 22076.09 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043063.878031 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198684.80 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 19868.48 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062932.357826 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178816.32 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 17881.63 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080813.989641 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160934.69 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 16093.47 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096907.458275 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144841.22 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 14484.12 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111391.580046 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130357.10 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 13035.71 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124427.289639 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117321.39 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 11732.14 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136159.428273 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105589.25 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 10558.92 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146718.353044 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95030.32 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 9503.03 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156221.385338 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85527.29 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 8552.73 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164774.114402 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76974.56 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 7697.46 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172471.570560 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69277.11 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 6927.71 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179399.281102 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62349.39 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 6234.94 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185634.220590 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56114.46 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 5611.45 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191245.666129 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50503.01 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 5050.30 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196295.967114 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45452.71 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 4545.27 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200841.238001 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40907.44 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 4090.74 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204931.981799 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36816.69 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3681.67 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208613.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33135.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212213.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29535.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215813.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25935.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219413.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22335.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223013.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18735.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226613.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15135.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230213.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11535.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233813.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7935.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237413.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4335.02 sec) 10:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2241013.651217 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 735.02 sec) 10:24:00 [GPUGRID] [rr_sim] 2241748.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985873737.62G/439.78G) 10:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852229.59 10:24:00 [---] pick_jobs_to_run() start 10:24:00 [---] pick_jobs_to_run() end 10:24:00 [---] [rr_sim] end 10:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:24:00 [---] [cpu_sched_debug] preliminary job list: 10:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:24:00 [---] [cpu_sched_debug] final job list: 10:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:24:00 [---] [cpu_sched_debug] enforce_run_list: end 10:24:00 [---] choose_project(): 37440.000000 10:24:00 [iThena.Measurements] doing RPC to get work 10:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398159.09 cpu 2143952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:24:00 [---] Got 0 tasks 10:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:25:00 [---] [rr_sim] doing sim: CPU sched 10:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985846147.26G/439.78G) 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241685.94 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 224168.59 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224168.593910 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017517.35 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 201751.73 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425920.328429 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815765.61 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 181576.56 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607496.889496 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634189.05 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 163418.90 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770915.794456 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470770.14 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 147077.01 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917992.808920 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323693.13 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 132369.31 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050362.121938 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191323.82 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 119132.38 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169494.503654 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072191.44 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 107219.14 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276713.647199 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964972.29 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 96497.23 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373210.876389 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868475.06 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 86847.51 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460058.382660 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781627.56 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 78162.76 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538221.138304 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703464.80 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 70346.48 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608567.618383 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633118.32 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 63311.83 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671879.450455 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569806.49 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 56980.65 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728860.099319 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512825.84 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 51282.58 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780142.683297 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461543.26 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 46154.33 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826297.008877 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415388.93 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 41538.89 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867835.901899 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373850.04 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 37385.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905220.905619 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336465.03 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 33646.50 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938867.408967 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302818.53 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 30281.85 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969149.261980 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272536.68 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 27253.67 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996402.929692 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245283.01 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 24528.30 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020931.230633 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220754.71 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 22075.47 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043006.701479 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198679.24 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 19867.92 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062874.625241 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178811.31 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 17881.13 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080755.756627 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160930.18 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 16093.02 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096848.774874 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144837.16 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 14483.72 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111332.491297 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130353.45 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 13035.34 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124367.836077 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117318.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 11731.81 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136099.646379 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105586.29 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 10558.63 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146658.275651 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95027.66 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 9502.77 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156161.041996 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85524.90 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 8552.49 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164713.531706 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76972.41 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 7697.24 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172410.772445 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69275.17 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 6927.52 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179338.289111 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62347.65 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 6234.76 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185573.054109 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56112.88 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 5611.29 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191184.342608 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50501.60 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 5050.16 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196234.502257 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45451.44 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 4545.14 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200779.645942 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40906.29 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 4090.63 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204870.275257 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36815.66 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3681.57 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208551.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33134.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212151.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29534.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215751.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25934.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219351.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22334.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222951.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18734.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226551.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15134.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230151.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11534.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233751.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7934.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237351.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4334.10 sec) 10:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240951.841641 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 734.10 sec) 10:25:00 [GPUGRID] [rr_sim] 2241685.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985846147.26G/439.78G) 10:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852226.85 10:25:00 [---] pick_jobs_to_run() start 10:25:00 [---] pick_jobs_to_run() end 10:25:00 [---] [rr_sim] end 10:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:25:00 [---] [cpu_sched_debug] preliminary job list: 10:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:25:00 [---] [cpu_sched_debug] final job list: 10:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:25:00 [---] [cpu_sched_debug] enforce_run_list: end 10:25:00 [---] choose_project(): 37500.000000 10:25:00 [iThena.Measurements] doing RPC to get work 10:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398099.09 cpu 2143892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:25:00 [---] Got 0 tasks 10:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:26:00 [---] [rr_sim] doing sim: CPU sched 10:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985818556.91G/439.78G) 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241623.20 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 224162.32 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224162.320222 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017460.88 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 201746.09 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425908.408421 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815714.79 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 181571.48 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607479.887801 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634143.31 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 163414.33 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770894.219242 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470728.98 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 147072.90 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917967.117540 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323656.08 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 132365.61 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050332.726007 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191290.48 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 119129.05 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169461.773628 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072161.43 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 107216.14 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276677.916487 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964945.29 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 96494.53 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373172.445060 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868450.76 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 86845.08 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1460017.520776 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781605.68 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 78160.57 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538178.088920 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703445.11 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 70344.51 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608522.600249 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633100.60 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 63310.06 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671832.660446 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569790.54 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 56979.05 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728811.714623 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512811.49 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 51281.15 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780092.863383 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461530.34 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 46153.03 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826245.897266 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415377.30 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 41537.73 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867783.627761 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373839.57 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 37383.96 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905167.585207 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336455.62 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 33645.56 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938813.146908 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302810.06 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 30281.01 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969094.152438 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272529.05 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 27252.90 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996347.057416 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245276.14 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 24527.61 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020874.671896 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220748.53 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 22074.85 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042949.524928 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198673.68 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 19867.37 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062816.892657 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178806.31 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 17880.63 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080697.523613 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160925.68 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 16092.57 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096790.091473 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144833.11 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 14483.31 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111273.402548 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130349.80 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 13034.98 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124308.382515 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117314.82 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 11731.48 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136039.864485 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105583.34 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 10558.33 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146598.198258 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95025.00 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 9502.50 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156100.698654 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85522.50 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 8552.25 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164652.949010 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76970.25 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 7697.03 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172349.974331 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69273.23 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 6927.32 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179277.297119 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62345.91 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 6234.59 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185511.887629 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56111.31 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 5611.13 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191123.019088 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50500.18 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 5050.02 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196173.037401 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45450.16 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 4545.02 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200718.053882 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40905.15 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 4090.51 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204808.568716 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36814.63 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3681.46 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208490.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33133.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212090.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29533.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215690.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25933.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219290.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22333.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222890.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18733.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226490.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15133.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230090.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11533.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233690.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7933.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237290.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4333.17 sec) 10:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240890.032066 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 733.17 sec) 10:26:00 [GPUGRID] [rr_sim] 2241623.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985818556.91G/439.78G) 10:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852224.11 10:26:00 [---] pick_jobs_to_run() start 10:26:00 [---] pick_jobs_to_run() end 10:26:00 [---] [rr_sim] end 10:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:26:00 [---] [cpu_sched_debug] preliminary job list: 10:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:26:00 [---] [cpu_sched_debug] final job list: 10:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:26:00 [---] [cpu_sched_debug] enforce_run_list: end 10:26:00 [---] choose_project(): 37560.000000 10:26:00 [iThena.Measurements] doing RPC to get work 10:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 398039.09 cpu 2143832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:26:00 [---] Got 0 tasks 10:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:27:00 [---] [rr_sim] doing sim: CPU sched 10:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985790966.55G/439.78G) 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241560.47 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 224156.05 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224156.046533 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017404.42 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 201740.44 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425896.488414 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815663.98 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 181566.40 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607462.886106 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634097.58 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 163409.76 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770872.644028 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470687.82 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 147068.78 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917941.426159 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323619.04 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 132361.90 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050303.330077 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191257.14 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 119125.71 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169429.043602 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072131.42 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 107213.14 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276642.185776 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964918.28 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 96491.83 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373134.013731 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868426.45 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 86842.65 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459976.658892 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781583.81 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 78158.38 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538135.039536 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703425.43 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 70342.54 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608477.582116 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633082.88 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 63308.29 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671785.870438 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569774.59 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 56977.46 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728763.329927 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512797.14 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 51279.71 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1780043.043468 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461517.42 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 46151.74 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826194.785655 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415365.68 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 41536.57 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867731.353623 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373829.11 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 37382.91 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905114.264794 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336446.20 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 33644.62 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938758.884848 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302801.58 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 30280.16 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969039.042897 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272521.42 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 27252.14 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996291.185140 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245269.28 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 24526.93 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020818.113160 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220742.35 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 22074.24 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042892.348377 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198668.12 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 19866.81 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062759.160073 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178801.31 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 17880.13 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080639.290599 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160921.17 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 16092.12 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096731.408073 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144829.06 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 14482.91 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111214.313799 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130346.15 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 13034.62 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124248.928952 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117311.54 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 11731.15 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135980.082590 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105580.38 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 10558.04 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146538.120865 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95022.34 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 9502.23 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156040.355312 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85520.11 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 8552.01 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164592.366314 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76968.10 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 7696.81 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172289.176216 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69271.29 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 6927.13 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179216.305128 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62344.16 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 6234.42 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185450.721149 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56109.74 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 5610.97 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191061.695567 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50498.77 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 5049.88 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196111.572544 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45448.89 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 4544.89 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200656.461823 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40904.00 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 4090.40 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204746.862174 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36813.60 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3681.36 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208428.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33132.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212028.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29532.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215628.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25932.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219228.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22332.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222828.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18732.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226428.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15132.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230028.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11532.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233628.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7932.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237228.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4332.24 sec) 10:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240828.222490 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 732.24 sec) 10:27:00 [GPUGRID] [rr_sim] 2241560.47: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985790966.55G/439.78G) 10:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852221.38 10:27:00 [---] pick_jobs_to_run() start 10:27:00 [---] pick_jobs_to_run() end 10:27:00 [---] [rr_sim] end 10:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:27:00 [---] [cpu_sched_debug] preliminary job list: 10:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:27:00 [---] [cpu_sched_debug] final job list: 10:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:27:00 [---] [cpu_sched_debug] enforce_run_list: end 10:27:00 [---] choose_project(): 37620.000000 10:27:00 [iThena.Measurements] doing RPC to get work 10:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397979.09 cpu 2143772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:27:00 [---] Got 0 tasks 10:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:28:00 [---] [rr_sim] doing sim: CPU sched 10:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985763376.20G/439.78G) 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241497.73 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 224149.77 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224149.772845 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017347.96 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 201734.80 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425884.568406 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815613.16 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 181561.32 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607445.884411 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634051.84 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 163405.18 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770851.068815 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470646.66 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 147064.67 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917915.734778 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323581.99 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 132358.20 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050273.934146 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191223.79 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 119122.38 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169396.313576 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072101.41 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 107210.14 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276606.455064 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964891.27 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 96489.13 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373095.582403 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868402.15 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 86840.21 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459935.797008 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781561.93 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 78156.19 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538091.990152 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703405.74 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 70340.57 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608432.563982 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633065.16 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 63306.52 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671739.080429 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569758.65 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 56975.86 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728714.945232 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512782.78 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 51278.28 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779993.223554 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461504.50 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 46150.45 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826143.674043 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415354.05 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 41535.41 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867679.079484 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373818.65 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 37381.86 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905060.944381 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336436.78 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 33643.68 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938704.622788 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302793.11 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 30279.31 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968983.933355 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272513.80 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 27251.38 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996235.312864 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245262.42 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 24526.24 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020761.554423 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220736.17 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 22073.62 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042835.171826 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198662.56 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 19866.26 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062701.427489 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178796.30 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 17879.63 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080581.057585 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160916.67 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 16091.67 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096672.724672 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144825.00 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 14482.50 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111155.225050 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130342.50 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 13034.25 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124189.475390 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117308.25 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 11730.83 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135920.300696 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105577.43 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 10557.74 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146478.043472 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95019.68 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 9501.97 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155980.011970 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85517.72 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 8551.77 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164531.783618 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76965.94 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 7696.59 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172228.378101 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69269.35 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 6926.94 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179155.313137 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62342.42 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 6234.24 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185389.554668 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56108.17 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 5610.82 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191000.372047 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50497.36 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 5049.74 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196050.107687 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45447.62 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 4544.76 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200594.869764 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40902.86 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 4090.29 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204685.155632 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36812.57 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3681.26 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208366.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33131.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211966.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29531.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215566.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25931.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219166.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22331.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222766.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18731.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226366.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15131.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229966.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11531.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233566.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7931.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237166.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4331.32 sec) 10:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240766.412914 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 731.32 sec) 10:28:00 [GPUGRID] [rr_sim] 2241497.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985763376.20G/439.78G) 10:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852218.64 10:28:00 [---] pick_jobs_to_run() start 10:28:00 [---] pick_jobs_to_run() end 10:28:00 [---] [rr_sim] end 10:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:28:00 [---] [cpu_sched_debug] preliminary job list: 10:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:28:00 [---] [cpu_sched_debug] final job list: 10:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:28:00 [---] [cpu_sched_debug] enforce_run_list: end 10:28:00 [---] choose_project(): 37680.000000 10:28:00 [iThena.Measurements] doing RPC to get work 10:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397919.09 cpu 2143712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:28:00 [---] Got 0 tasks 10:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:29:00 [---] [rr_sim] doing sim: CPU sched 10:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985735785.84G/439.78G) 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241434.99 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 224143.50 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224143.499157 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017291.49 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 201729.15 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425872.648398 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815562.34 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 181556.23 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607428.882715 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1634006.11 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 163400.61 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770829.493601 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470605.50 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 147060.55 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917890.043398 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323544.95 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 132354.49 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050244.538215 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191190.45 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 119119.05 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169363.583551 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072071.41 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 107207.14 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276570.724352 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964864.27 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 96486.43 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373057.151074 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868377.84 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 86837.78 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459894.935124 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781540.06 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 78154.01 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538048.940768 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703386.05 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 70338.61 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608387.545849 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633047.45 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 63304.74 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671692.290421 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569742.70 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 56974.27 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728666.560536 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512768.43 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 51276.84 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779943.403639 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461491.59 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 46149.16 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826092.562432 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415342.43 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 41534.24 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867626.805346 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373808.19 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 37380.82 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905007.623968 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336427.37 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 33642.74 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938650.360729 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302784.63 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 30278.46 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968928.823813 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272506.17 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 27250.62 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996179.440588 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245255.55 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 24525.56 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020704.995687 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220730.00 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 22073.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042777.995275 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198657.00 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 19865.70 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062643.694904 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178791.30 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 17879.13 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080522.824571 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160912.17 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 16091.22 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096614.041271 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144820.95 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 14482.10 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111096.136301 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130338.86 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 13033.89 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124130.021828 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117304.97 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 11730.50 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135860.518802 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105574.47 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 10557.45 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146417.966079 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95017.03 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 9501.70 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155919.668628 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85515.32 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 8551.53 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164471.200922 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76963.79 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 7696.38 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172167.579987 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69267.41 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 6926.74 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179094.321145 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62340.67 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 6234.07 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185328.388188 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56106.60 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 5610.66 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190939.048526 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50495.94 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 5049.59 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195988.642830 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45446.35 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 4544.63 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200533.277704 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40901.71 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 4090.17 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204623.449091 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36811.54 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3681.15 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208304.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33130.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211904.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29530.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215504.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25930.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219104.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22330.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222704.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18730.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226304.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15130.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229904.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11530.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233504.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7930.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237104.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4330.39 sec) 10:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240704.603339 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 730.39 sec) 10:29:00 [GPUGRID] [rr_sim] 2241434.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985735785.84G/439.78G) 10:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852215.90 10:29:00 [---] pick_jobs_to_run() start 10:29:00 [---] pick_jobs_to_run() end 10:29:00 [---] [rr_sim] end 10:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:29:00 [---] [cpu_sched_debug] preliminary job list: 10:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:29:00 [---] [cpu_sched_debug] final job list: 10:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:29:00 [---] [cpu_sched_debug] enforce_run_list: end 10:29:00 [---] choose_project(): 37740.000000 10:29:00 [iThena.Measurements] doing RPC to get work 10:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397859.09 cpu 2143652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:29:00 [---] Got 0 tasks 10:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:30:00 [---] [rr_sim] doing sim: CPU sched 10:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985708195.48G/439.78G) 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241372.25 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 224137.23 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224137.225469 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017235.03 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 201723.50 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425860.728391 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815511.53 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 181551.15 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607411.881020 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633960.37 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 163396.04 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770807.918387 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470564.34 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 147056.43 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917864.352017 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323507.90 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 132350.79 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050215.142284 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191157.11 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 119115.71 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169330.853525 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072041.40 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 107204.14 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276534.993641 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964837.26 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 96483.73 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1373018.719746 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868353.53 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 86835.35 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459854.073240 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781518.18 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 78151.82 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1538005.891385 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703366.36 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 70336.64 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608342.527715 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633029.73 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 63302.97 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671645.500412 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569726.75 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 56972.68 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728618.175840 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512754.08 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 51275.41 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779893.583725 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461478.67 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 46147.87 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826041.450821 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415330.80 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 41533.08 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867574.531208 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373797.72 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 37379.77 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904954.303556 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336417.95 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 33641.80 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938596.098669 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302776.16 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 30277.62 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968873.714271 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272498.54 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 27249.85 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996123.568312 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245248.69 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 24524.87 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020648.436950 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220723.82 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 22072.38 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042720.818724 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198651.44 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 19865.14 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062585.962320 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178786.29 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 17878.63 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080464.591557 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160907.66 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 16090.77 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096555.357870 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144816.90 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 14481.69 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111037.047552 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130335.21 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 13033.52 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124070.568265 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117301.69 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 11730.17 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135800.736908 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105571.52 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 10557.15 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146357.888686 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95014.37 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 9501.44 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155859.325286 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85512.93 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 8551.29 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164410.618226 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76961.64 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 7696.16 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172106.781872 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69265.47 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 6926.55 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179033.329154 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62338.93 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 6233.89 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185267.221707 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56105.03 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 5610.50 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190877.725005 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50494.53 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 5049.45 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195927.177974 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45445.08 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 4544.51 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200471.685645 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40900.57 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 4090.06 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204561.742549 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36810.51 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3681.05 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208242.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33129.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211842.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29529.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215442.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25929.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219042.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22329.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222642.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18729.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226242.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15129.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229842.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11529.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233442.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7929.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237042.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4329.46 sec) 10:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240642.793763 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 729.46 sec) 10:30:00 [GPUGRID] [rr_sim] 2241372.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985708195.48G/439.78G) 10:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852213.17 10:30:00 [---] pick_jobs_to_run() start 10:30:00 [---] pick_jobs_to_run() end 10:30:00 [---] [rr_sim] end 10:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:30:00 [---] [cpu_sched_debug] preliminary job list: 10:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:30:00 [---] [cpu_sched_debug] final job list: 10:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:30:00 [---] [cpu_sched_debug] enforce_run_list: end 10:30:00 [---] choose_project(): 37800.000000 10:30:00 [iThena.Measurements] doing RPC to get work 10:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397799.09 cpu 2143592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:30:00 [---] Got 0 tasks 10:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:31:00 [---] [rr_sim] doing sim: CPU sched 10:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985680605.13G/439.78G) 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241309.52 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 224130.95 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224130.951781 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017178.57 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 201717.86 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425848.808383 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815460.71 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 181546.07 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607394.879325 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633914.64 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 163391.46 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770786.343173 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470523.17 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 147052.32 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917838.660637 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323470.86 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 132347.09 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050185.746354 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191123.77 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 119112.38 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169298.123499 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1072011.39 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 107201.14 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276499.262929 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964810.25 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 96481.03 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372980.288417 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868329.23 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 86832.92 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459813.211356 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781496.31 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 78149.63 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537962.842001 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703346.68 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 70334.67 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608297.509581 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633012.01 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 63301.20 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671598.710404 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569710.81 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 56971.08 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728569.791144 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512739.73 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 51273.97 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779843.763810 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461465.75 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 46146.58 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825990.339210 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415319.18 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 41531.92 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867522.257069 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373787.26 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 37378.73 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904900.983143 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336408.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 33640.85 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938541.836609 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302767.68 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 30276.77 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968818.604729 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272490.91 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 27249.09 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996067.696037 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245241.82 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 24524.18 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020591.878213 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220717.64 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 22071.76 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042663.642173 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198645.88 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 19864.59 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062528.229736 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178781.29 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 17878.13 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080406.358543 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160903.16 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 16090.32 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096496.674469 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144812.84 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 14481.28 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110977.958803 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130331.56 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 13033.16 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124011.114703 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117298.40 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 11729.84 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135740.955013 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105568.56 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 10556.86 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146297.811293 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95011.71 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 9501.17 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155798.981944 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85510.54 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 8551.05 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164350.035530 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76959.48 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 7695.95 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172045.983758 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69263.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 6926.35 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178972.337162 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62337.18 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 6233.72 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185206.055227 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56103.46 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 5610.35 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190816.401485 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50493.12 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 5049.31 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195865.713117 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45443.80 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 4544.38 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200410.093586 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40899.42 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 4089.94 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204500.036008 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36809.48 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3680.95 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208180.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33128.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211780.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29528.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215380.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25928.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218980.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22328.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222580.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18728.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226180.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15128.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229780.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11528.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233380.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7928.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236980.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4328.53 sec) 10:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240580.984187 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 728.53 sec) 10:31:00 [GPUGRID] [rr_sim] 2241309.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985680605.13G/439.78G) 10:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852210.43 10:31:00 [---] pick_jobs_to_run() start 10:31:00 [---] pick_jobs_to_run() end 10:31:00 [---] [rr_sim] end 10:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:31:00 [---] [cpu_sched_debug] preliminary job list: 10:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:31:00 [---] [cpu_sched_debug] final job list: 10:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:31:00 [---] [cpu_sched_debug] enforce_run_list: end 10:31:00 [---] choose_project(): 37860.000000 10:31:00 [iThena.Measurements] doing RPC to get work 10:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397739.09 cpu 2143532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:31:00 [---] Got 0 tasks 10:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:32:00 [---] [rr_sim] doing sim: CPU sched 10:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985653014.77G/439.78G) 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241246.78 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 224124.68 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224124.678092 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017122.10 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 201712.21 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425836.888375 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815409.89 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 181540.99 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607377.877630 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633868.90 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 163386.89 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770764.767960 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470482.01 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 147048.20 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917812.969256 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323433.81 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 132343.38 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050156.350423 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191090.43 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 119109.04 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169265.393473 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071981.39 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 107198.14 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276463.532218 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964783.25 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 96478.32 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372941.857088 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868304.92 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 86830.49 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459772.349472 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781474.43 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 78147.44 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537919.792617 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703326.99 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 70332.70 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608252.491448 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632994.29 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 63299.43 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671551.920395 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569694.86 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 56969.49 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728521.406448 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512725.37 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 51272.54 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779793.943896 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461452.84 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 46145.28 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825939.227598 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415307.55 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 41530.76 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867469.982931 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373776.80 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 37377.68 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904847.662730 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336399.12 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 33639.91 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938487.574550 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302759.21 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 30275.92 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968763.495187 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272483.29 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 27248.33 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996011.823761 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245234.96 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 24523.50 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020535.319477 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220711.46 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 22071.15 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042606.465622 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198640.32 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 19864.03 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062470.497152 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178776.28 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 17877.63 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080348.125529 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160898.66 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 16089.87 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096437.991068 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144808.79 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 14480.88 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110918.870054 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130327.91 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 13032.79 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123951.661141 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117295.12 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 11729.51 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135681.173119 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105565.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 10556.56 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146237.733900 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95009.05 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 9500.90 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155738.638602 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85508.14 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 8550.81 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164289.452834 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76957.33 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 7695.73 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171985.185643 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69261.60 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 6926.16 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178911.345171 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62335.44 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 6233.54 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185144.888746 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56101.89 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 5610.19 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190755.077964 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50491.70 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 5049.17 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195804.248260 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45442.53 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 4544.25 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200348.501526 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40898.28 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 4089.83 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204438.329466 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36808.45 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3680.85 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208119.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33127.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211719.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29527.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215319.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25927.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218919.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22327.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222519.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18727.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226119.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15127.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229719.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11527.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233319.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7927.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236919.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4327.61 sec) 10:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240519.174612 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 727.61 sec) 10:32:00 [GPUGRID] [rr_sim] 2241246.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985653014.77G/439.78G) 10:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852207.69 10:32:00 [---] pick_jobs_to_run() start 10:32:00 [---] pick_jobs_to_run() end 10:32:00 [---] [rr_sim] end 10:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:32:00 [---] [cpu_sched_debug] preliminary job list: 10:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:32:00 [---] [cpu_sched_debug] final job list: 10:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:32:00 [---] [cpu_sched_debug] enforce_run_list: end 10:32:00 [---] choose_project(): 37920.000000 10:32:00 [iThena.Measurements] doing RPC to get work 10:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397679.09 cpu 2143472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:32:00 [---] Got 0 tasks 10:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:33:00 [---] [rr_sim] doing sim: CPU sched 10:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985625424.41G/439.78G) 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241184.04 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 224118.40 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224118.404404 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017065.64 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 201706.56 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425824.968368 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815359.08 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 181535.91 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607360.875935 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633823.17 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 163382.32 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770743.192746 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470440.85 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 147044.09 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917787.277875 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323396.77 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 132339.68 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050126.954492 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191057.09 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 119105.71 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169232.663447 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071951.38 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 107195.14 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276427.801506 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964756.24 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 96475.62 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372903.425760 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868280.62 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 86828.06 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459731.487588 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781452.56 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 78145.26 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537876.743233 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703307.30 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 70330.73 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608207.473314 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632976.57 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 63297.66 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671505.130387 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569678.91 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 56967.89 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728473.021752 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512711.02 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 51271.10 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779744.123981 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461439.92 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 46143.99 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825888.115987 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415295.93 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 41529.59 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867417.708793 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373766.34 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 37376.63 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904794.342318 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336389.70 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 33638.97 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938433.312490 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302750.73 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 30275.07 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968708.385645 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272475.66 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 27247.57 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995955.951485 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245228.09 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 24522.81 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020478.760740 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220705.28 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 22070.53 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042549.289070 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198634.75 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 19863.48 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062412.764568 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178771.28 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 17877.13 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080289.892515 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160894.15 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 16089.42 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096379.307668 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144804.74 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 14480.47 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110859.781305 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130324.26 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 13032.43 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123892.207579 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117291.84 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 11729.18 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135621.391225 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105562.65 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 10556.27 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146177.656507 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95006.39 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 9500.64 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155678.295260 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85505.75 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 8550.57 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164228.870138 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76955.17 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 7695.52 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171924.387528 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69259.66 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 6925.97 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178850.353180 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62333.69 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 6233.37 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185083.722266 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56100.32 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 5610.03 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190693.754443 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50490.29 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 5049.03 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195742.783403 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45441.26 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 4544.13 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200286.909467 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40897.13 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 4089.71 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204376.622924 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36807.42 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3680.74 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208057.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33126.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211657.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29526.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215257.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25926.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218857.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22326.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222457.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18726.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226057.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15126.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229657.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11526.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233257.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7926.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236857.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4326.68 sec) 10:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240457.365036 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 726.68 sec) 10:33:00 [GPUGRID] [rr_sim] 2241184.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985625424.41G/439.78G) 10:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852204.96 10:33:00 [---] pick_jobs_to_run() start 10:33:00 [---] pick_jobs_to_run() end 10:33:00 [---] [rr_sim] end 10:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:33:00 [---] [cpu_sched_debug] preliminary job list: 10:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:33:00 [---] [cpu_sched_debug] final job list: 10:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:33:00 [---] [cpu_sched_debug] enforce_run_list: end 10:33:00 [---] choose_project(): 37980.000000 10:33:00 [iThena.Measurements] doing RPC to get work 10:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397619.09 cpu 2143412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:33:00 [---] Got 0 tasks 10:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:34:00 [---] [rr_sim] doing sim: CPU sched 10:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985597834.06G/439.78G) 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241121.31 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 224112.13 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224112.130716 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2017009.18 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 201700.92 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425813.048360 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815308.26 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 181530.83 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607343.874240 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633777.43 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 163377.74 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770721.617532 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470399.69 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 147039.97 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917761.586495 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323359.72 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 132335.97 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050097.558561 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1191023.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 119102.37 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169199.933421 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071921.37 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 107192.14 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276392.070795 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964729.24 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 96472.92 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372864.994431 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868256.31 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 86825.63 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459690.625704 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781430.68 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 78143.07 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537833.693850 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703287.61 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 70328.76 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608162.455181 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632958.85 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 63295.89 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671458.340378 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569662.97 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 56966.30 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728424.637056 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512696.67 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 51269.67 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779694.304067 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461427.00 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 46142.70 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825837.004376 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415284.30 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 41528.43 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867365.434654 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373755.87 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 37375.59 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904741.021905 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336380.29 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 33638.03 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938379.050430 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302742.26 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 30274.23 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968653.276103 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272468.03 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 27246.80 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995900.079209 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245221.23 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 24522.12 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020422.202004 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220699.11 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 22069.91 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042492.112519 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198629.19 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 19862.92 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062355.031983 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178766.28 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 17876.63 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080231.659501 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160889.65 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 16088.96 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096320.624267 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144800.68 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 14480.07 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110800.692556 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130320.61 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 13032.06 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123832.754016 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117288.55 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 11728.86 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135561.609331 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105559.70 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 10555.97 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146117.579113 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95003.73 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 9500.37 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155617.951918 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85503.36 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 8550.34 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164168.287442 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76953.02 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 7695.30 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171863.589414 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69257.72 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 6925.77 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178789.361188 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62331.95 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 6233.19 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185022.555785 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56098.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 5609.88 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190632.430923 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50488.88 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 5048.89 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195681.318546 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45439.99 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 4544.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200225.317408 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40895.99 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 4089.60 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204314.916383 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36806.39 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3680.64 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207995.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33125.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211595.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29525.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215195.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25925.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218795.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22325.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222395.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18725.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225995.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15125.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229595.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11525.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233195.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7925.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236795.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4325.75 sec) 10:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240395.555461 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 725.75 sec) 10:34:00 [GPUGRID] [rr_sim] 2241121.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985597834.06G/439.78G) 10:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852202.22 10:34:00 [---] pick_jobs_to_run() start 10:34:00 [---] pick_jobs_to_run() end 10:34:00 [---] [rr_sim] end 10:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:34:00 [---] [cpu_sched_debug] preliminary job list: 10:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:34:00 [---] [cpu_sched_debug] final job list: 10:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:34:00 [---] [cpu_sched_debug] enforce_run_list: end 10:34:00 [---] choose_project(): 38040.000000 10:34:00 [iThena.Measurements] doing RPC to get work 10:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397559.09 cpu 2143352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:34:00 [---] Got 0 tasks 10:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:35:00 [---] [rr_sim] doing sim: CPU sched 10:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985570243.70G/439.78G) 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2241058.57 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 224105.86 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224105.857028 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016952.71 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 201695.27 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425801.128353 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815257.44 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 181525.74 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607326.872545 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633731.70 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 163373.17 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770700.042318 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470358.53 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 147035.85 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917735.895114 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323322.68 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 132332.27 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050068.162630 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190990.41 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 119099.04 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169167.203395 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071891.37 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 107189.14 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276356.340083 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964702.23 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 96470.22 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372826.563103 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868232.01 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 86823.20 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459649.763820 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781408.81 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 78140.88 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537790.644466 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703267.93 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 70326.79 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608117.437047 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632941.13 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 63294.11 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671411.550370 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569647.02 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 56964.70 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728376.252361 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512682.32 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 51268.23 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779644.484152 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461414.09 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 46141.41 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825785.892765 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415272.68 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 41527.27 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867313.160516 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373745.41 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 37374.54 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904687.701492 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336370.87 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 33637.09 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938324.788371 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302733.78 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 30273.38 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968598.166561 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272460.40 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 27246.04 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995844.206933 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245214.36 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 24521.44 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020365.643267 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220692.93 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 22069.29 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042434.935968 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198623.63 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 19862.36 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062297.299399 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178761.27 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 17876.13 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080173.426487 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160885.14 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 16088.51 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096261.940866 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144796.63 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 14479.66 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110741.603807 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130316.97 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 13031.70 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123773.300454 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117285.27 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 11728.53 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135501.827436 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105556.74 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 10555.67 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146057.501720 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95001.07 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 9500.11 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155557.608576 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85500.96 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 8550.10 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164107.704746 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76950.87 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 7695.09 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171802.791299 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69255.78 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 6925.58 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178728.369197 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62330.20 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 6233.02 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184961.389305 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56097.18 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 5609.72 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190571.107402 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50487.46 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 5048.75 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195619.853690 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45438.72 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 4543.87 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200163.725348 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40894.84 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 4089.48 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204253.209841 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36805.36 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3680.54 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207933.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33124.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211533.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29524.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215133.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25924.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218733.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22324.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222333.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18724.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225933.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15124.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229533.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11524.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233133.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7924.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236733.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4324.82 sec) 10:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240333.745885 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 724.82 sec) 10:35:00 [GPUGRID] [rr_sim] 2241058.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985570243.70G/439.78G) 10:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852199.48 10:35:00 [---] pick_jobs_to_run() start 10:35:00 [---] pick_jobs_to_run() end 10:35:00 [---] [rr_sim] end 10:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:35:00 [---] [cpu_sched_debug] preliminary job list: 10:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:35:00 [---] [cpu_sched_debug] final job list: 10:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:35:00 [---] [cpu_sched_debug] enforce_run_list: end 10:35:00 [---] choose_project(): 38100.000000 10:35:00 [iThena.Measurements] doing RPC to get work 10:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397499.09 cpu 2143292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:35:00 [---] Got 0 tasks 10:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:36:00 [---] [rr_sim] doing sim: CPU sched 10:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985542653.35G/439.78G) 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240995.83 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 224099.58 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224099.583339 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016896.25 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 201689.63 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425789.208345 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815206.63 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 181520.66 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607309.870850 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633685.96 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 163368.60 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770678.467104 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470317.37 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 147031.74 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917710.203734 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323285.63 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 132328.56 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050038.766700 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190957.07 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 119095.71 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169134.473369 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071861.36 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 107186.14 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276320.609372 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964675.22 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 96467.52 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372788.131774 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868207.70 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 86820.77 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459608.901936 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781386.93 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 78138.69 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537747.595082 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703248.24 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 70324.82 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608072.418913 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632923.41 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 63292.34 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671364.760361 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569631.07 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 56963.11 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728327.867665 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512667.97 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 51266.80 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779594.664238 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461401.17 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 46140.12 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825734.781154 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415261.05 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 41526.11 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867260.886378 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373734.95 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 37373.49 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904634.381079 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336361.45 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 33636.15 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938270.526311 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302725.31 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 30272.53 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968543.057019 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272452.78 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 27245.28 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995788.334657 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245207.50 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 24520.75 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020309.084531 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220686.75 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 22068.67 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042377.759417 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198618.07 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 19861.81 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062239.566815 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178756.27 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 17875.63 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080115.193473 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160880.64 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 16088.06 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096203.257465 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144792.58 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 14479.26 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110682.515058 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130313.32 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 13031.33 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123713.846892 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117281.99 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 11728.20 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135442.045542 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105553.79 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 10555.38 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145997.424327 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94998.41 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 9499.84 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155497.265234 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85498.57 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 8549.86 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164047.122050 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76948.71 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 7694.87 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171741.993185 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69253.84 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 6925.38 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178667.377206 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62328.46 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 6232.85 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184900.222825 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56095.61 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 5609.56 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190509.783882 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50486.05 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 5048.60 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195558.388833 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45437.44 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 4543.74 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200102.133289 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40893.70 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 4089.37 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204191.503300 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36804.33 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3680.43 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207871.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33123.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211471.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29523.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215071.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25923.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218671.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22323.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222271.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18723.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225871.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15123.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229471.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11523.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233071.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7923.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236671.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4323.90 sec) 10:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240271.936309 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 723.90 sec) 10:36:00 [GPUGRID] [rr_sim] 2240995.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985542653.35G/439.78G) 10:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852196.75 10:36:00 [---] pick_jobs_to_run() start 10:36:00 [---] pick_jobs_to_run() end 10:36:00 [---] [rr_sim] end 10:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:36:00 [---] [cpu_sched_debug] preliminary job list: 10:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:36:00 [---] [cpu_sched_debug] final job list: 10:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:36:00 [---] [cpu_sched_debug] enforce_run_list: end 10:36:00 [---] choose_project(): 38160.000000 10:36:00 [iThena.Measurements] doing RPC to get work 10:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397439.09 cpu 2143232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:36:00 [---] Got 0 tasks 10:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:37:00 [---] [rr_sim] doing sim: CPU sched 10:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985515062.99G/439.78G) 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240933.10 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 224093.31 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224093.309651 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016839.79 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 201683.98 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425777.288337 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815155.81 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 181515.58 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607292.869155 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633640.23 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 163364.02 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770656.891891 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470276.20 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 147027.62 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917684.512353 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323248.58 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 132324.86 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1050009.370769 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190923.73 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 119092.37 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169101.743343 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071831.35 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 107183.14 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276284.878660 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964648.22 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 96464.82 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372749.700445 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868183.40 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 86818.34 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459568.040052 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781365.06 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 78136.51 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537704.545698 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703228.55 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 70322.86 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1608027.400780 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632905.70 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 63290.57 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671317.970353 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569615.13 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 56961.51 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728279.482969 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512653.61 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 51265.36 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779544.844323 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461388.25 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 46138.83 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825683.669542 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415249.43 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 41524.94 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867208.612239 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373724.48 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 37372.45 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904581.060667 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336352.04 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 33635.20 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938216.264251 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302716.83 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 30271.68 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968487.947477 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272445.15 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 27244.51 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995732.462381 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245200.63 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 24520.06 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020252.525794 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220680.57 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 22068.06 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042320.582866 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198612.51 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 19861.25 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062181.834231 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178751.26 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 17875.13 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080056.960459 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160876.14 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 16087.61 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096144.574064 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144788.52 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 14478.85 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110623.426309 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130309.67 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 13030.97 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123654.393329 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117278.70 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 11727.87 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135382.263648 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105550.83 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 10555.08 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145937.346934 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94995.75 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 9499.57 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155436.921892 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85496.17 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 8549.62 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163986.539354 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76946.56 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 7694.66 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171681.195070 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69251.90 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 6925.19 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178606.385214 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62326.71 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 6232.67 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184839.056344 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56094.04 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 5609.40 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190448.460361 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50484.64 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 5048.46 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195496.923976 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45436.17 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 4543.62 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200040.541230 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40892.56 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 4089.26 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204129.796758 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36803.30 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3680.33 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207810.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33122.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211410.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29522.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215010.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25922.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218610.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22322.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222210.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18722.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225810.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15122.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229410.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11522.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233010.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7922.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236610.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4322.97 sec) 10:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240210.126734 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 722.97 sec) 10:37:00 [GPUGRID] [rr_sim] 2240933.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985515062.99G/439.78G) 10:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852194.01 10:37:00 [---] pick_jobs_to_run() start 10:37:00 [---] pick_jobs_to_run() end 10:37:00 [---] [rr_sim] end 10:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:37:00 [---] [cpu_sched_debug] preliminary job list: 10:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:37:00 [---] [cpu_sched_debug] final job list: 10:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:37:00 [---] [cpu_sched_debug] enforce_run_list: end 10:37:00 [---] choose_project(): 38220.000000 10:37:00 [iThena.Measurements] doing RPC to get work 10:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397379.09 cpu 2143172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:37:00 [---] Got 0 tasks 10:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:38:00 [---] [rr_sim] doing sim: CPU sched 10:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985487472.63G/439.78G) 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240870.36 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 224087.04 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224087.035963 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016783.32 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 201678.33 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425765.368330 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815104.99 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 181510.50 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607275.867460 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633594.49 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 163359.45 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770635.316677 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470235.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 147023.50 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917658.820972 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323211.54 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 132321.15 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049979.974838 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190890.38 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 119089.04 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169069.013317 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071801.35 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 107180.13 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276249.147949 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964621.21 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 96462.12 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372711.269117 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868159.09 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 86815.91 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459527.178168 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781343.18 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 78134.32 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537661.496314 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703208.86 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 70320.89 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607982.382646 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632887.98 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 63288.80 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671271.180345 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569599.18 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 56959.92 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728231.098273 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512639.26 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 51263.93 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779495.024409 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461375.34 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 46137.53 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825632.557931 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415237.80 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 41523.78 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867156.338101 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373714.02 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 37371.40 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904527.740254 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336342.62 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 33634.26 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938162.002192 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302708.36 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 30270.84 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968432.837935 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272437.52 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 27243.75 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995676.590105 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245193.77 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 24519.38 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020195.967058 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220674.39 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 22067.44 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042263.406315 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198606.95 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 19860.70 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062124.101646 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178746.26 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 17874.63 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079998.727445 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160871.63 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 16087.16 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096085.890663 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144784.47 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 14478.45 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110564.337560 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130306.02 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 13030.60 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123594.939767 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117275.42 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 11727.54 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135322.481753 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105547.88 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 10554.79 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145877.269541 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94993.09 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 9499.31 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155376.578550 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85493.78 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 8549.38 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163925.956658 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76944.40 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 7694.44 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171620.396955 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69249.96 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 6925.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178545.393223 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62324.97 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 6232.50 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184777.889864 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56092.47 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 5609.25 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190387.136840 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50483.22 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 5048.32 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195435.459119 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45434.90 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 4543.49 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199978.949170 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40891.41 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 4089.14 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204068.090216 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36802.27 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3680.23 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207748.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33122.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211348.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29522.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214948.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25922.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218548.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22322.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222148.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18722.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225748.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15122.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229348.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11522.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232948.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7922.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236548.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4322.04 sec) 10:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240148.317158 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 722.04 sec) 10:38:00 [GPUGRID] [rr_sim] 2240870.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985487472.63G/439.78G) 10:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852191.27 10:38:00 [---] pick_jobs_to_run() start 10:38:00 [---] pick_jobs_to_run() end 10:38:00 [---] [rr_sim] end 10:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:38:00 [---] [cpu_sched_debug] preliminary job list: 10:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:38:00 [---] [cpu_sched_debug] final job list: 10:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:38:00 [---] [cpu_sched_debug] enforce_run_list: end 10:38:00 [---] choose_project(): 38280.000000 10:38:00 [iThena.Measurements] doing RPC to get work 10:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397319.09 cpu 2143112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:38:00 [---] Got 0 tasks 10:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:39:00 [---] [rr_sim] doing sim: CPU sched 10:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985459882.28G/439.78G) 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240807.62 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 224080.76 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224080.762275 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016726.86 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 201672.69 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425753.448322 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815054.17 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 181505.42 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607258.865765 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633548.76 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 163354.88 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770613.741463 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470193.88 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 147019.39 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917633.129592 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323174.49 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 132317.45 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049950.578907 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190857.04 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 119085.70 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169036.283291 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071771.34 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 107177.13 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276213.417237 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964594.21 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 96459.42 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372672.837788 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868134.78 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 86813.48 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459486.316284 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781321.31 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 78132.13 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537618.446931 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703189.18 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 70318.92 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607937.364512 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632870.26 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 63287.03 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671224.390336 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569583.23 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 56958.32 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728182.713577 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512624.91 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 51262.49 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779445.204494 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461362.42 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 46136.24 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825581.446320 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415226.18 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 41522.62 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867104.063963 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373703.56 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 37370.36 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904474.419841 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336333.20 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 33633.32 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938107.740132 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302699.88 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 30269.99 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968377.728394 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272429.89 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 27242.99 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995620.717829 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245186.90 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 24518.69 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020139.408321 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220668.21 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 22066.82 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042206.229764 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198601.39 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 19860.14 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062066.369062 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178741.25 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 17874.13 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079940.494431 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160867.13 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 16086.71 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096027.207263 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144780.42 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 14478.04 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110505.248811 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130302.37 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 13030.24 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123535.486205 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117272.14 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 11727.21 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135262.699859 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105544.92 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 10554.49 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145817.192148 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94990.43 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 9499.04 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155316.235208 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85491.39 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 8549.14 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163865.373962 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76942.25 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 7694.22 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171559.598841 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69248.02 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 6924.80 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178484.401232 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62323.22 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 6232.32 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184716.723383 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56090.90 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 5609.09 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190325.813320 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50481.81 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 5048.18 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195373.994263 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45433.63 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 4543.36 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199917.357111 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40890.27 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 4089.03 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204006.383675 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36801.24 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3680.12 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207686.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33121.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211286.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29521.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214886.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25921.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218486.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22321.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222086.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18721.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225686.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15121.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229286.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11521.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232886.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7921.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236486.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4321.12 sec) 10:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240086.507582 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 721.12 sec) 10:39:00 [GPUGRID] [rr_sim] 2240807.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985459882.28G/439.78G) 10:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852188.54 10:39:00 [---] pick_jobs_to_run() start 10:39:00 [---] pick_jobs_to_run() end 10:39:00 [---] [rr_sim] end 10:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:39:00 [---] [cpu_sched_debug] preliminary job list: 10:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:39:00 [---] [cpu_sched_debug] final job list: 10:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:39:00 [---] [cpu_sched_debug] enforce_run_list: end 10:39:00 [---] choose_project(): 38340.000000 10:39:00 [iThena.Measurements] doing RPC to get work 10:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 570059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397259.09 cpu 2143052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2143052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2143052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:39:00 [---] Got 0 tasks 10:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:40:00 [---] [rr_sim] doing sim: CPU sched 10:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985432291.92G/439.78G) 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240744.89 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 224074.49 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224074.488587 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016670.40 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 201667.04 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425741.528315 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1815003.36 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 181500.34 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607241.864070 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633503.02 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 163350.30 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770592.166249 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470152.72 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 147015.27 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917607.438211 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323137.45 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 132313.74 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049921.182977 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190823.70 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 119082.37 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1169003.553266 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071741.33 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 107174.13 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276177.686526 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964567.20 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 96456.72 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372634.406460 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868110.48 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 86811.05 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459445.454400 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781299.43 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 78129.94 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537575.397547 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703169.49 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 70316.95 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607892.346379 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632852.54 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 63285.25 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671177.600328 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569567.29 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 56956.73 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728134.328881 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512610.56 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 51261.06 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779395.384580 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461349.50 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 46134.95 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825530.334709 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415214.55 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 41521.46 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867051.789824 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373693.10 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 37369.31 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904421.099428 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336323.79 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 33632.38 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938053.478072 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302691.41 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 30269.14 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968322.618852 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272422.27 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 27242.23 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995564.845553 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245180.04 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 24518.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020082.849584 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220662.04 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 22066.20 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042149.053213 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198595.83 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 19859.58 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062008.636478 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178736.25 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 17873.62 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079882.261417 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160862.62 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 16086.26 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095968.523862 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144776.36 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 14477.64 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110446.160062 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130298.73 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 13029.87 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123476.032643 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117268.85 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 11726.89 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135202.917965 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105541.97 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 10554.20 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145757.114755 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94987.77 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 9498.78 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155255.891866 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85488.99 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 8548.90 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163804.791266 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76940.09 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 7694.01 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171498.800726 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69246.09 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 6924.61 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178423.409240 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62321.48 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 6232.15 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184655.556903 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56089.33 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 5608.93 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190264.489799 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50480.40 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 5048.04 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195312.529406 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45432.36 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 4543.24 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199855.765052 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40889.12 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 4088.91 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203944.677133 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36800.21 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3680.02 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207624.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33120.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211224.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29520.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214824.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25920.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218424.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22320.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222024.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18720.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225624.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15120.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229224.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11520.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232824.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7920.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236424.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4320.19 sec) 10:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2240024.698007 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 720.19 sec) 10:40:00 [GPUGRID] [rr_sim] 2240744.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985432291.92G/439.78G) 10:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852185.80 10:40:00 [---] pick_jobs_to_run() start 10:40:00 [---] pick_jobs_to_run() end 10:40:00 [---] [rr_sim] end 10:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:40:00 [---] [cpu_sched_debug] preliminary job list: 10:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:40:00 [---] [cpu_sched_debug] final job list: 10:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:40:00 [---] [cpu_sched_debug] enforce_run_list: end 10:40:00 [---] choose_project(): 38400.000000 10:40:00 [iThena.Measurements] doing RPC to get work 10:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397199.09 cpu 2142992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:40:00 [---] Got 0 tasks 10:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:41:00 [---] [rr_sim] doing sim: CPU sched 10:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985404701.56G/439.78G) 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240682.15 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 224068.21 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224068.214898 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016613.93 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 201661.39 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425729.608307 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814952.54 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 181495.25 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607224.862375 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633457.29 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 163345.73 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770570.591036 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470111.56 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 147011.16 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917581.746830 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323100.40 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 132310.04 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049891.787046 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190790.36 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 119079.04 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168970.823240 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071711.33 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 107171.13 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276141.955814 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964540.19 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 96454.02 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372595.975131 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868086.17 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 86808.62 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459404.592516 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781277.56 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 78127.76 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537532.348163 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703149.80 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 70314.98 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607847.328245 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632834.82 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 63283.48 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671130.810319 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569551.34 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 56955.13 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728085.944186 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512596.20 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 51259.62 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779345.564665 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461336.58 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 46133.66 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825479.223097 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415202.93 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 41520.29 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866999.515686 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373682.63 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 37368.26 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904367.779016 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336314.37 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 33631.44 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937999.216013 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302682.93 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 30268.29 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968267.509310 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272414.64 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 27241.46 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995508.973277 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245173.18 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 24517.32 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020026.290848 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220655.86 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 22065.59 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042091.876661 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198590.27 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 19859.03 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061950.903894 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178731.25 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 17873.12 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079824.028403 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160858.12 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 16085.81 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095909.840461 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144772.31 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 14477.23 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110387.071313 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130295.08 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 13029.51 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123416.579080 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117265.57 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 11726.56 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135143.136071 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105539.01 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 10553.90 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145697.037362 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94985.11 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 9498.51 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155195.548524 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85486.60 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 8548.66 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163744.208570 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76937.94 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 7693.79 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171438.002612 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69244.15 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 6924.41 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178362.417249 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62319.73 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 6231.97 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184594.390422 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56087.76 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 5608.78 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190203.166279 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50478.98 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 5047.90 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195251.064549 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45431.08 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 4543.11 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199794.172993 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40887.98 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 4088.80 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203882.970592 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36799.18 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3679.92 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207562.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33119.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211162.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29519.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214762.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25919.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218362.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22319.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221962.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18719.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225562.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15119.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229162.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11519.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232762.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7919.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236362.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4319.26 sec) 10:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239962.888431 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 719.26 sec) 10:41:00 [GPUGRID] [rr_sim] 2240682.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985404701.56G/439.78G) 10:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852183.06 10:41:00 [---] pick_jobs_to_run() start 10:41:00 [---] pick_jobs_to_run() end 10:41:00 [---] [rr_sim] end 10:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:41:00 [---] [cpu_sched_debug] preliminary job list: 10:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:41:00 [---] [cpu_sched_debug] final job list: 10:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:41:00 [---] [cpu_sched_debug] enforce_run_list: end 10:41:00 [---] choose_project(): 38460.000000 10:41:00 [iThena.Measurements] doing RPC to get work 10:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397139.09 cpu 2142932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:41:00 [---] Got 0 tasks 10:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:42:00 [---] [rr_sim] doing sim: CPU sched 10:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985377111.21G/439.78G) 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240619.41 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 224061.94 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224061.941210 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016557.47 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 201655.75 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425717.688299 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814901.72 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 181490.17 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607207.860680 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633411.55 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 163341.16 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770549.015822 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470070.40 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 147007.04 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917556.055450 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323063.36 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 132306.34 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049862.391115 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190757.02 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 119075.70 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168938.093214 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071681.32 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 107168.13 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276106.225103 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964513.19 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 96451.32 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372557.543802 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868061.87 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 86806.19 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459363.730632 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781255.68 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 78125.57 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537489.298779 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703130.11 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 70313.01 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607802.310112 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632817.10 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 63281.71 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671084.020311 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569535.39 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 56953.54 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1728037.559490 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512581.85 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 51258.19 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779295.744751 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461323.67 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 46132.37 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825428.111486 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415191.30 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 41519.13 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866947.241548 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373672.17 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 37367.22 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904314.458603 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336304.95 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 33630.50 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937944.953953 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302674.46 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 30267.45 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968212.399768 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272407.01 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 27240.70 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995453.101001 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245166.31 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 24516.63 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019969.732111 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220649.68 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 22064.97 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042034.700110 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198584.71 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 19858.47 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061893.171310 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178726.24 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 17872.62 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079765.795389 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160853.62 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 16085.36 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095851.157060 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144768.26 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 14476.83 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110327.982564 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130291.43 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 13029.14 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123357.125518 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117262.29 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 11726.23 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135083.354176 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105536.06 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 10553.61 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145636.959969 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94982.45 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 9498.25 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155135.205182 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85484.21 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 8548.42 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163683.625874 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76935.79 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 7693.58 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171377.204497 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69242.21 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 6924.22 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178301.425257 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62317.99 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 6231.80 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184533.223942 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56086.19 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 5608.62 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190141.842758 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50477.57 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 5047.76 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195189.599692 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45429.81 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 4542.98 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199732.580933 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40886.83 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 4088.68 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203821.264050 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36798.15 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3679.81 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207501.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33118.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211101.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29518.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214701.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25918.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218301.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22318.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221901.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18718.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225501.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15118.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229101.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11518.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232701.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7918.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236301.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4318.33 sec) 10:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239901.078855 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 718.33 sec) 10:42:00 [GPUGRID] [rr_sim] 2240619.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985377111.21G/439.78G) 10:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852180.32 10:42:00 [---] pick_jobs_to_run() start 10:42:00 [---] pick_jobs_to_run() end 10:42:00 [---] [rr_sim] end 10:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:42:00 [---] [cpu_sched_debug] preliminary job list: 10:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:42:00 [---] [cpu_sched_debug] final job list: 10:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:42:00 [---] [cpu_sched_debug] enforce_run_list: end 10:42:00 [---] choose_project(): 38520.000000 10:42:00 [iThena.Measurements] doing RPC to get work 10:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397079.09 cpu 2142872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:42:00 [---] Got 0 tasks 10:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:43:00 [---] [rr_sim] doing sim: CPU sched 10:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985349520.85G/439.78G) 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240556.68 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 224055.67 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224055.667522 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016501.01 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 201650.10 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425705.768292 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814850.91 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 181485.09 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607190.858985 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633365.82 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 163336.58 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770527.440608 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1470029.23 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 147002.92 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917530.364069 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1323026.31 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 132302.63 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049832.995184 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190723.68 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 119072.37 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168905.363188 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071651.31 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 107165.13 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276070.494391 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964486.18 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 96448.62 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372519.112474 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868037.56 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 86803.76 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459322.868748 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781233.81 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 78123.38 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537446.249396 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703110.43 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 70311.04 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607757.291978 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632799.38 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 63279.94 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1671037.230302 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569519.44 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 56951.94 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727989.174794 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512567.50 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 51256.75 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779245.924836 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461310.75 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 46131.08 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825376.999875 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415179.68 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 41517.97 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866894.967409 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373661.71 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 37366.17 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904261.138190 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336295.54 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 33629.55 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937890.691893 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302665.98 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 30266.60 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968157.290226 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272399.38 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 27239.94 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995397.228725 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245159.45 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 24515.94 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019913.173375 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220643.50 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 22064.35 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041977.523559 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198579.15 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 19857.92 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061835.438725 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178721.24 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 17872.12 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079707.562375 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160849.11 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 16084.91 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095792.473659 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144764.20 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 14476.42 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110268.893815 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130287.78 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 13028.78 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123297.671956 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117259.00 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 11725.90 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135023.572282 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105533.10 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 10553.31 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145576.882576 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94979.79 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 9497.98 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155074.861840 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85481.81 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 8548.18 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163623.043178 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76933.63 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 7693.36 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171316.406382 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69240.27 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 6924.03 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178240.433266 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62316.24 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 6231.62 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184472.057461 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56084.62 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 5608.46 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190080.519237 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50476.16 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 5047.62 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195128.134836 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45428.54 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 4542.85 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199670.988874 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40885.69 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 4088.57 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203759.557509 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36797.12 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3679.71 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207439.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33117.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211039.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29517.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214639.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25917.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218239.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22317.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221839.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18717.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225439.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15117.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229039.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11517.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232639.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7917.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236239.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4317.41 sec) 10:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239839.269280 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 717.41 sec) 10:43:00 [GPUGRID] [rr_sim] 2240556.68: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985349520.85G/439.78G) 10:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852177.59 10:43:00 [---] pick_jobs_to_run() start 10:43:00 [---] pick_jobs_to_run() end 10:43:00 [---] [rr_sim] end 10:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:43:00 [---] [cpu_sched_debug] preliminary job list: 10:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:43:00 [---] [cpu_sched_debug] final job list: 10:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:43:00 [---] [cpu_sched_debug] enforce_run_list: end 10:43:00 [---] choose_project(): 38580.000000 10:43:00 [iThena.Measurements] doing RPC to get work 10:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 397019.09 cpu 2142812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:43:00 [---] Got 0 tasks 10:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:44:00 [---] [rr_sim] doing sim: CPU sched 10:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985321930.50G/439.78G) 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240493.94 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 224049.39 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224049.393834 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016444.54 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 201644.45 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425693.848284 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814800.09 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 181480.01 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607173.857289 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633320.08 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 163332.01 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770505.865394 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469988.07 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 146998.81 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917504.672689 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322989.27 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 132298.93 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049803.599253 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190690.34 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 119069.03 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168872.633162 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071621.31 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 107162.13 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1276034.763679 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964459.17 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 96445.92 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372480.681145 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 868013.26 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 86801.33 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459282.006864 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781211.93 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 78121.19 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537403.200012 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703090.74 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 70309.07 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607712.273844 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632781.66 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 63278.17 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670990.440294 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569503.50 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 56950.35 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727940.790098 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512553.15 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 51255.31 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779196.104922 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461297.83 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 46129.78 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825325.888264 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415168.05 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 41516.81 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866842.693271 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373651.25 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 37365.12 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904207.817778 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336286.12 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 33628.61 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937836.429834 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302657.51 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 30265.75 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968102.180684 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272391.76 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 27239.18 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995341.356449 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245152.58 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 24515.26 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019856.614638 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220637.32 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 22063.73 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041920.347008 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198573.59 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 19857.36 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061777.706141 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178716.23 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 17871.62 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079649.329361 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160844.61 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 16084.46 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095733.790258 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144760.15 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 14476.01 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110209.805066 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130284.13 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 13028.41 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123238.218393 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117255.72 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 11725.57 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134963.790388 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105530.15 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 10553.01 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145516.805183 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94977.13 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 9497.71 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155014.518498 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85479.42 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 8547.94 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163562.460482 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76931.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 7693.15 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171255.608268 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69238.33 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 6923.83 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178179.441275 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62314.50 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 6231.45 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184410.890981 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56083.05 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 5608.30 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190019.195717 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50474.74 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 5047.47 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195066.669979 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45427.27 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 4542.73 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199609.396815 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40884.54 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 4088.45 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203697.850967 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36796.09 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3679.61 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207377.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33116.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210977.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29516.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214577.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25916.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218177.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22316.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221777.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18716.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225377.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15116.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228977.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11516.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232577.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7916.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236177.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4316.48 sec) 10:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239777.459704 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 716.48 sec) 10:44:00 [GPUGRID] [rr_sim] 2240493.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985321930.50G/439.78G) 10:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852174.85 10:44:00 [---] pick_jobs_to_run() start 10:44:00 [---] pick_jobs_to_run() end 10:44:00 [---] [rr_sim] end 10:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:44:00 [---] [cpu_sched_debug] preliminary job list: 10:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:44:00 [---] [cpu_sched_debug] final job list: 10:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:44:00 [---] [cpu_sched_debug] enforce_run_list: end 10:44:00 [---] choose_project(): 38640.000000 10:44:00 [iThena.Measurements] doing RPC to get work 10:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396959.09 cpu 2142752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:44:00 [---] Got 0 tasks 10:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:45:00 [---] [rr_sim] doing sim: CPU sched 10:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985294340.14G/439.78G) 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240431.20 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 224043.12 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224043.120146 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016388.08 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 201638.81 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425681.928277 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814749.27 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 181474.93 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607156.855594 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633274.35 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 163327.43 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770484.290181 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469946.91 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 146994.69 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917478.981308 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322952.22 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 132295.22 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049774.203323 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190657.00 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 119065.70 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168839.903136 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071591.30 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 107159.13 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275999.032968 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964432.17 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 96443.22 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372442.249817 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867988.95 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 86798.90 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459241.144981 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781190.06 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 78119.01 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537360.150628 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703071.05 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 70307.11 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607667.255711 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632763.95 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 63276.39 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670943.650285 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569487.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 56948.76 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727892.405402 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512538.80 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 51253.88 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779146.285008 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461284.92 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 46128.49 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825274.776652 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415156.42 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 41515.64 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866790.419133 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373640.78 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 37364.08 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904154.497365 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336276.70 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 33627.67 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937782.167774 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302649.03 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 30264.90 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968047.071142 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272384.13 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 27238.41 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995285.484173 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245145.72 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 24514.57 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019800.055902 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220631.15 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 22063.11 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041863.170457 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198568.03 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 19856.80 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061719.973557 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178711.23 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 17871.12 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079591.096347 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160840.11 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 16084.01 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095675.106858 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144756.09 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 14475.61 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110150.716317 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130280.49 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 13028.05 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123178.764831 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117252.44 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 11725.24 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134904.008494 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105527.19 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 10552.72 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145456.727790 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94974.47 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 9497.45 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154954.175156 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85477.03 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 8547.70 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163501.877786 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76929.32 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 7692.93 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171194.810153 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69236.39 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 6923.64 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178118.449283 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62312.75 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 6231.28 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184349.724501 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56081.48 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 5608.15 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189957.872196 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50473.33 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 5047.33 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195005.205122 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45426.00 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 4542.60 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199547.804755 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40883.40 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 4088.34 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203636.144425 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36795.06 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3679.51 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207315.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33115.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210915.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29515.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214515.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25915.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218115.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22315.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221715.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18715.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225315.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15115.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228915.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11515.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232515.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7915.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236115.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4315.55 sec) 10:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239715.650128 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 715.55 sec) 10:45:00 [GPUGRID] [rr_sim] 2240431.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985294340.14G/439.78G) 10:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852172.11 10:45:00 [---] pick_jobs_to_run() start 10:45:00 [---] pick_jobs_to_run() end 10:45:00 [---] [rr_sim] end 10:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:45:00 [---] [cpu_sched_debug] preliminary job list: 10:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:45:00 [---] [cpu_sched_debug] final job list: 10:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:45:00 [---] [cpu_sched_debug] enforce_run_list: end 10:45:00 [---] choose_project(): 38700.000000 10:45:00 [iThena.Measurements] doing RPC to get work 10:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396899.09 cpu 2142692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:45:00 [---] Got 0 tasks 10:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:46:00 [---] [rr_sim] doing sim: CPU sched 10:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985266749.78G/439.78G) 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240368.46 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 224036.85 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224036.846457 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016331.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 201633.16 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425670.008269 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814698.46 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 181469.85 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607139.853899 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633228.61 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 163322.86 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770462.714967 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469905.75 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 146990.57 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917453.289927 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322915.17 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 132291.52 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049744.807392 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190623.66 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 119062.37 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168807.173110 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071561.29 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 107156.13 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275963.302256 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964405.16 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 96440.52 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372403.818488 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867964.65 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 86796.46 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459200.283097 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781168.18 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 78116.82 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537317.101244 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703051.36 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 70305.14 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607622.237577 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632746.23 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 63274.62 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670896.860277 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569471.60 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 56947.16 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727844.020706 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512524.44 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 51252.44 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779096.465093 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461272.00 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 46127.20 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825223.665041 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415144.80 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 41514.48 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866738.144994 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373630.32 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 37363.03 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904101.176952 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336267.29 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 33626.73 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937727.905714 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302640.56 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 30264.06 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967991.961600 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272376.50 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 27237.65 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995229.611897 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245138.85 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 24513.89 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019743.497165 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220624.97 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 22062.50 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041805.993906 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198562.47 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 19856.25 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061662.240973 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178706.22 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 17870.62 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079532.863333 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160835.60 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 16083.56 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095616.423457 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144752.04 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 14475.20 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110091.627568 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130276.84 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 13027.68 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123119.311269 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117249.15 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 11724.92 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134844.226599 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105524.24 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 10552.42 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145396.650397 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94971.81 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 9497.18 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154893.831814 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85474.63 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 8547.46 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163441.295090 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76927.17 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 7692.72 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171134.012039 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69234.45 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 6923.45 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178057.457292 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62311.01 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 6231.10 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184288.558020 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56079.91 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 5607.99 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189896.548675 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50471.92 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 5047.19 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194943.740265 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45424.72 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 4542.47 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199486.212696 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40882.25 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 4088.23 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203574.437884 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36794.03 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3679.40 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207253.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33114.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210853.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29514.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214453.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25914.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218053.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22314.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221653.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18714.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225253.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15114.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228853.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11514.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232453.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7914.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236053.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4314.62 sec) 10:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239653.840553 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 714.62 sec) 10:46:00 [GPUGRID] [rr_sim] 2240368.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985266749.78G/439.78G) 10:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852169.38 10:46:00 [---] pick_jobs_to_run() start 10:46:00 [---] pick_jobs_to_run() end 10:46:00 [---] [rr_sim] end 10:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:46:00 [---] [cpu_sched_debug] preliminary job list: 10:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:46:00 [---] [cpu_sched_debug] final job list: 10:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:46:00 [---] [cpu_sched_debug] enforce_run_list: end 10:46:00 [---] choose_project(): 38760.000000 10:46:00 [iThena.Measurements] doing RPC to get work 10:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396839.09 cpu 2142632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:46:00 [---] Got 0 tasks 10:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:47:00 [---] [rr_sim] doing sim: CPU sched 10:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985239159.43G/439.78G) 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240305.73 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 224030.57 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224030.572769 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016275.15 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 201627.52 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425658.088261 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814647.64 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 181464.76 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607122.852204 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633182.88 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 163318.29 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770441.139753 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469864.59 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 146986.46 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917427.598547 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322878.13 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 132287.81 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049715.411461 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190590.32 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 119059.03 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168774.443084 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071531.28 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 107153.13 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275927.571545 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964378.16 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 96437.82 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372365.387159 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867940.34 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 86794.03 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459159.421213 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781146.31 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 78114.63 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537274.051860 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703031.68 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 70303.17 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607577.219444 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632728.51 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 63272.85 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670850.070268 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569455.66 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 56945.57 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727795.636011 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512510.09 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 51251.01 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1779046.645179 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461259.08 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 46125.91 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825172.553430 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415133.17 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 41513.32 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866685.870856 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373619.86 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 37361.99 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904047.856539 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336257.87 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 33625.79 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937673.643655 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302632.08 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 30263.21 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967936.852058 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272368.88 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 27236.89 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995173.739622 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245131.99 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 24513.20 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019686.938429 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220618.79 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 22061.88 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041748.817355 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198556.91 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 19855.69 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061604.508388 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178701.22 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 17870.12 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079474.630319 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160831.10 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 16083.11 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095557.740056 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144747.99 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 14474.80 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110032.538819 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130273.19 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 13027.32 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123059.857707 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117245.87 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 11724.59 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134784.444705 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105521.28 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 10552.13 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145336.573004 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94969.15 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 9496.92 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154833.488472 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85472.24 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 8547.22 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163380.712394 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76925.02 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 7692.50 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171073.213924 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69232.51 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 6923.25 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177996.465301 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62309.26 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 6230.93 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184227.391540 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56078.34 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 5607.83 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189835.225155 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50470.50 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 5047.05 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194882.275408 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45423.45 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 4542.35 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199424.620637 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40881.11 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 4088.11 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203512.731342 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36793.00 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3679.30 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207192.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33113.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210792.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29513.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214392.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25913.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217992.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22313.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221592.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18713.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225192.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15113.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228792.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11513.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232392.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7913.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235992.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4313.70 sec) 10:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239592.030977 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 713.70 sec) 10:47:00 [GPUGRID] [rr_sim] 2240305.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985239159.43G/439.78G) 10:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852166.64 10:47:00 [---] pick_jobs_to_run() start 10:47:00 [---] pick_jobs_to_run() end 10:47:00 [---] [rr_sim] end 10:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:47:00 [---] [cpu_sched_debug] preliminary job list: 10:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:47:00 [---] [cpu_sched_debug] final job list: 10:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:47:00 [---] [cpu_sched_debug] enforce_run_list: end 10:47:00 [---] choose_project(): 38820.000000 10:47:00 [iThena.Measurements] doing RPC to get work 10:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396779.09 cpu 2142572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:47:00 [---] Got 0 tasks 10:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:48:00 [---] [rr_sim] doing sim: CPU sched 10:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985211569.07G/439.78G) 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240242.99 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 224024.30 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224024.299081 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016218.69 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 201621.87 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425646.168254 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814596.82 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 181459.68 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607105.850509 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633137.14 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 163313.71 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770419.564539 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469823.43 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 146982.34 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917401.907166 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322841.08 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 132284.11 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049686.015530 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190556.98 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 119055.70 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168741.713058 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071501.28 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 107150.13 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275891.840833 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964351.15 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 96435.11 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372326.955831 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867916.03 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 86791.60 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459118.559329 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781124.43 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 78112.44 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537231.002477 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703011.99 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 70301.20 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607532.201310 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632710.79 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 63271.08 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670803.280260 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569439.71 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 56943.97 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727747.251315 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512495.74 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 51249.57 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778996.825264 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461246.17 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 46124.62 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825121.441819 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415121.55 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 41512.15 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866633.596718 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373609.39 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 37360.94 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903994.536127 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336248.45 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 33624.85 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937619.381595 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302623.61 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 30262.36 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967881.742516 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272361.25 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 27236.12 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995117.867346 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245125.12 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 24512.51 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019630.379692 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220612.61 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 22061.26 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041691.640804 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198551.35 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 19855.14 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061546.775804 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178696.22 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 17869.62 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079416.397305 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160826.59 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 16082.66 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095499.056655 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144743.93 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 14474.39 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109973.450070 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130269.54 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 13026.95 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123000.404144 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117242.59 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 11724.26 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134724.662811 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105518.33 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 10551.83 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145276.495611 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94966.50 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 9496.65 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154773.145130 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85469.85 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 8546.98 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163320.129698 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76922.86 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 7692.29 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171012.415809 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69230.57 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 6923.06 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177935.473309 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62307.52 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 6230.75 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184166.225059 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56076.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 5607.68 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189773.901634 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50469.09 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 5046.91 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194820.810552 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45422.18 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 4542.22 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199363.028577 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40879.96 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 4088.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203451.024801 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36791.97 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3679.20 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207130.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33112.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210730.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29512.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214330.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25912.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217930.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22312.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221530.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18712.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225130.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15112.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228730.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11512.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232330.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7912.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235930.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4312.77 sec) 10:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239530.221401 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 712.77 sec) 10:48:00 [GPUGRID] [rr_sim] 2240242.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985211569.07G/439.78G) 10:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852163.90 10:48:00 [---] pick_jobs_to_run() start 10:48:00 [---] pick_jobs_to_run() end 10:48:00 [---] [rr_sim] end 10:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:48:00 [---] [cpu_sched_debug] preliminary job list: 10:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:48:00 [---] [cpu_sched_debug] final job list: 10:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:48:00 [---] [cpu_sched_debug] enforce_run_list: end 10:48:00 [---] choose_project(): 38880.000000 10:48:00 [iThena.Measurements] doing RPC to get work 10:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396719.09 cpu 2142512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:48:00 [---] Got 0 tasks 10:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:49:00 [---] [rr_sim] doing sim: CPU sched 10:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985183978.71G/439.78G) 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240180.25 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 224018.03 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224018.025393 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016162.23 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 201616.22 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425634.248246 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814546.01 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 181454.60 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607088.848814 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633091.41 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 163309.14 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770397.989325 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469782.26 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 146978.23 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917376.215786 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322804.04 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 132280.40 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049656.619600 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190523.63 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 119052.36 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168708.983032 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071471.27 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 107147.13 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275856.110122 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964324.14 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 96432.41 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372288.524502 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867891.73 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 86789.17 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459077.697445 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781102.56 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 78110.26 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537187.953093 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702992.30 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 70299.23 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607487.183176 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632693.07 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 63269.31 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670756.490251 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569423.76 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 56942.38 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727698.866619 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512481.39 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 51248.14 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778947.005350 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461233.25 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 46123.32 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825070.330207 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415109.92 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 41510.99 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866581.322579 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373598.93 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 37359.89 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903941.215714 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336239.04 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 33623.90 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937565.119535 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302615.13 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 30261.51 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967826.632974 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272353.62 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 27235.36 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995061.995070 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245118.26 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 24511.83 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019573.820955 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220606.43 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 22060.64 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041634.464253 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198545.79 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 19854.58 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061489.043220 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178691.21 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 17869.12 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079358.164291 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160822.09 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 16082.21 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095440.373254 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144739.88 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 14473.99 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109914.361321 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130265.89 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 13026.59 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122940.950582 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117239.30 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 11723.93 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134664.880916 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105515.37 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 10551.54 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145216.418217 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94963.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 9496.38 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154712.801788 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85467.45 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 8546.75 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163259.547002 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76920.71 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 7692.07 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170951.617695 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69228.64 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 6922.86 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177874.481318 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62305.77 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 6230.58 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184105.058579 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56075.20 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 5607.52 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189712.578114 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50467.68 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 5046.77 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194759.345695 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45420.91 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 4542.09 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199301.436518 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40878.82 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 4087.88 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203389.318259 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36790.94 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3679.09 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207068.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33111.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210668.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29511.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214268.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25911.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217868.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22311.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221468.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18711.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225068.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15111.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228668.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11511.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232268.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7911.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235868.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4311.84 sec) 10:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239468.411826 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 711.84 sec) 10:49:00 [GPUGRID] [rr_sim] 2240180.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985183978.71G/439.78G) 10:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852161.17 10:49:00 [---] pick_jobs_to_run() start 10:49:00 [---] pick_jobs_to_run() end 10:49:00 [---] [rr_sim] end 10:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:49:00 [---] [cpu_sched_debug] preliminary job list: 10:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:49:00 [---] [cpu_sched_debug] final job list: 10:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:49:00 [---] [cpu_sched_debug] enforce_run_list: end 10:49:00 [---] choose_project(): 38940.000000 10:49:00 [iThena.Measurements] doing RPC to get work 10:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396659.09 cpu 2142452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:49:00 [---] Got 0 tasks 10:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:50:00 [---] [rr_sim] doing sim: CPU sched 10:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985156388.36G/439.78G) 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240117.52 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 224011.75 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224011.751704 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016105.77 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 201610.58 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425622.328238 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814495.19 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 181449.52 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607071.847119 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1633045.67 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 163304.57 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770376.414112 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469741.10 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 146974.11 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917350.524405 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322766.99 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 132276.70 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049627.223669 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190490.29 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 119049.03 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168676.253006 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071441.26 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 107144.13 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275820.379410 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964297.14 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 96429.71 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372250.093174 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867867.42 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 86786.74 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1459036.835561 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781080.68 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 78108.07 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537144.903709 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702972.61 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 70297.26 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607442.165043 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632675.35 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 63267.54 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670709.700243 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569407.82 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 56940.78 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727650.481923 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512467.04 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 51246.70 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778897.185435 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461220.33 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 46122.03 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825019.218596 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415098.30 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 41509.83 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866529.048441 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373588.47 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 37358.85 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903887.895301 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336229.62 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 33622.96 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937510.857476 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302606.66 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 30260.67 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967771.523433 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272345.99 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 27234.60 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995006.122794 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245111.39 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 24511.14 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019517.262219 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220600.25 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 22060.03 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041577.287701 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198540.23 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 19854.02 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061431.310636 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178686.21 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 17868.62 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079299.931277 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160817.59 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 16081.76 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095381.689853 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144735.83 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 14473.58 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109855.272573 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130262.24 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 13026.22 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122881.497020 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117236.02 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 11723.60 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134605.099022 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105512.42 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 10551.24 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145156.340824 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94961.18 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 9496.12 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154652.458446 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85465.06 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 8546.51 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163198.964306 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76918.55 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 7691.86 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170890.819580 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69226.70 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 6922.67 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177813.489327 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62304.03 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 6230.40 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184043.892098 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56073.62 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 5607.36 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189651.254593 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50466.26 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 5046.63 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194697.880838 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45419.64 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 4541.96 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199239.844459 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40877.67 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 4087.77 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203327.611717 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36789.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3678.99 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207006.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33110.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210606.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29510.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214206.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25910.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217806.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22310.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221406.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18710.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225006.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15110.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228606.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11510.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232206.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7910.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235806.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4310.91 sec) 10:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239406.602250 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 710.91 sec) 10:50:00 [GPUGRID] [rr_sim] 2240117.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985156388.36G/439.78G) 10:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852158.43 10:50:00 [---] pick_jobs_to_run() start 10:50:00 [---] pick_jobs_to_run() end 10:50:00 [---] [rr_sim] end 10:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:50:00 [---] [cpu_sched_debug] preliminary job list: 10:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:50:00 [---] [cpu_sched_debug] final job list: 10:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:50:00 [---] [cpu_sched_debug] enforce_run_list: end 10:50:00 [---] choose_project(): 39000.000000 10:50:00 [iThena.Measurements] doing RPC to get work 10:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396599.09 cpu 2142392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:50:00 [---] Got 0 tasks 10:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:51:00 [---] [rr_sim] doing sim: CPU sched 10:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985128798.00G/439.78G) 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2240054.78 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 224005.48 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 224005.478016 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2016049.30 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 201604.93 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425610.408231 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814444.37 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 181444.44 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607054.845424 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632999.93 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 163299.99 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770354.838898 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469699.94 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 146969.99 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917324.833024 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322729.95 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 132272.99 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049597.827738 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190456.95 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 119045.70 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168643.522981 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071411.26 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 107141.13 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275784.648699 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964270.13 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 96427.01 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372211.661845 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867843.12 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 86784.31 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458995.973677 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781058.81 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 78105.88 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537101.854325 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702952.93 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 70295.29 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607397.146909 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632657.63 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 63265.76 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670662.910234 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569391.87 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 56939.19 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727602.097227 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512452.68 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 51245.27 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778847.365521 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461207.41 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 46120.74 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824968.106985 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415086.67 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 41508.67 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866476.774303 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373578.01 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 37357.80 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903834.574889 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336220.21 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 33622.02 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937456.595416 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302598.18 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 30259.82 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967716.413891 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272338.37 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 27233.84 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994950.250518 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245104.53 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 24510.45 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019460.703482 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220594.08 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 22059.41 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041520.111150 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198534.67 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 19853.47 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061373.578051 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178681.20 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 17868.12 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079241.698263 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160813.08 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 16081.31 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095323.006453 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144731.77 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 14473.18 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109796.183824 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130258.60 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 13025.86 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122822.043457 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117232.74 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 11723.27 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134545.317128 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105509.46 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 10550.95 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145096.263431 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94958.52 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 9495.85 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154592.115104 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85462.67 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 8546.27 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163138.381610 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76916.40 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 7691.64 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170830.021465 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69224.76 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 6922.48 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177752.497335 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62302.28 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 6230.23 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183982.725618 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56072.05 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 5607.21 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189589.931072 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50464.85 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 5046.48 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194636.415981 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45418.36 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 4541.84 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199178.252399 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40876.53 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 4087.65 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203265.905176 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36788.87 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3678.89 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206944.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33109.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210544.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29509.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214144.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25909.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217744.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22309.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221344.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18709.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224944.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15109.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228544.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11509.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232144.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7909.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235744.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4309.99 sec) 10:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239344.792674 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709.99 sec) 10:51:00 [GPUGRID] [rr_sim] 2240054.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985128798.00G/439.78G) 10:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852155.69 10:51:00 [---] pick_jobs_to_run() start 10:51:00 [---] pick_jobs_to_run() end 10:51:00 [---] [rr_sim] end 10:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:51:00 [---] [cpu_sched_debug] preliminary job list: 10:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:51:00 [---] [cpu_sched_debug] final job list: 10:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:51:00 [---] [cpu_sched_debug] enforce_run_list: end 10:51:00 [---] choose_project(): 39060.000000 10:51:00 [iThena.Measurements] doing RPC to get work 10:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396539.09 cpu 2142332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:51:00 [---] Got 0 tasks 10:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:52:00 [---] [rr_sim] doing sim: CPU sched 10:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985101207.65G/439.78G) 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239992.04 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 223999.20 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223999.204328 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015992.84 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 201599.28 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425598.488223 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814393.56 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 181439.36 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607037.843729 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632954.20 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 163295.42 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770333.263684 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469658.78 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 146965.88 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917299.141644 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322692.90 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 132269.29 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049568.431807 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190423.61 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 119042.36 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168610.792955 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071381.25 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 107138.13 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275748.917987 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964243.13 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 96424.31 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372173.230516 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867818.81 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 86781.88 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458955.111793 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781036.93 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 78103.69 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537058.804942 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702933.24 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 70293.32 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607352.128775 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632639.91 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 63263.99 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670616.120226 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569375.92 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 56937.59 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727553.712531 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512438.33 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 51243.83 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778797.545606 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461194.50 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 46119.45 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824916.995374 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415075.05 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 41507.50 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866424.500164 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373567.54 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 37356.75 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903781.254476 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336210.79 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 33621.08 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937402.333356 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302589.71 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 30258.97 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967661.304349 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272330.74 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 27233.07 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994894.378242 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245097.67 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 24509.77 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019404.144746 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220587.90 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 22058.79 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041462.934599 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198529.11 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 19852.91 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061315.845467 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178676.20 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 17867.62 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079183.465249 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160808.58 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 16080.86 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095264.323052 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144727.72 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 14472.77 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109737.095075 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130254.95 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 13025.49 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122762.589895 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117229.45 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 11722.95 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134485.535234 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105506.51 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 10550.65 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145036.186038 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94955.86 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 9495.59 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154531.771762 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85460.27 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 8546.03 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163077.798914 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76914.24 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 7691.42 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170769.223351 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69222.82 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 6922.28 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177691.505344 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62300.54 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 6230.05 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183921.559137 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56070.48 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 5607.05 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189528.607552 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50463.44 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 5046.34 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194574.951125 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45417.09 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 4541.71 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199116.660340 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40875.38 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 4087.54 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203204.198634 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36787.84 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3678.78 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206882.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33109.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210482.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29509.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214082.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25909.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217682.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22309.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221282.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18709.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224882.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15109.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228482.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11509.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232082.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7909.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235682.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4309.06 sec) 10:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239282.983099 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 709.06 sec) 10:52:00 [GPUGRID] [rr_sim] 2239992.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985101207.65G/439.78G) 10:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852152.96 10:52:00 [---] pick_jobs_to_run() start 10:52:00 [---] pick_jobs_to_run() end 10:52:00 [---] [rr_sim] end 10:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:52:00 [---] [cpu_sched_debug] preliminary job list: 10:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:52:00 [---] [cpu_sched_debug] final job list: 10:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:52:00 [---] [cpu_sched_debug] enforce_run_list: end 10:52:00 [---] choose_project(): 39120.000000 10:52:00 [iThena.Measurements] doing RPC to get work 10:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396479.09 cpu 2142272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:52:00 [---] Got 0 tasks 10:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:53:00 [---] [rr_sim] doing sim: CPU sched 10:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985073617.29G/439.78G) 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239929.31 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 223992.93 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223992.930640 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015936.38 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 201593.64 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425586.568216 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814342.74 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 181434.27 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607020.842034 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632908.46 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 163290.85 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770311.688470 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469617.62 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 146961.76 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917273.450263 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322655.86 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 132265.59 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049539.035877 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190390.27 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 119039.03 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168578.062929 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071351.24 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 107135.12 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275713.187276 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964216.12 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 96421.61 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372134.799188 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867794.51 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 86779.45 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458914.249909 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 781015.06 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 78101.51 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1537015.755558 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702913.55 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 70291.36 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607307.110642 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632622.20 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 63262.22 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670569.330217 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569359.98 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 56936.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727505.327836 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512423.98 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 51242.40 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778747.725692 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461181.58 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 46118.16 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824865.883762 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415063.42 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 41506.34 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866372.226026 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373557.08 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 37355.71 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903727.934063 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336201.37 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 33620.14 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937348.071297 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302581.24 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 30258.12 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967606.194807 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272323.11 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 27232.31 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994838.505966 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245090.80 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 24509.08 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019347.586009 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220581.72 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 22058.17 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041405.758048 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198523.55 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 19852.35 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061258.112883 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178671.19 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 17867.12 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079125.232235 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160804.07 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 16080.41 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095205.639651 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144723.67 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 14472.37 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109678.006326 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130251.30 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 13025.13 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122703.136333 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117226.17 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 11722.62 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134425.753339 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105503.55 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 10550.36 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144976.108645 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94953.20 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 9495.32 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154471.428421 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85457.88 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 8545.79 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163017.216218 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76912.09 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 7691.21 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170708.425236 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69220.88 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 6922.09 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177630.513352 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62298.79 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 6229.88 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183860.392657 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56068.91 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 5606.89 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189467.284031 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50462.02 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 5046.20 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194513.486268 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45415.82 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 4541.58 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199055.068281 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40874.24 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 4087.42 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203142.492093 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36786.81 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3678.68 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206821.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33108.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210421.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29508.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214021.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25908.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217621.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22308.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221221.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18708.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224821.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15108.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228421.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11508.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232021.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7908.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235621.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4308.13 sec) 10:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239221.173523 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 708.13 sec) 10:53:00 [GPUGRID] [rr_sim] 2239929.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985073617.29G/439.78G) 10:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852150.22 10:53:00 [---] pick_jobs_to_run() start 10:53:00 [---] pick_jobs_to_run() end 10:53:00 [---] [rr_sim] end 10:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:53:00 [---] [cpu_sched_debug] preliminary job list: 10:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:53:00 [---] [cpu_sched_debug] final job list: 10:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:53:00 [---] [cpu_sched_debug] enforce_run_list: end 10:53:00 [---] choose_project(): 39180.000000 10:53:00 [iThena.Measurements] doing RPC to get work 10:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396419.09 cpu 2142212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:53:00 [---] Got 0 tasks 10:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:54:00 [---] [rr_sim] doing sim: CPU sched 10:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985046026.93G/439.78G) 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239866.57 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 223986.66 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223986.656952 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015879.91 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 201587.99 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425574.648208 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814291.92 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 181429.19 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 607003.840339 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632862.73 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 163286.27 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770290.113257 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469576.46 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 146957.65 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917247.758882 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322618.81 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 132261.88 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049509.639946 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190356.93 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 119035.69 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168545.332903 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071321.24 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 107132.12 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275677.456564 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964189.11 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 96418.91 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372096.367859 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867770.20 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 86777.02 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458873.388025 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780993.18 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 78099.32 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536972.706174 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702893.86 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 70289.39 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607262.092508 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632604.48 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 63260.45 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670522.540209 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569344.03 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 56934.40 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727456.943140 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512409.63 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 51240.96 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778697.905777 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461168.66 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 46116.87 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824814.772151 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415051.80 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 41505.18 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866319.951888 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373546.62 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 37354.66 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903674.613650 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336191.96 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 33619.20 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937293.809237 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302572.76 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 30257.28 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967551.085265 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272315.48 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 27231.55 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994782.633690 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245083.94 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 24508.39 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019291.027273 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220575.54 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 22057.55 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041348.581497 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198517.99 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 19851.80 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061200.380299 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178666.19 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 17866.62 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079066.999221 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160799.57 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 16079.96 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095146.956250 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144719.61 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 14471.96 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109618.917577 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130247.65 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 13024.77 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122643.682771 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117222.89 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 11722.29 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134365.971445 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105500.60 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 10550.06 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144916.031252 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94950.54 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 9495.05 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154411.085079 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85455.48 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 8545.55 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162956.633522 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76909.94 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 7690.99 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170647.627122 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69218.94 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 6921.89 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177569.521361 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62297.05 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 6229.70 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183799.226177 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56067.34 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 5606.73 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189405.960510 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50460.61 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 5046.06 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194452.021411 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45414.55 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 4541.45 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198993.476222 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40873.09 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 4087.31 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203080.785551 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36785.78 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3678.58 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206759.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33107.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210359.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29507.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213959.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25907.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217559.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22307.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221159.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18707.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224759.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15107.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228359.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11507.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231959.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7907.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235559.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4307.21 sec) 10:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239159.363947 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 707.21 sec) 10:54:00 [GPUGRID] [rr_sim] 2239866.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985046026.93G/439.78G) 10:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852147.48 10:54:00 [---] pick_jobs_to_run() start 10:54:00 [---] pick_jobs_to_run() end 10:54:00 [---] [rr_sim] end 10:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:54:00 [---] [cpu_sched_debug] preliminary job list: 10:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:54:00 [---] [cpu_sched_debug] final job list: 10:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:54:00 [---] [cpu_sched_debug] enforce_run_list: end 10:54:00 [---] choose_project(): 39240.000000 10:54:00 [iThena.Measurements] doing RPC to get work 10:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396359.09 cpu 2142152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:54:00 [---] Got 0 tasks 10:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:55:00 [---] [rr_sim] doing sim: CPU sched 10:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (985018436.58G/439.78G) 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239803.83 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 223980.38 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223980.383263 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015823.45 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 201582.34 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425562.728200 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814241.10 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 181424.11 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606986.838644 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632816.99 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 163281.70 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770268.538043 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469535.29 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 146953.53 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917222.067502 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322581.77 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 132258.18 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049480.244015 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190323.59 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 119032.36 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168512.602877 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071291.23 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 107129.12 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275641.725853 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964162.11 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 96416.21 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372057.936531 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867745.90 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 86774.59 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458832.526141 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780971.31 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 78097.13 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536929.656790 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702874.18 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 70287.42 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607217.074375 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632586.76 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 63258.68 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670475.750201 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569328.08 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 56932.81 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727408.558444 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512395.27 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 51239.53 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778648.085863 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461155.75 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 46115.57 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824763.660540 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415040.17 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 41504.02 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866267.677749 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373536.15 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 37353.62 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903621.293238 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336182.54 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 33618.25 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937239.547177 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302564.29 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 30256.43 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967495.975723 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272307.86 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 27230.79 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994726.761414 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245077.07 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 24507.71 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019234.468536 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220569.36 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 22056.94 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041291.404946 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198512.43 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 19851.24 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061142.647715 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178661.18 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 17866.12 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079008.766207 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160795.07 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 16079.51 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095088.272849 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144715.56 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 14471.56 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109559.828828 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130244.00 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 13024.40 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122584.229208 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117219.60 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 11721.96 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134306.189551 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105497.64 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 10549.76 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144855.953859 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94947.88 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 9494.79 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154350.741737 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85453.09 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 8545.31 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162896.050826 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76907.78 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 7690.78 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170586.829007 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69217.00 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 6921.70 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177508.529370 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62295.30 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 6229.53 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183738.059696 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56065.77 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 5606.58 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189344.636990 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50459.20 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 5045.92 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194390.556554 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45413.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 4541.33 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198931.884162 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40871.95 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 4087.19 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203019.079009 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36784.75 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3678.48 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206697.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33106.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210297.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29506.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213897.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25906.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217497.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22306.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221097.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18706.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224697.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15106.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228297.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11506.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231897.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7906.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235497.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4306.28 sec) 10:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239097.554372 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 706.28 sec) 10:55:00 [GPUGRID] [rr_sim] 2239803.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (985018436.58G/439.78G) 10:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852144.75 10:55:00 [---] pick_jobs_to_run() start 10:55:00 [---] pick_jobs_to_run() end 10:55:00 [---] [rr_sim] end 10:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:55:00 [---] [cpu_sched_debug] preliminary job list: 10:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:55:00 [---] [cpu_sched_debug] final job list: 10:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:55:00 [---] [cpu_sched_debug] enforce_run_list: end 10:55:00 [---] choose_project(): 39300.000000 10:55:00 [iThena.Measurements] doing RPC to get work 10:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396299.09 cpu 2142092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:55:00 [---] Got 0 tasks 10:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:56:00 [---] [rr_sim] doing sim: CPU sched 10:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984990846.22G/439.78G) 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239741.10 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 223974.11 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223974.109575 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015766.99 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 201576.70 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425550.808193 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814190.29 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 181419.03 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606969.836949 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632771.26 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 163277.13 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770246.962829 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469494.13 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 146949.41 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917196.376121 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322544.72 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 132254.47 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049450.848084 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190290.25 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 119029.02 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168479.872851 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071261.22 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 107126.12 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275605.995141 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964135.10 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 96413.51 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1372019.505202 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867721.59 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 86772.16 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458791.664257 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780949.43 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 78094.94 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536886.607406 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702854.49 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 70285.45 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607172.056241 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632569.04 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 63256.90 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670428.960192 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569312.14 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 56931.21 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727360.173748 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512380.92 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 51238.09 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778598.265948 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461142.83 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 46114.28 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824712.548929 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415028.55 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 41502.85 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866215.403611 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373525.69 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 37352.57 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903567.972825 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336173.12 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 33617.31 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937185.285118 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302555.81 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 30255.58 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967440.866181 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272300.23 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 27230.02 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994670.889138 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245070.21 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 24507.02 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019177.909799 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220563.19 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 22056.32 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041234.228395 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198506.87 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 19850.69 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061084.915130 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178656.18 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 17865.62 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078950.533192 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160790.56 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 16079.06 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095029.589448 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144711.51 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 14471.15 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109500.740079 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130240.36 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 13024.04 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122524.775646 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117216.32 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 11721.63 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134246.407657 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105494.69 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 10549.47 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144795.876466 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94945.22 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 9494.52 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154290.398395 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85450.70 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 8545.07 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162835.468130 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76905.63 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 7690.56 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170526.030892 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69215.06 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 6921.51 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177447.537378 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62293.56 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 6229.36 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183676.893216 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56064.20 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 5606.42 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189283.313469 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50457.78 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 5045.78 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194329.091697 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45412.00 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 4541.20 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198870.292103 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40870.80 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 4087.08 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202957.372468 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36783.72 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3678.37 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206635.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33105.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210235.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29505.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213835.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25905.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217435.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22305.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221035.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18705.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224635.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15105.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228235.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11505.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231835.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7905.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235435.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4305.35 sec) 10:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2239035.744796 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 705.35 sec) 10:56:00 [GPUGRID] [rr_sim] 2239741.10: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984990846.22G/439.78G) 10:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852142.01 10:56:00 [---] pick_jobs_to_run() start 10:56:00 [---] pick_jobs_to_run() end 10:56:00 [---] [rr_sim] end 10:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:56:00 [---] [cpu_sched_debug] preliminary job list: 10:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:56:00 [---] [cpu_sched_debug] final job list: 10:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:56:00 [---] [cpu_sched_debug] enforce_run_list: end 10:56:00 [---] choose_project(): 39360.000000 10:56:00 [iThena.Measurements] doing RPC to get work 10:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 569039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396239.09 cpu 2142032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2142032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2142032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:56:00 [---] Got 0 tasks 10:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:57:00 [---] [rr_sim] doing sim: CPU sched 10:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984963255.86G/439.78G) 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239678.36 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 223967.84 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223967.835887 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015710.52 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 201571.05 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425538.888185 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814139.47 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 181413.95 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606952.835254 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632725.52 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 163272.55 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770225.387615 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469452.97 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 146945.30 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917170.684741 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322507.67 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 132250.77 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049421.452153 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190256.91 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 119025.69 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168447.142825 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071231.22 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 107123.12 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275570.264430 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964108.09 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 96410.81 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371981.073873 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867697.28 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 86769.73 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458750.802373 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780927.56 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 78092.76 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536843.558023 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702834.80 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 70283.48 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607127.038107 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632551.32 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 63255.13 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670382.170184 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569296.19 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 56929.62 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727311.789052 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512366.57 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 51236.66 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778548.446034 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461129.91 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 46112.99 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824661.437317 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415016.92 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 41501.69 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866163.129473 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373515.23 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 37351.52 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903514.652412 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336163.71 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 33616.37 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937131.023058 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302547.34 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 30254.73 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967385.756639 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272292.60 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 27229.26 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994615.016862 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245063.34 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 24506.33 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019121.351063 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220557.01 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 22055.70 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041177.051844 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198501.31 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 19850.13 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061027.182546 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178651.18 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 17865.12 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078892.300178 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160786.06 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 16078.61 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094970.906048 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144707.45 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 14470.75 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109441.651330 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130236.71 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 13023.67 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122465.322084 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117213.04 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 11721.30 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134186.625762 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105491.73 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 10549.17 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144735.799073 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94942.56 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 9494.26 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154230.055053 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85448.30 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 8544.83 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162774.885434 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76903.47 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 7690.35 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170465.232778 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69213.13 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 6921.31 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177386.545387 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62291.81 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 6229.18 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183615.726735 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56062.63 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 5606.26 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189221.989949 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50456.37 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 5045.64 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194267.626841 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45410.73 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 4541.07 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198808.700044 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40869.66 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 4086.97 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202895.665926 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36782.69 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3678.27 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206573.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33104.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210173.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29504.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213773.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25904.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217373.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22304.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220973.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18704.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224573.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15104.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228173.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11504.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231773.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7904.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235373.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4304.42 sec) 10:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238973.935220 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 704.42 sec) 10:57:00 [GPUGRID] [rr_sim] 2239678.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984963255.86G/439.78G) 10:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852139.27 10:57:00 [---] pick_jobs_to_run() start 10:57:00 [---] pick_jobs_to_run() end 10:57:00 [---] [rr_sim] end 10:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:57:00 [---] [cpu_sched_debug] preliminary job list: 10:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:57:00 [---] [cpu_sched_debug] final job list: 10:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:57:00 [---] [cpu_sched_debug] enforce_run_list: end 10:57:00 [---] choose_project(): 39420.000000 10:57:00 [iThena.Measurements] doing RPC to get work 10:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396179.09 cpu 2141972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:57:00 [---] Got 0 tasks 10:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:58:00 [---] [rr_sim] doing sim: CPU sched 10:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984935665.51G/439.78G) 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239615.62 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 223961.56 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223961.562199 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015654.06 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 201565.41 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425526.968178 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814088.65 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 181408.87 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606935.833559 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632679.79 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 163267.98 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770203.812401 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469411.81 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 146941.18 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917144.993360 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322470.63 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 132247.06 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049392.056223 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190223.57 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 119022.36 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168414.412799 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071201.21 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 107120.12 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275534.533718 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964081.09 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 96408.11 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371942.642545 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867672.98 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 86767.30 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458709.940489 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780905.68 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 78090.57 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536800.508639 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702815.11 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 70281.51 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607082.019974 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632533.60 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 63253.36 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670335.380175 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569280.24 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 56928.02 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727263.404356 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512352.22 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 51235.22 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778498.626119 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461117.00 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 46111.70 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824610.325706 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415005.30 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 41500.53 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866110.855334 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373504.77 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 37350.48 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903461.332000 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336154.29 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 33615.43 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937076.760998 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302538.86 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 30253.89 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967330.647097 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272284.97 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 27228.50 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994559.144586 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245056.48 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 24505.65 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019064.792326 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220550.83 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 22055.08 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041119.875292 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198495.75 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 19849.57 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060969.449962 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178646.17 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 17864.62 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078834.067164 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160781.55 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 16078.16 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094912.222647 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144703.40 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 14470.34 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109382.562581 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130233.06 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 13023.31 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122405.868521 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117209.75 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 11720.98 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134126.843868 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105488.78 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 10548.88 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144675.721680 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94939.90 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 9493.99 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154169.711711 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85445.91 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 8544.59 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162714.302738 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76901.32 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 7690.13 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170404.434663 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69211.19 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 6921.12 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177325.553396 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62290.07 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 6229.01 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183554.560255 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56061.06 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 5606.11 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189160.666428 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50454.96 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 5045.50 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194206.161984 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45409.46 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 4540.95 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198747.107984 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40868.51 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 4086.85 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202833.959385 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36781.66 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3678.17 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206512.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33103.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210112.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29503.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213712.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25903.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217312.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22303.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220912.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18703.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224512.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15103.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228112.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11503.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231712.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7903.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235312.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4303.50 sec) 10:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238912.125645 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 703.50 sec) 10:58:00 [GPUGRID] [rr_sim] 2239615.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984935665.51G/439.78G) 10:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852136.53 10:58:00 [---] pick_jobs_to_run() start 10:58:00 [---] pick_jobs_to_run() end 10:58:00 [---] [rr_sim] end 10:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:58:00 [---] [cpu_sched_debug] preliminary job list: 10:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:58:00 [---] [cpu_sched_debug] final job list: 10:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:58:00 [---] [cpu_sched_debug] enforce_run_list: end 10:58:00 [---] choose_project(): 39480.000000 10:58:00 [iThena.Measurements] doing RPC to get work 10:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396119.09 cpu 2141912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:58:00 [---] Got 0 tasks 10:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 10:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 10:59:00 [---] [rr_sim] doing sim: CPU sched 10:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984908075.15G/439.78G) 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239552.89 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 223955.29 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223955.288511 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015597.60 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 201559.76 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425515.048170 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1814037.84 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 181403.78 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606918.831863 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632634.05 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 163263.41 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770182.237188 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469370.65 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 146937.06 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917119.301979 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322433.58 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 132243.36 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049362.660292 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190190.22 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 119019.02 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168381.682773 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071171.20 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 107117.12 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275498.803006 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964054.08 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 96405.41 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371904.211216 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867648.67 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 86764.87 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458669.078605 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780883.81 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 78088.38 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536757.459255 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702795.43 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 70279.54 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1607037.001840 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632515.88 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 63251.59 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670288.590167 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569264.29 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 56926.43 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727215.019660 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512337.87 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 51233.79 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778448.806205 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461104.08 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 46110.41 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824559.214095 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414993.67 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 41499.37 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866058.581196 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373494.30 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 37349.43 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903408.011587 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336144.87 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 33614.49 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937022.498939 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302530.39 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 30253.04 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967275.537555 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272277.35 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 27227.73 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994503.272310 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245049.61 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 24504.96 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019008.233590 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220544.65 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 22054.47 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041062.698741 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198490.19 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 19849.02 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060911.717378 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178641.17 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 17864.12 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078775.834150 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160777.05 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 16077.71 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094853.539246 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144699.35 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 14469.93 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109323.473832 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130229.41 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 13022.94 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122346.414959 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117206.47 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 11720.65 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134067.061974 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105485.82 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 10548.58 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144615.644287 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94937.24 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 9493.72 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154109.368369 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85443.52 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 8544.35 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162653.720042 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76899.17 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 7689.92 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170343.636549 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69209.25 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 6920.92 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177264.561404 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62288.32 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 6228.83 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183493.393774 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56059.49 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 5605.95 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189099.342907 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50453.54 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 5045.35 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194144.697127 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45408.19 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 4540.82 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198685.515925 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40867.37 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 4086.74 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202772.252843 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36780.63 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3678.06 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206450.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33102.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210050.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29502.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213650.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25902.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217250.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22302.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220850.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18702.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224450.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15102.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228050.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11502.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231650.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7902.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235250.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4302.57 sec) 10:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238850.316069 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702.57 sec) 10:59:00 [GPUGRID] [rr_sim] 2239552.89: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984908075.15G/439.78G) 10:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852133.80 10:59:00 [---] pick_jobs_to_run() start 10:59:00 [---] pick_jobs_to_run() end 10:59:00 [---] [rr_sim] end 10:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 10:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 10:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 10:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 10:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 10:59:00 [---] [cpu_sched_debug] preliminary job list: 10:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:59:00 [---] [cpu_sched_debug] final job list: 10:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 10:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 10:59:00 [---] [cpu_sched_debug] enforce_run_list: end 10:59:00 [---] choose_project(): 39540.000000 10:59:00 [iThena.Measurements] doing RPC to get work 10:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 396059.09 cpu 2141852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 10:59:00 [---] Got 0 tasks 11:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:00:00 [---] [rr_sim] doing sim: CPU sched 11:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984880484.80G/439.78G) 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239490.15 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 223949.01 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223949.014822 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015541.13 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 201554.11 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425503.128162 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813987.02 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 181398.70 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606901.830168 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632588.32 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 163258.83 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770160.661974 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469329.49 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 146932.95 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917093.610599 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322396.54 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 132239.65 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049333.264361 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190156.88 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 119015.69 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168348.952747 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071141.20 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 107114.12 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275463.072295 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964027.08 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 96402.71 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371865.779888 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867624.37 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 86762.44 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458628.216721 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780861.93 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 78086.19 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536714.409871 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702775.74 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 70277.57 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606991.983707 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632498.16 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 63249.82 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670241.800158 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569248.35 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 56924.83 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727166.634965 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512323.51 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 51232.35 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778398.986290 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461091.16 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 46109.12 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824508.102484 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414982.05 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 41498.20 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866006.307058 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373483.84 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 37348.38 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903354.691174 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336135.46 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 33613.55 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936968.236879 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302521.91 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 30252.19 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967220.428013 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272269.72 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 27226.97 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994447.400034 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245042.75 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 24504.27 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018951.674853 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220538.47 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 22053.85 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041005.522190 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198484.63 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 19848.46 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060853.984793 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178636.16 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 17863.62 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078717.601136 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160772.55 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 16077.25 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094794.855845 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144695.29 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 14469.53 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109264.385083 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130225.76 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 13022.58 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122286.961397 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117203.19 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 11720.32 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134007.280079 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105482.87 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 10548.29 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144555.566894 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94934.58 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 9493.46 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154049.025027 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85441.12 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 8544.11 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162593.137346 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76897.01 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 7689.70 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170282.838434 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69207.31 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 6920.73 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177203.569413 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62286.58 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 6228.66 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183432.227294 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56057.92 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 5605.79 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189038.019387 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50452.13 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 5045.21 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194083.232270 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45406.92 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 4540.69 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198623.923866 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40866.22 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 4086.62 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202710.546301 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36779.60 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3677.96 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206388.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33101.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209988.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29501.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213588.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25901.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217188.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22301.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220788.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18701.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224388.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15101.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227988.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11501.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231588.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7901.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235188.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4301.64 sec) 11:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238788.506494 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701.64 sec) 11:00:00 [GPUGRID] [rr_sim] 2239490.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984880484.80G/439.78G) 11:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852131.06 11:00:00 [---] pick_jobs_to_run() start 11:00:00 [---] pick_jobs_to_run() end 11:00:00 [---] [rr_sim] end 11:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:00:00 [---] [cpu_sched_debug] preliminary job list: 11:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:00:00 [---] [cpu_sched_debug] final job list: 11:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:00:00 [---] [cpu_sched_debug] enforce_run_list: end 11:00:00 [---] choose_project(): 39600.000000 11:00:00 [iThena.Measurements] doing RPC to get work 11:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395999.09 cpu 2141792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:00:00 [---] Got 0 tasks 11:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:01:00 [---] [rr_sim] doing sim: CPU sched 11:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984852894.44G/439.78G) 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239427.41 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 223942.74 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223942.741134 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015484.67 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 201548.47 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425491.208155 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813936.20 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 181393.62 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606884.828473 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632542.58 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 163254.26 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770139.086760 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469288.32 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 146928.83 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917067.919218 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322359.49 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 132235.95 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049303.868430 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190123.54 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 119012.35 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168316.222721 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071111.19 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 107111.12 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275427.341583 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 964000.07 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 96400.01 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371827.348559 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867600.06 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 86760.01 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458587.354837 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780840.06 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 78084.01 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536671.360488 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702756.05 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 70275.61 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606946.965573 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632480.45 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 63248.04 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670195.010150 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569232.40 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 56923.24 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727118.250269 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512309.16 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 51230.92 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778349.166376 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461078.24 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 46107.82 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824456.990872 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414970.42 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 41497.04 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865954.032919 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373473.38 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 37347.34 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903301.370761 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336126.04 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 33612.60 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936913.974819 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302513.44 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 30251.34 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967165.318472 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272262.09 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 27226.21 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994391.527758 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245035.88 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 24503.59 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018895.116117 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220532.30 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 22053.23 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040948.345639 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198479.07 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 19847.91 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060796.252209 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178631.16 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 17863.12 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078659.368122 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160768.04 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 16076.80 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094736.172444 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144691.24 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 14469.12 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109205.296334 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130222.12 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 13022.21 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122227.507835 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117199.90 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 11719.99 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133947.498185 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105479.91 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 10547.99 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144495.489501 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94931.92 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 9493.19 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153988.681685 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85438.73 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 8543.87 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162532.554650 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76894.86 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 7689.49 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170222.040319 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69205.37 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 6920.54 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177142.577422 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62284.83 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 6228.48 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183371.060813 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56056.35 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 5605.64 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188976.695866 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50450.72 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 5045.07 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194021.767414 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45405.64 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 4540.56 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198562.331806 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40865.08 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 4086.51 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202648.839760 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36778.57 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3677.86 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206326.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33100.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209926.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29500.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213526.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25900.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217126.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22300.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220726.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18700.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224326.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15100.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227926.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11500.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231526.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7900.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235126.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4300.71 sec) 11:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238726.696918 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700.71 sec) 11:01:00 [GPUGRID] [rr_sim] 2239427.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984852894.44G/439.78G) 11:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852128.32 11:01:00 [---] pick_jobs_to_run() start 11:01:00 [---] pick_jobs_to_run() end 11:01:00 [---] [rr_sim] end 11:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:01:00 [---] [cpu_sched_debug] preliminary job list: 11:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:01:00 [---] [cpu_sched_debug] final job list: 11:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:01:00 [---] [cpu_sched_debug] enforce_run_list: end 11:01:00 [---] choose_project(): 39660.000000 11:01:00 [iThena.Measurements] doing RPC to get work 11:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395939.09 cpu 2141732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:01:00 [---] Got 0 tasks 11:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:02:00 [---] [rr_sim] doing sim: CPU sched 11:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984825304.08G/439.78G) 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239364.67 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 223936.47 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223936.467446 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015428.21 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 201542.82 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425479.288147 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813885.39 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 181388.54 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606867.826778 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632496.85 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 163249.68 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770117.511546 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469247.16 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 146924.72 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917042.227838 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322322.45 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 132232.24 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049274.472500 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190090.20 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 119009.02 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168283.492696 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071081.18 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 107108.12 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275391.610872 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963973.06 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 96397.31 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371788.917231 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867575.76 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 86757.58 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458546.492953 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780818.18 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 78081.82 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536628.311104 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702736.36 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 70273.64 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606901.947439 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632462.73 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 63246.27 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670148.220141 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569216.45 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 56921.65 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727069.865573 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512294.81 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 51229.48 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778299.346462 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461065.33 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 46106.53 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824405.879261 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414958.80 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 41495.88 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865901.758781 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373462.92 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 37346.29 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903248.050349 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336116.62 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 33611.66 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936859.712760 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302504.96 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 30250.50 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967110.208930 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272254.47 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 27225.45 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994335.655482 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245029.02 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 24502.90 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018838.557380 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220526.12 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 22052.61 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040891.169088 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198473.51 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 19847.35 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060738.519625 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178626.15 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 17862.62 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078601.135108 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160763.54 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 16076.35 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094677.489043 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144687.19 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 14468.72 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109146.207585 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130218.47 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 13021.85 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122168.054272 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117196.62 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 11719.66 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133887.716291 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105476.96 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 10547.70 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144435.412108 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94929.26 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 9492.93 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153928.338343 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85436.34 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 8543.63 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162471.971954 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76892.70 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 7689.27 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170161.242205 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69203.43 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 6920.34 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177081.585430 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62283.09 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 6228.31 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183309.894333 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56054.78 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 5605.48 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188915.372346 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50449.30 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 5044.93 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193960.302557 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45404.37 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 4540.44 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198500.739747 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40863.93 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 4086.39 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202587.133218 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36777.54 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3677.75 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206264.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33099.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209864.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29499.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213464.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25899.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217064.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22299.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220664.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18699.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224264.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15099.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227864.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11499.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231464.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7899.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235064.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4299.79 sec) 11:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238664.887342 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699.79 sec) 11:02:00 [GPUGRID] [rr_sim] 2239364.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984825304.08G/439.78G) 11:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852125.59 11:02:00 [---] pick_jobs_to_run() start 11:02:00 [---] pick_jobs_to_run() end 11:02:00 [---] [rr_sim] end 11:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:02:00 [---] [cpu_sched_debug] preliminary job list: 11:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:02:00 [---] [cpu_sched_debug] final job list: 11:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:02:00 [---] [cpu_sched_debug] enforce_run_list: end 11:02:00 [---] choose_project(): 39720.000000 11:02:00 [iThena.Measurements] doing RPC to get work 11:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395879.09 cpu 2141672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:02:00 [---] Got 0 tasks 11:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:03:00 [---] [rr_sim] doing sim: CPU sched 11:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984797713.73G/439.78G) 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239301.94 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 223930.19 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223930.193758 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015371.74 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 201537.17 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425467.368140 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813834.57 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 181383.46 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606850.825083 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632451.11 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 163245.11 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770095.936333 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469206.00 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 146920.60 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 917016.536457 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322285.40 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 132228.54 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049245.076569 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190056.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 119005.69 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168250.762670 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071051.17 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 107105.12 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275355.880160 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963946.06 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 96394.61 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371750.485902 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867551.45 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 86755.15 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458505.631069 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780796.31 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 78079.63 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536585.261720 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702716.68 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 70271.67 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606856.929306 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632445.01 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 63244.50 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670101.430133 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569200.51 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 56920.05 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1727021.480877 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512280.46 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 51228.05 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778249.526547 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461052.41 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 46105.24 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824354.767650 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414947.17 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 41494.72 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865849.484643 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373452.45 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 37345.25 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903194.729936 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336107.21 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 33610.72 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936805.450700 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302496.49 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 30249.65 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967055.099388 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272246.84 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 27224.68 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994279.783207 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245022.15 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 24502.22 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018781.998644 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220519.94 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 22051.99 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040833.992537 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198467.95 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 19846.79 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060680.787041 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178621.15 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 17862.12 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078542.902094 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160759.04 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 16075.90 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094618.805643 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144683.13 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 14468.31 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109087.118836 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130214.82 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 13021.48 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122108.600710 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117193.34 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 11719.33 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133827.934397 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105474.00 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 10547.40 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144375.334715 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94926.60 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 9492.66 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153867.995001 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85433.94 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 8543.39 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162411.389258 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76890.55 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 7689.05 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170100.444090 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69201.49 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 6920.15 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177020.593439 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62281.34 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 6228.13 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183248.727853 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56053.21 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 5605.32 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188854.048825 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50447.89 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 5044.79 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193898.837700 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45403.10 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 4540.31 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198439.147688 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40862.79 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 4086.28 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202525.426677 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36776.51 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3677.65 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206203.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33098.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209803.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29498.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213403.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25898.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217003.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22298.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220603.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18698.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224203.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15098.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227803.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11498.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231403.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7898.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235003.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4298.86 sec) 11:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238603.077767 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698.86 sec) 11:03:00 [GPUGRID] [rr_sim] 2239301.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984797713.73G/439.78G) 11:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852122.85 11:03:00 [---] pick_jobs_to_run() start 11:03:00 [---] pick_jobs_to_run() end 11:03:00 [---] [rr_sim] end 11:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:03:00 [---] [cpu_sched_debug] preliminary job list: 11:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:03:00 [---] [cpu_sched_debug] final job list: 11:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:03:00 [---] [cpu_sched_debug] enforce_run_list: end 11:03:00 [---] choose_project(): 39780.000000 11:03:00 [iThena.Measurements] doing RPC to get work 11:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395819.09 cpu 2141612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:03:00 [---] Got 0 tasks 11:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:04:00 [---] [rr_sim] doing sim: CPU sched 11:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984770123.37G/439.78G) 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239239.20 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 223923.92 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223923.920069 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015315.28 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 201531.53 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425455.448132 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813783.75 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 181378.38 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606833.823388 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632405.38 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 163240.54 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770074.361119 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469164.84 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 146916.48 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916990.845076 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322248.36 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 132224.84 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049215.680638 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1190023.52 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 119002.35 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168218.032644 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1071021.17 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 107102.12 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275320.149449 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963919.05 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 96391.91 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371712.054573 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867527.15 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 86752.71 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458464.769185 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780774.43 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 78077.44 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536542.212336 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702696.99 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 70269.70 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606811.911172 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632427.29 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 63242.73 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670054.640124 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569184.56 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 56918.46 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726973.096181 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512266.10 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 51226.61 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778199.706633 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461039.49 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 46103.95 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824303.656039 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414935.54 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 41493.55 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865797.210504 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373441.99 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 37344.20 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903141.409523 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336097.79 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 33609.78 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936751.188640 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302488.01 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 30248.80 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966999.989846 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272239.21 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 27223.92 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994223.910931 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245015.29 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 24501.53 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018725.439907 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220513.76 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 22051.38 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040776.815986 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198462.38 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 19846.24 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060623.054457 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178616.15 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 17861.61 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078484.669080 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160754.53 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 16075.45 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094560.122242 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144679.08 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 14467.91 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109028.030087 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130211.17 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 13021.12 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122049.147148 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117190.05 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 11719.01 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133768.152502 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105471.05 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 10547.10 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144315.257322 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94923.94 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 9492.39 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153807.651659 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85431.55 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 8543.15 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162350.806562 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76888.39 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 7688.84 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170039.645976 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69199.55 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 6919.96 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176959.601447 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62279.60 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 6227.96 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183187.561372 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56051.64 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 5605.16 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188792.725304 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50446.48 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 5044.65 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193837.372843 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45401.83 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 4540.18 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198377.555628 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40861.65 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 4086.16 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202463.720135 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36775.48 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3677.55 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206141.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33097.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209741.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29497.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213341.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25897.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216941.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22297.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220541.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18697.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224141.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15097.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227741.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11497.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231341.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7897.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234941.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4297.93 sec) 11:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238541.268191 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697.93 sec) 11:04:00 [GPUGRID] [rr_sim] 2239239.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984770123.37G/439.78G) 11:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852120.11 11:04:00 [---] pick_jobs_to_run() start 11:04:00 [---] pick_jobs_to_run() end 11:04:00 [---] [rr_sim] end 11:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:04:00 [---] [cpu_sched_debug] preliminary job list: 11:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:04:00 [---] [cpu_sched_debug] final job list: 11:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:04:00 [---] [cpu_sched_debug] enforce_run_list: end 11:04:00 [---] choose_project(): 39840.000000 11:04:00 [iThena.Measurements] doing RPC to get work 11:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395759.09 cpu 2141552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:04:00 [---] Got 0 tasks 11:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:05:00 [---] [rr_sim] doing sim: CPU sched 11:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984742533.02G/439.78G) 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239176.46 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 223917.65 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223917.646381 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015258.82 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 201525.88 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425443.528124 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813732.94 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 181373.29 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606816.821693 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632359.64 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 163235.96 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770052.785905 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469123.68 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 146912.37 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916965.153696 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322211.31 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 132221.13 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049186.284707 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189990.18 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 118999.02 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168185.302618 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070991.16 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 107099.12 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275284.418737 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963892.05 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 96389.20 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371673.623245 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867502.84 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 86750.28 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458423.907301 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780752.56 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 78075.26 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536499.162953 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702677.30 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 70267.73 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606766.893038 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632409.57 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 63240.96 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1670007.850116 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569168.61 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 56916.86 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726924.711485 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512251.75 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 51225.18 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778149.886718 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461026.58 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 46102.66 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824252.544428 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414923.92 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 41492.39 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865744.936366 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373431.53 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 37343.15 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903088.089111 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336088.37 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 33608.84 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936696.926581 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302479.54 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 30247.95 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966944.880304 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272231.58 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 27223.16 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994168.038655 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245008.43 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 24500.84 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018668.881170 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220507.58 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 22050.76 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040719.639435 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198456.82 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 19845.68 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060565.321872 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178611.14 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 17861.11 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078426.436066 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160750.03 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 16075.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094501.438841 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144675.02 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 14467.50 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108968.941338 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130207.52 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 13020.75 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121989.693585 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117186.77 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 11718.68 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133708.370608 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105468.09 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 10546.81 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144255.179928 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94921.28 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 9492.13 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153747.308317 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85429.16 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 8542.92 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162290.223866 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76886.24 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 7688.62 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169978.847861 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69197.62 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 6919.76 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176898.609456 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62277.85 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 6227.79 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183126.394892 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56050.07 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 5605.01 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188731.401784 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50445.06 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 5044.51 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193775.907987 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45400.56 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 4540.06 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198315.963569 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40860.50 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 4086.05 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202402.013593 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36774.45 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3677.45 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206079.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33097.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209679.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29497.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213279.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25897.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216879.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22297.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220479.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18697.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224079.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15097.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227679.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11497.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231279.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7897.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234879.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4297.01 sec) 11:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238479.458615 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697.01 sec) 11:05:00 [GPUGRID] [rr_sim] 2239176.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984742533.02G/439.78G) 11:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852117.38 11:05:00 [---] pick_jobs_to_run() start 11:05:00 [---] pick_jobs_to_run() end 11:05:00 [---] [rr_sim] end 11:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:05:00 [---] [cpu_sched_debug] preliminary job list: 11:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:05:00 [---] [cpu_sched_debug] final job list: 11:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:05:00 [---] [cpu_sched_debug] enforce_run_list: end 11:05:00 [---] choose_project(): 39900.000000 11:05:00 [iThena.Measurements] doing RPC to get work 11:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395699.09 cpu 2141492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:05:00 [---] Got 0 tasks 11:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:06:00 [---] [rr_sim] doing sim: CPU sched 11:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984714942.66G/439.78G) 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239113.73 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 223911.37 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223911.372693 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015202.35 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 201520.24 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425431.608117 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813682.12 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 181368.21 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606799.819998 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632313.91 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 163231.39 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770031.210691 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469082.52 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 146908.25 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916939.462315 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322174.26 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 132217.43 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049156.888777 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189956.84 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 118995.68 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168152.572592 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070961.15 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 107096.12 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275248.688026 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963865.04 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 96386.50 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371635.191916 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867478.54 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 86747.85 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458383.045417 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780730.68 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 78073.07 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536456.113569 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702657.61 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 70265.76 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606721.874905 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632391.85 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 63239.19 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669961.060107 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569152.67 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 56915.27 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726876.326790 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512237.40 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 51223.74 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778100.066804 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461013.66 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 46101.37 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824201.432816 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414912.29 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 41491.23 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865692.662228 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373421.06 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 37342.11 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903034.768698 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336078.96 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 33607.90 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936642.664521 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302471.06 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 30247.11 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966889.770762 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272223.96 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 27222.40 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994112.166379 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245001.56 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 24500.16 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018612.322434 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220501.40 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 22050.14 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040662.462883 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198451.26 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 19845.13 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060507.589288 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178606.14 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 17860.61 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078368.203052 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160745.52 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 16074.55 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094442.755440 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144670.97 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 14467.10 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108909.852589 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130203.87 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 13020.39 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121930.240023 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117183.49 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 11718.35 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133648.588714 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105465.14 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 10546.51 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144195.102535 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94918.62 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 9491.86 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153686.964975 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85426.76 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 8542.68 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162229.641170 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76884.09 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 7688.41 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169918.049746 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69195.68 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 6919.57 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176837.617465 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62276.11 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 6227.61 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183065.228411 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56048.50 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 5604.85 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188670.078263 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50443.65 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 5044.36 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193714.443130 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45399.28 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 4539.93 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198254.371510 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40859.36 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 4085.94 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202340.307052 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36773.42 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3677.34 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206017.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33096.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209617.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29496.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213217.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25896.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216817.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22296.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220417.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18696.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224017.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15096.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227617.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11496.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231217.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7896.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234817.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4296.08 sec) 11:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238417.649040 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696.08 sec) 11:06:00 [GPUGRID] [rr_sim] 2239113.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984714942.66G/439.78G) 11:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852114.64 11:06:00 [---] pick_jobs_to_run() start 11:06:00 [---] pick_jobs_to_run() end 11:06:00 [---] [rr_sim] end 11:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:06:00 [---] [cpu_sched_debug] preliminary job list: 11:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:06:00 [---] [cpu_sched_debug] final job list: 11:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:06:00 [---] [cpu_sched_debug] enforce_run_list: end 11:06:00 [---] choose_project(): 39960.000000 11:06:00 [iThena.Measurements] doing RPC to get work 11:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395639.09 cpu 2141432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:06:00 [---] Got 0 tasks 11:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:07:00 [---] [rr_sim] doing sim: CPU sched 11:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984687352.30G/439.78G) 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2239050.99 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 223905.10 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223905.099005 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015145.89 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 201514.59 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425419.688109 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813631.30 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 181363.13 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606782.818303 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632268.17 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 163226.82 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 770009.635477 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469041.35 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 146904.14 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916913.770934 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322137.22 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 132213.72 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049127.492846 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189923.50 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 118992.35 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168119.842566 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070931.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 107093.11 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275212.957314 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963838.03 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 96383.80 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371596.760588 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867454.23 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 86745.42 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458342.183534 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780708.81 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 78070.88 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536413.064185 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702637.93 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 70263.79 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606676.856771 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632374.13 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 63237.41 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669914.270099 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569136.72 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 56913.67 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726827.942094 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512223.05 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 51222.30 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778050.246889 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461000.74 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 46100.07 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824150.321205 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414900.67 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 41490.07 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865640.388089 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373410.60 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 37341.06 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902981.448285 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336069.54 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 33606.95 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936588.402461 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302462.59 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 30246.26 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966834.661220 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272216.33 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 27221.63 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994056.294103 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244994.70 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 24499.47 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018555.763697 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220495.23 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 22049.52 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040605.286332 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198445.70 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 19844.57 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060449.856704 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178601.13 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 17860.11 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078309.970038 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160741.02 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 16074.10 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094384.072039 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144666.92 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 14466.69 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108850.763840 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130200.23 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 13020.02 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121870.786461 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117180.20 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 11718.02 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133588.806820 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105462.18 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 10546.22 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144135.025142 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94915.96 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 9491.60 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153626.621633 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85424.37 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 8542.44 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162169.058474 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76881.93 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 7688.19 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169857.251632 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69193.74 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 6919.37 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176776.625473 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62274.36 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 6227.44 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183004.061931 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56046.93 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 5604.69 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188608.754742 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50442.24 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 5044.22 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193652.978273 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45398.01 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 4539.80 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198192.779450 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40858.21 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 4085.82 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202278.600510 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36772.39 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3677.24 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205955.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33095.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209555.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29495.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213155.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25895.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216755.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22295.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220355.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18695.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223955.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15095.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227555.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11495.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231155.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7895.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234755.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4295.15 sec) 11:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238355.839464 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695.15 sec) 11:07:00 [GPUGRID] [rr_sim] 2239050.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984687352.30G/439.78G) 11:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852111.90 11:07:00 [---] pick_jobs_to_run() start 11:07:00 [---] pick_jobs_to_run() end 11:07:00 [---] [rr_sim] end 11:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:07:00 [---] [cpu_sched_debug] preliminary job list: 11:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:07:00 [---] [cpu_sched_debug] final job list: 11:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:07:00 [---] [cpu_sched_debug] enforce_run_list: end 11:07:00 [---] choose_project(): 40020.000000 11:07:00 [iThena.Measurements] doing RPC to get work 11:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395579.09 cpu 2141372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:07:00 [---] Got 0 tasks 11:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:08:00 [---] [rr_sim] doing sim: CPU sched 11:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984659761.95G/439.78G) 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238988.25 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 223898.83 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223898.825317 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015089.43 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 201508.94 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425407.768101 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813580.49 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 181358.05 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606765.816608 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632222.44 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 163222.24 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769988.060264 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1469000.19 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 146900.02 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916888.079554 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322100.17 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 132210.02 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049098.096915 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189890.16 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 118989.02 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168087.112540 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070901.14 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 107090.11 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275177.226603 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963811.03 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 96381.10 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371558.329259 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867429.92 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 86742.99 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458301.321650 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780686.93 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 78068.69 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536370.014801 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702618.24 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 70261.82 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606631.838638 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632356.41 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 63235.64 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669867.480090 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569120.77 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 56912.08 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726779.557398 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512208.70 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 51220.87 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1778000.426975 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460987.83 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 46098.78 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824099.209594 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414889.04 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 41488.90 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865588.113951 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373400.14 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 37340.01 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902928.127872 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336060.13 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 33606.01 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936534.140402 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302454.11 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 30245.41 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966779.551678 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272208.70 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 27220.87 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994000.421827 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244987.83 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 24498.78 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018499.204961 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220489.05 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 22048.90 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040548.109781 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198440.14 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 19844.01 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060392.124120 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178596.13 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 17859.61 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078251.737024 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160736.52 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 16073.65 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094325.388638 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144662.86 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 14466.29 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108791.675091 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130196.58 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 13019.66 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121811.332899 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117176.92 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 11717.69 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133529.024925 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105459.23 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 10545.92 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144074.947749 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94913.31 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 9491.33 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153566.278291 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85421.97 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 8542.20 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162108.475778 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76879.78 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 7687.98 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169796.453517 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69191.80 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 6919.18 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176715.633482 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62272.62 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 6227.26 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182942.895450 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56045.36 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 5604.54 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188547.431222 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50440.82 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 5044.08 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193591.513416 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45396.74 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 4539.67 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198131.187391 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40857.07 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 4085.71 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202216.893969 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36771.36 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3677.14 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205894.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33094.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209494.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29494.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213094.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25894.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216694.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22294.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220294.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18694.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223894.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15094.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227494.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11494.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231094.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7894.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234694.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4294.22 sec) 11:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238294.029888 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694.22 sec) 11:08:00 [GPUGRID] [rr_sim] 2238988.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984659761.95G/439.78G) 11:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852109.17 11:08:00 [---] pick_jobs_to_run() start 11:08:00 [---] pick_jobs_to_run() end 11:08:00 [---] [rr_sim] end 11:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:08:00 [---] [cpu_sched_debug] preliminary job list: 11:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:08:00 [---] [cpu_sched_debug] final job list: 11:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:08:00 [---] [cpu_sched_debug] enforce_run_list: end 11:08:00 [---] choose_project(): 40080.000000 11:08:00 [iThena.Measurements] doing RPC to get work 11:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395519.09 cpu 2141312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:08:00 [---] Got 0 tasks 11:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:09:00 [---] [rr_sim] doing sim: CPU sched 11:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984632171.59G/439.78G) 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238925.52 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 223892.55 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223892.551628 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2015032.96 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 201503.30 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425395.848094 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813529.67 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 181352.97 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606748.814913 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632176.70 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 163217.67 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769966.485050 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468959.03 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 146895.90 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916862.388173 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322063.13 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 132206.31 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049068.700984 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189856.82 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 118985.68 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168054.382514 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070871.13 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 107087.11 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275141.495891 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963784.02 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 96378.40 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371519.897930 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867405.62 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 86740.56 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458260.459766 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780665.06 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 78066.51 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536326.965417 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702598.55 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 70259.86 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606586.820504 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632338.70 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 63233.87 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669820.690082 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569104.83 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 56910.48 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726731.172702 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512194.34 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 51219.43 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777950.607060 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460974.91 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 46097.49 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1824048.097983 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414877.42 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 41487.74 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865535.839813 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373389.68 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 37338.97 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902874.807460 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336050.71 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 33605.07 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936479.878342 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302445.64 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 30244.56 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966724.442136 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272201.07 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 27220.11 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993944.549551 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244980.97 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 24498.10 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018442.646224 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220482.87 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 22048.29 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040490.933230 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198434.58 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 19843.46 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060334.391535 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178591.12 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 17859.11 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078193.504010 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160732.01 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 16073.20 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094266.705238 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144658.81 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 14465.88 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108732.586342 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130192.93 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 13019.29 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121751.879336 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117173.64 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 11717.36 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133469.243031 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105456.27 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 10545.63 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144014.870356 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94910.65 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 9491.06 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153505.934949 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85419.58 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 8541.96 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162047.893082 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76877.62 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 7687.76 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169735.655402 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69189.86 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 6918.99 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176654.641491 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62270.87 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 6227.09 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182881.728970 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56043.79 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 5604.38 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188486.107701 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50439.41 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 5043.94 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193530.048559 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45395.47 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 4539.55 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198069.595332 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40855.92 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 4085.59 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202155.187427 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36770.33 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3677.03 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205832.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33093.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209432.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29493.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213032.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25893.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216632.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22293.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220232.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18693.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223832.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15093.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227432.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11493.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231032.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7893.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234632.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4293.30 sec) 11:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238232.220313 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693.30 sec) 11:09:00 [GPUGRID] [rr_sim] 2238925.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984632171.59G/439.78G) 11:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852106.43 11:09:00 [---] pick_jobs_to_run() start 11:09:00 [---] pick_jobs_to_run() end 11:09:00 [---] [rr_sim] end 11:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:09:00 [---] [cpu_sched_debug] preliminary job list: 11:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:09:00 [---] [cpu_sched_debug] final job list: 11:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:09:00 [---] [cpu_sched_debug] enforce_run_list: end 11:09:00 [---] choose_project(): 40140.000000 11:09:00 [iThena.Measurements] doing RPC to get work 11:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395459.09 cpu 2141252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:09:00 [---] Got 0 tasks 11:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:10:00 [---] [rr_sim] doing sim: CPU sched 11:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984604581.23G/439.78G) 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238862.78 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 223886.28 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223886.277940 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014976.50 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 201497.65 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425383.928086 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813478.85 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 181347.89 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606731.813218 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632130.97 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 163213.10 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769944.909836 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468917.87 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 146891.79 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916836.696793 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1322026.08 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 132202.61 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049039.305053 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189823.47 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 118982.35 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1168021.652488 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070841.13 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 107084.11 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275105.765180 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963757.01 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 96375.70 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371481.466602 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867381.31 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 86738.13 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458219.597882 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780643.18 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 78064.32 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536283.916034 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702578.86 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 70257.89 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606541.802370 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632320.98 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 63232.10 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669773.900073 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569088.88 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 56908.89 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726682.788006 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512179.99 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 51218.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777900.787146 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460961.99 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 46096.20 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823996.986371 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414865.79 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 41486.58 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865483.565674 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373379.21 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 37337.92 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902821.487047 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336041.29 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 33604.13 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936425.616282 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302437.16 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 30243.72 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966669.332594 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272193.45 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 27219.34 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993888.677275 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244974.10 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 24497.41 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018386.087488 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220476.69 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 22047.67 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040433.756679 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198429.02 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 19842.90 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060276.658951 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178586.12 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 17858.61 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078135.270996 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160727.51 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 16072.75 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094208.021837 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144654.76 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 14465.48 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108673.497593 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130189.28 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 13018.93 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121692.425774 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117170.35 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 11717.04 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133409.461137 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105453.32 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 10545.33 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143954.792963 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94907.99 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 9490.80 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153445.591607 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85417.19 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 8541.72 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161987.310386 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76875.47 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 7687.55 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169674.857288 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69187.92 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 6918.79 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176593.649499 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62269.13 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 6226.91 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182820.562489 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56042.22 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 5604.22 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188424.784181 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50438.00 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 5043.80 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193468.583703 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45394.20 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 4539.42 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198008.003273 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40854.78 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 4085.48 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202093.480885 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36769.30 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3676.93 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205770.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33092.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209370.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29492.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212970.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25892.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216570.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22292.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220170.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18692.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223770.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15092.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227370.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11492.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230970.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7892.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234570.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4292.37 sec) 11:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238170.410737 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692.37 sec) 11:10:00 [GPUGRID] [rr_sim] 2238862.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984604581.23G/439.78G) 11:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852103.69 11:10:00 [---] pick_jobs_to_run() start 11:10:00 [---] pick_jobs_to_run() end 11:10:00 [---] [rr_sim] end 11:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:10:00 [---] [cpu_sched_debug] preliminary job list: 11:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:10:00 [---] [cpu_sched_debug] final job list: 11:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:10:00 [---] [cpu_sched_debug] enforce_run_list: end 11:10:00 [---] choose_project(): 40200.000000 11:10:00 [iThena.Measurements] doing RPC to get work 11:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395399.09 cpu 2141192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:10:00 [---] Got 0 tasks 11:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:11:00 [---] [rr_sim] doing sim: CPU sched 11:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984576990.88G/439.78G) 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238800.04 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 223880.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223880.004252 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014920.04 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 201492.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425372.008079 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813428.03 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 181342.80 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606714.811523 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632085.23 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 163208.52 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769923.334622 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468876.71 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 146887.67 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916811.005412 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321989.04 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 132198.90 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1049009.909123 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189790.13 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 118979.01 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167988.922462 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070811.12 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 107081.11 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275070.034468 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963730.01 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 96373.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371443.035273 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867357.01 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 86735.70 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458178.735998 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780621.31 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 78062.13 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536240.866650 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702559.18 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 70255.92 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606496.784237 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632303.26 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 63230.33 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669727.110065 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569072.93 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 56907.29 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726634.403310 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512165.64 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 51216.56 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777850.967231 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460949.08 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 46094.91 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823945.874760 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414854.17 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 41485.42 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865431.291536 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373368.75 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 37336.88 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902768.166634 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336031.88 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 33603.19 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936371.354223 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302428.69 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 30242.87 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966614.223052 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272185.82 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 27218.58 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993832.804999 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244967.24 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 24496.72 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018329.528751 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220470.51 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 22047.05 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040376.580128 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198423.46 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 19842.35 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060218.926367 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178581.12 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 17858.11 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078077.037982 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160723.00 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 16072.30 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094149.338436 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144650.70 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 14465.07 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108614.408844 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130185.63 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 13018.56 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121632.972212 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117167.07 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 11716.71 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133349.679242 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105450.36 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 10545.04 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143894.715570 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94905.33 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 9490.53 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153385.248265 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85414.79 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 8541.48 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161926.727690 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76873.31 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 7687.33 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169614.059173 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69185.98 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 6918.60 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176532.657508 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62267.39 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 6226.74 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182759.396009 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56040.65 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 5604.06 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188363.460660 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50436.58 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 5043.66 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193407.118846 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45392.92 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 4539.29 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197946.411213 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40853.63 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 4085.36 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202031.774344 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36768.27 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3676.83 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205708.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33091.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209308.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29491.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212908.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25891.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216508.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22291.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220108.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18691.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223708.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15091.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227308.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11491.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230908.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7891.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234508.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4291.44 sec) 11:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238108.601161 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691.44 sec) 11:11:00 [GPUGRID] [rr_sim] 2238800.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984576990.88G/439.78G) 11:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852100.96 11:11:00 [---] pick_jobs_to_run() start 11:11:00 [---] pick_jobs_to_run() end 11:11:00 [---] [rr_sim] end 11:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:11:00 [---] [cpu_sched_debug] preliminary job list: 11:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:11:00 [---] [cpu_sched_debug] final job list: 11:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:11:00 [---] [cpu_sched_debug] enforce_run_list: end 11:11:00 [---] choose_project(): 40260.000000 11:11:00 [iThena.Measurements] doing RPC to get work 11:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395339.09 cpu 2141132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:11:00 [---] Got 0 tasks 11:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:12:00 [---] [rr_sim] doing sim: CPU sched 11:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984549400.52G/439.78G) 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238737.31 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 223873.73 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223873.730564 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014863.58 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 201486.36 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425360.088071 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813377.22 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 181337.72 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606697.809828 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1632039.50 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 163203.95 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769901.759409 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468835.55 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 146883.55 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916785.314031 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321951.99 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 132195.20 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048980.513192 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189756.79 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 118975.68 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167956.192436 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070781.11 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 107078.11 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1275034.303756 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963703.00 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 96370.30 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371404.603945 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867332.70 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 86733.27 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458137.874114 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780599.43 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 78059.94 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536197.817266 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702539.49 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 70253.95 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606451.766103 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632285.54 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 63228.55 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669680.320057 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569056.99 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 56905.70 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726586.018615 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512151.29 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 51215.13 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777801.147317 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460936.16 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 46093.62 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823894.763149 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414842.54 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 41484.25 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865379.017398 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373358.29 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 37335.83 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902714.846222 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336022.46 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 33602.25 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936317.092163 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302420.21 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 30242.02 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966559.113510 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272178.19 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 27217.82 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993776.932723 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244960.37 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 24496.04 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018272.970015 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220464.34 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 22046.43 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040319.403577 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198417.90 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 19841.79 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060161.193783 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178576.11 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 17857.61 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078018.804968 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160718.50 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 16071.85 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094090.655035 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144646.65 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 14464.67 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108555.320095 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130181.99 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 13018.20 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121573.518649 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117163.79 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 11716.38 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133289.897348 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105447.41 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 10544.74 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143834.638177 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94902.67 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 9490.27 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153324.904923 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85412.40 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 8541.24 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161866.144994 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76871.16 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 7687.12 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169553.261059 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69184.04 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 6918.40 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176471.665516 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62265.64 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 6226.56 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182698.229529 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56039.08 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 5603.91 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188302.137139 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50435.17 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 5043.52 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193345.653989 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45391.65 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 4539.17 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197884.819154 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40852.49 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 4085.25 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201970.067802 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36767.24 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3676.72 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205646.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33090.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209246.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29490.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212846.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25890.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216446.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22290.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220046.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18690.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223646.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15090.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227246.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11490.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230846.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7890.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234446.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4290.51 sec) 11:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2238046.791586 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690.51 sec) 11:12:00 [GPUGRID] [rr_sim] 2238737.31: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984549400.52G/439.78G) 11:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852098.22 11:12:00 [---] pick_jobs_to_run() start 11:12:00 [---] pick_jobs_to_run() end 11:12:00 [---] [rr_sim] end 11:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:12:00 [---] [cpu_sched_debug] preliminary job list: 11:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:12:00 [---] [cpu_sched_debug] final job list: 11:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:12:00 [---] [cpu_sched_debug] enforce_run_list: end 11:12:00 [---] choose_project(): 40320.000000 11:12:00 [iThena.Measurements] doing RPC to get work 11:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395279.09 cpu 2141072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:12:00 [---] Got 0 tasks 11:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:13:00 [---] [rr_sim] doing sim: CPU sched 11:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984521810.17G/439.78G) 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238674.57 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 223867.46 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223867.456875 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014807.11 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 201480.71 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425348.168063 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813326.40 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 181332.64 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606680.808133 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631993.76 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 163199.38 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769880.184195 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468794.38 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 146879.44 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916759.622651 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321914.95 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 132191.49 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048951.117261 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189723.45 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 118972.35 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167923.462411 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070751.11 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 107075.11 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274998.573045 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963676.00 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 96367.60 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371366.172616 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867308.40 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 86730.84 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458097.012230 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780577.56 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 78057.76 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536154.767882 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702519.80 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 70251.98 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606406.747970 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632267.82 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 63226.78 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669633.530048 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569041.04 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 56904.10 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726537.633919 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512136.93 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 51213.69 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777751.327402 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460923.24 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 46092.32 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823843.651538 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414830.92 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 41483.09 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865326.743259 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373347.83 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 37334.78 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902661.525809 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336013.04 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 33601.30 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936262.830103 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302411.74 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 30241.17 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966504.003969 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272170.56 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 27217.06 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993721.060447 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244953.51 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 24495.35 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018216.411278 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220458.16 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 22045.82 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040262.227026 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198412.34 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 19841.23 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060103.461199 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178571.11 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 17857.11 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077960.571954 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160714.00 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 16071.40 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094031.971634 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144642.60 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 14464.26 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108496.231346 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130178.34 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 13017.83 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121514.065087 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117160.50 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 11716.05 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133230.115454 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105444.45 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 10544.45 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143774.560784 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94900.01 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 9490.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153264.561581 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85410.01 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 8541.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161805.562298 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76869.01 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 7686.90 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169492.462944 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69182.11 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 6918.21 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176410.673525 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62263.90 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 6226.39 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182637.063048 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56037.51 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 5603.75 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188240.813619 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50433.76 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 5043.38 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193284.189132 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45390.38 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 4539.04 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197823.227095 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40851.34 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 4085.13 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201908.361261 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36766.21 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3676.62 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205584.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33089.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209184.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29489.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212784.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25889.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216384.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22289.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219984.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18689.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223584.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15089.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227184.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11489.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230784.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7889.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234384.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4289.59 sec) 11:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237984.982010 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689.59 sec) 11:13:00 [GPUGRID] [rr_sim] 2238674.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984521810.17G/439.78G) 11:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852095.48 11:13:00 [---] pick_jobs_to_run() start 11:13:00 [---] pick_jobs_to_run() end 11:13:00 [---] [rr_sim] end 11:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:13:00 [---] [cpu_sched_debug] preliminary job list: 11:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:13:00 [---] [cpu_sched_debug] final job list: 11:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:13:00 [---] [cpu_sched_debug] enforce_run_list: end 11:13:00 [---] choose_project(): 40380.000000 11:13:00 [iThena.Measurements] doing RPC to get work 11:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 568019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395219.09 cpu 2141012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2141012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2141012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:13:00 [---] Got 0 tasks 11:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:14:00 [---] [rr_sim] doing sim: CPU sched 11:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984494219.81G/439.78G) 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238611.83 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 223861.18 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223861.183187 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014750.65 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 201475.06 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425336.248056 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813275.58 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 181327.56 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606663.806437 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631948.03 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 163194.80 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769858.608981 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468753.22 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 146875.32 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916733.931270 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321877.90 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 132187.79 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048921.721330 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189690.11 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 118969.01 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167890.732385 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070721.10 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 107072.11 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274962.842333 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963648.99 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 96364.90 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371327.741287 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867284.09 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 86728.41 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458056.150346 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780555.68 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 78055.57 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536111.718499 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702500.11 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 70250.01 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606361.729836 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632250.10 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 63225.01 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669586.740040 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569025.09 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 56902.51 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726489.249223 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512122.58 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 51212.26 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777701.507488 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460910.32 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 46091.03 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823792.539926 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414819.29 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 41481.93 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865274.469121 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373337.36 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 37333.74 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902608.205396 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336003.63 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 33600.36 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936208.568044 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302403.26 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 30240.33 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966448.894427 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272162.94 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 27216.29 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993665.188171 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244946.64 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 24494.66 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018159.852541 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220451.98 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 22045.20 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040205.050475 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198406.78 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 19840.68 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060045.728614 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178566.10 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 17856.61 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077902.338940 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160709.49 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 16070.95 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093973.288233 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144638.54 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 14463.85 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108437.142597 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130174.69 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 13017.47 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121454.611525 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117157.22 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 11715.72 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133170.333560 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105441.50 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 10544.15 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143714.483391 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94897.35 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 9489.73 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153204.218239 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85407.61 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 8540.76 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161744.979602 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76866.85 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 7686.69 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169431.664829 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69180.17 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 6918.02 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176349.681534 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62262.15 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 6226.22 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182575.896568 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56035.94 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 5603.59 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188179.490098 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50432.34 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 5043.23 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193222.724276 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45389.11 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 4538.91 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197761.635035 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40850.20 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 4085.02 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201846.654719 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36765.18 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3676.52 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205523.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33088.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209123.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29488.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212723.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25888.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216323.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22288.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219923.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18688.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223523.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15088.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227123.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11488.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230723.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7888.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234323.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4288.66 sec) 11:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237923.172434 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688.66 sec) 11:14:00 [GPUGRID] [rr_sim] 2238611.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984494219.81G/439.78G) 11:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852092.74 11:14:00 [---] pick_jobs_to_run() start 11:14:00 [---] pick_jobs_to_run() end 11:14:00 [---] [rr_sim] end 11:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:14:00 [---] [cpu_sched_debug] preliminary job list: 11:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:14:00 [---] [cpu_sched_debug] final job list: 11:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:14:00 [---] [cpu_sched_debug] enforce_run_list: end 11:14:00 [---] choose_project(): 40440.000000 11:14:00 [iThena.Measurements] doing RPC to get work 11:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395159.09 cpu 2140952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:14:00 [---] Got 0 tasks 11:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:15:00 [---] [rr_sim] doing sim: CPU sched 11:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984466629.45G/439.78G) 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238549.09 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 223854.91 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223854.909499 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014694.19 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 201469.42 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425324.328048 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813224.77 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 181322.48 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606646.804742 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631902.29 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 163190.23 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769837.033767 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468712.06 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 146871.21 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916708.239890 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321840.86 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 132184.09 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048892.325400 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189656.77 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 118965.68 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167858.002359 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070691.09 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 107069.11 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274927.111622 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963621.98 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 96362.20 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371289.309959 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867259.79 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 86725.98 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1458015.288462 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780533.81 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 78053.38 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536068.669115 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702480.43 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 70248.04 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606316.711702 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632232.38 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 63223.24 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669539.950031 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569009.14 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 56900.91 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726440.864527 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512108.23 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 51210.82 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777651.687573 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460897.41 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 46089.74 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823741.428315 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414807.67 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 41480.77 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865222.194983 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373326.90 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 37332.69 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902554.884983 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335994.21 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 33599.42 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936154.305984 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302394.79 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 30239.48 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966393.784885 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272155.31 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 27215.53 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993609.315895 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244939.78 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 24493.98 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018103.293805 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220445.80 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 22044.58 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040147.873923 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198401.22 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 19840.12 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059987.996030 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178561.10 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 17856.11 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077844.105926 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160704.99 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 16070.50 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093914.604833 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144634.49 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 14463.45 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108378.053848 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130171.04 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 13017.10 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121395.157963 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117153.94 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 11715.39 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133110.551665 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105438.54 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 10543.85 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143654.405998 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94894.69 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 9489.47 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153143.874897 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85405.22 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 8540.52 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161684.396906 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76864.70 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 7686.47 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169370.866715 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69178.23 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 6917.82 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176288.689542 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62260.41 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 6226.04 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182514.730087 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56034.36 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 5603.44 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188118.166578 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50430.93 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 5043.09 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193161.259419 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45387.84 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 4538.78 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197700.042976 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40849.05 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 4084.91 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201784.948177 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36764.15 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3676.41 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205461.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33087.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209061.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29487.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212661.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25887.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216261.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22287.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219861.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18687.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223461.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15087.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227061.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11487.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230661.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7887.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234261.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4287.73 sec) 11:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237861.362859 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687.73 sec) 11:15:00 [GPUGRID] [rr_sim] 2238549.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984466629.45G/439.78G) 11:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852090.01 11:15:00 [---] pick_jobs_to_run() start 11:15:00 [---] pick_jobs_to_run() end 11:15:00 [---] [rr_sim] end 11:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:15:00 [---] [cpu_sched_debug] preliminary job list: 11:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:15:00 [---] [cpu_sched_debug] final job list: 11:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:15:00 [---] [cpu_sched_debug] enforce_run_list: end 11:15:00 [---] choose_project(): 40500.000000 11:15:00 [iThena.Measurements] doing RPC to get work 11:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395099.09 cpu 2140892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:15:00 [---] Got 0 tasks 11:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:16:00 [---] [rr_sim] doing sim: CPU sched 11:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984439039.10G/439.78G) 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238486.36 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 223848.64 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223848.635811 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014637.72 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 201463.77 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425312.408041 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813173.95 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 181317.40 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606629.803047 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631856.56 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 163185.66 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769815.458553 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468670.90 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 146867.09 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916682.548509 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321803.81 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 132180.38 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048862.929469 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189623.43 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 118962.34 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167825.272333 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070661.09 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 107066.11 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274891.380910 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963594.98 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 96359.50 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371250.878630 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867235.48 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 86723.55 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457974.426578 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780511.93 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 78051.19 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1536025.619731 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702460.74 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 70246.07 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606271.693569 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632214.66 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 63221.47 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669493.160023 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568993.20 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 56899.32 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726392.479831 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512093.88 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 51209.39 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777601.867659 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460884.49 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 46088.45 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823690.316704 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414796.04 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 41479.60 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865169.920844 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373316.44 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 37331.64 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902501.564571 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335984.79 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 33598.48 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936100.043924 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302386.31 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 30238.63 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966338.675343 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272147.68 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 27214.77 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993553.443619 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244932.91 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 24493.29 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018046.735068 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220439.62 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 22043.96 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040090.697372 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198395.66 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 19839.57 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059930.263446 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178556.09 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 17855.61 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077785.872912 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160700.49 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 16070.05 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093855.921432 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144630.44 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 14463.04 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108318.965099 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130167.39 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 13016.74 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121335.704400 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117150.65 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 11715.07 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133050.769771 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105435.59 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 10543.56 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143594.328605 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94892.03 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 9489.20 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153083.531555 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85402.83 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 8540.28 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161623.814210 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76862.54 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 7686.25 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169310.068600 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69176.29 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 6917.63 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176227.697551 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62258.66 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 6225.87 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182453.563607 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56032.79 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 5603.28 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188056.843057 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50429.52 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 5042.95 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193099.794562 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45386.56 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 4538.66 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197638.450917 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40847.91 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 4084.79 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201723.241636 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36763.12 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3676.31 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205399.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33086.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208999.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29486.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212599.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25886.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216199.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22286.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219799.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18686.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223399.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15086.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226999.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11486.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230599.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7886.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234199.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4286.80 sec) 11:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237799.553283 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 686.80 sec) 11:16:00 [GPUGRID] [rr_sim] 2238486.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984439039.10G/439.78G) 11:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852087.27 11:16:00 [---] pick_jobs_to_run() start 11:16:00 [---] pick_jobs_to_run() end 11:16:00 [---] [rr_sim] end 11:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:16:00 [---] [cpu_sched_debug] preliminary job list: 11:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:16:00 [---] [cpu_sched_debug] final job list: 11:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:16:00 [---] [cpu_sched_debug] enforce_run_list: end 11:16:00 [---] choose_project(): 40560.000000 11:16:00 [iThena.Measurements] doing RPC to get work 11:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 395039.09 cpu 2140832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:16:00 [---] Got 0 tasks 11:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:17:00 [---] [rr_sim] doing sim: CPU sched 11:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984411448.74G/439.78G) 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238423.62 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 223842.36 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223842.362123 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014581.26 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 201458.13 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425300.488033 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813123.13 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 181312.31 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606612.801352 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631810.82 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 163181.08 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769793.883340 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468629.74 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 146862.97 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916656.857128 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321766.76 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 132176.68 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048833.533538 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189590.09 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 118959.01 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167792.542307 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070631.08 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 107063.11 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274855.650199 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963567.97 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 96356.80 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371212.447302 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867211.17 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 86721.12 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457933.564694 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780490.06 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 78049.01 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535982.570347 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702441.05 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 70244.11 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606226.675435 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632196.95 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 63219.69 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669446.370014 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568977.25 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 56897.73 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726344.095135 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512079.53 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 51207.95 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777552.047744 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460871.57 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 46087.16 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823639.205093 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414784.42 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 41478.44 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865117.646706 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373305.97 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 37330.60 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902448.244158 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335975.38 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 33597.54 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936045.781865 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302377.84 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 30237.78 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966283.565801 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272140.06 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 27214.01 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993497.571343 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244926.05 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 24492.60 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017990.176332 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220433.44 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 22043.34 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040033.520821 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198390.10 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 19839.01 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059872.530862 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178551.09 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 17855.11 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077727.639898 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160695.98 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 16069.60 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093797.238031 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144626.38 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 14462.64 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108259.876350 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130163.74 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 13016.37 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121276.250838 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117147.37 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 11714.74 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132990.987877 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105432.63 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 10543.26 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143534.251212 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94889.37 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 9488.94 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153023.188213 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85400.43 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 8540.04 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161563.231514 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76860.39 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 7686.04 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169249.270486 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69174.35 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 6917.44 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176166.705560 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62256.92 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 6225.69 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182392.397126 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56031.22 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 5603.12 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187995.519536 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50428.10 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 5042.81 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193038.329705 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45385.29 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 4538.53 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197576.858857 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40846.76 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 4084.68 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201661.535094 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36762.09 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3676.21 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205337.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33085.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208937.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29485.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212537.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25885.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216137.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22285.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219737.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18685.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223337.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15085.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226937.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11485.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230537.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7885.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234137.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4285.88 sec) 11:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237737.743707 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 685.88 sec) 11:17:00 [GPUGRID] [rr_sim] 2238423.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984411448.74G/439.78G) 11:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852084.53 11:17:00 [---] pick_jobs_to_run() start 11:17:00 [---] pick_jobs_to_run() end 11:17:00 [---] [rr_sim] end 11:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:17:00 [---] [cpu_sched_debug] preliminary job list: 11:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:17:00 [---] [cpu_sched_debug] final job list: 11:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:17:00 [---] [cpu_sched_debug] enforce_run_list: end 11:17:00 [---] choose_project(): 40620.000000 11:17:00 [iThena.Measurements] doing RPC to get work 11:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394979.09 cpu 2140772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:17:00 [---] Got 0 tasks 11:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:18:00 [---] [rr_sim] doing sim: CPU sched 11:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984383858.38G/439.78G) 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238360.88 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 223836.09 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223836.088434 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014524.80 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 201452.48 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425288.568025 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813072.32 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 181307.23 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606595.799657 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631765.08 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 163176.51 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769772.308126 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468588.58 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 146858.86 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916631.165748 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321729.72 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 132172.97 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048804.137607 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189556.75 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 118955.67 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167759.812281 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070601.07 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 107060.11 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274819.919487 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963540.96 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 96354.10 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371174.015973 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867186.87 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 86718.69 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457892.702810 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780468.18 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 78046.82 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535939.520963 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702421.36 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 70242.14 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606181.657301 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632179.23 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 63217.92 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669399.580006 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568961.30 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 56896.13 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726295.710440 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512065.17 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 51206.52 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777502.227830 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460858.66 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 46085.87 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823588.093481 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414772.79 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 41477.28 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865065.372568 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373295.51 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 37329.55 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902394.923745 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335965.96 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 33596.60 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935991.519805 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302369.36 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 30236.94 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966228.456259 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272132.43 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 27213.24 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993441.699068 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244919.19 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 24491.92 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017933.617595 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220427.27 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 22042.73 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039976.344270 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198384.54 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 19838.45 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059814.798277 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178546.09 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 17854.61 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077669.406884 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160691.48 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 16069.15 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093738.554630 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144622.33 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 14462.23 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108200.787601 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130160.10 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 13016.01 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121216.797276 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117144.09 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 11714.41 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132931.205983 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105429.68 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 10542.97 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143474.173819 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94886.71 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 9488.67 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152962.844871 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85398.04 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 8539.80 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161502.648818 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76858.24 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 7685.82 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169188.472371 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69172.41 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 6917.24 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176105.713568 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62255.17 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 6225.52 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182331.230646 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56029.65 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 5602.97 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187934.196016 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50426.69 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 5042.67 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192976.864849 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45384.02 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 4538.40 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197515.266798 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40845.62 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 4084.56 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201599.828553 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36761.06 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3676.11 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205275.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33084.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208875.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29484.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212475.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25884.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216075.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22284.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219675.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18684.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223275.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15084.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226875.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11484.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230475.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7884.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234075.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4284.95 sec) 11:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237675.934132 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 684.95 sec) 11:18:00 [GPUGRID] [rr_sim] 2238360.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984383858.38G/439.78G) 11:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852081.80 11:18:00 [---] pick_jobs_to_run() start 11:18:00 [---] pick_jobs_to_run() end 11:18:00 [---] [rr_sim] end 11:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:18:00 [---] [cpu_sched_debug] preliminary job list: 11:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:18:00 [---] [cpu_sched_debug] final job list: 11:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:18:00 [---] [cpu_sched_debug] enforce_run_list: end 11:18:00 [---] choose_project(): 40680.000000 11:18:00 [iThena.Measurements] doing RPC to get work 11:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394919.09 cpu 2140712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:18:00 [---] Got 0 tasks 11:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:19:00 [---] [rr_sim] doing sim: CPU sched 11:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984356268.03G/439.78G) 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238298.15 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 223829.81 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223829.814746 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014468.33 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 201446.83 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425276.648018 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1813021.50 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 181302.15 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606578.797962 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631719.35 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 163171.93 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769750.732912 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468547.41 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 146854.74 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916605.474367 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321692.67 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 132169.27 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048774.741677 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189523.41 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 118952.34 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167727.082255 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070571.07 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 107057.11 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274784.188776 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963513.96 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 96351.40 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371135.584644 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867162.56 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 86716.26 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457851.840926 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780446.31 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 78044.63 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535896.471580 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702401.68 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 70240.17 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606136.639168 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632161.51 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 63216.15 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669352.789997 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568945.36 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 56894.54 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726247.325744 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512050.82 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 51205.08 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777452.407916 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460845.74 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 46084.57 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823536.981870 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414761.17 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 41476.12 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865013.098429 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373285.05 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 37328.50 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902341.603333 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335956.54 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 33595.65 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935937.257745 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302360.89 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 30236.09 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966173.346717 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272124.80 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 27212.48 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993385.826792 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244912.32 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 24491.23 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017877.058859 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220421.09 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 22042.11 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039919.167719 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198378.98 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 19837.90 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059757.065693 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178541.08 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 17854.11 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077611.173870 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160686.97 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 16068.70 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093679.871229 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144618.28 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 14461.83 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108141.698852 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130156.45 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 13015.64 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121157.343713 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117140.80 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 11714.08 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132871.424088 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105426.72 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 10542.67 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143414.096426 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94884.05 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 9488.41 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152902.501529 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85395.65 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 8539.56 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161442.066122 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76856.08 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 7685.61 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169127.674256 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69170.47 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 6917.05 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176044.721577 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62253.43 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 6225.34 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182270.064165 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56028.08 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 5602.81 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187872.872495 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50425.27 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 5042.53 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192915.399992 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45382.75 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 4538.27 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197453.674739 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40844.47 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 4084.45 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201538.122011 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36760.03 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3676.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205214.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33084.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208814.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29484.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212414.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25884.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216014.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22284.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219614.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18684.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223214.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15084.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226814.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11484.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230414.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7884.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234014.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4284.02 sec) 11:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237614.124556 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 684.02 sec) 11:19:00 [GPUGRID] [rr_sim] 2238298.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984356268.03G/439.78G) 11:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852079.06 11:19:00 [---] pick_jobs_to_run() start 11:19:00 [---] pick_jobs_to_run() end 11:19:00 [---] [rr_sim] end 11:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:19:00 [---] [cpu_sched_debug] preliminary job list: 11:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:19:00 [---] [cpu_sched_debug] final job list: 11:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:19:00 [---] [cpu_sched_debug] enforce_run_list: end 11:19:00 [---] choose_project(): 40740.000000 11:19:00 [iThena.Measurements] doing RPC to get work 11:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394859.09 cpu 2140652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:19:00 [---] Got 0 tasks 11:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:20:00 [---] [rr_sim] doing sim: CPU sched 11:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984328677.67G/439.78G) 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238235.41 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 223823.54 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223823.541058 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014411.87 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 201441.19 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425264.728010 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812970.68 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 181297.07 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606561.796267 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631673.61 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 163167.36 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769729.157698 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468506.25 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 146850.63 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916579.782986 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321655.63 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 132165.56 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048745.345746 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189490.06 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 118949.01 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167694.352229 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070541.06 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 107054.11 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274748.458064 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963486.95 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 96348.70 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371097.153316 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867138.26 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 86713.83 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457810.979042 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780424.43 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 78042.44 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535853.422196 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702381.99 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 70238.20 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606091.621034 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632143.79 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 63214.38 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669305.999989 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568929.41 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 56892.94 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726198.941048 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512036.47 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 51203.65 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777402.588001 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460832.82 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 46083.28 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823485.870259 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414749.54 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 41474.95 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864960.824291 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373274.59 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 37327.46 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902288.282920 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335947.13 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 33594.71 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935882.995686 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302352.41 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 30235.24 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966118.237175 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272117.17 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 27211.72 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993329.954516 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244905.46 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 24490.55 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017820.500122 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220414.91 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 22041.49 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039861.991168 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198373.42 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 19837.34 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059699.333109 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178536.08 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 17853.61 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077552.940856 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160682.47 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 16068.25 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093621.187828 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144614.22 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 14461.42 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108082.610104 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130152.80 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 13015.28 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121097.890151 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117137.52 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 11713.75 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132811.642194 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105423.77 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 10542.38 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143354.019033 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94881.39 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 9488.14 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152842.158187 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85393.25 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 8539.33 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161381.483426 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76853.93 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 7685.39 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169066.876142 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69168.53 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 6916.85 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175983.729586 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62251.68 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 6225.17 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182208.897685 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56026.51 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 5602.65 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187811.548974 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50423.86 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 5042.39 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192853.935135 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45381.48 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 4538.15 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197392.082679 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40843.33 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 4084.33 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201476.415469 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36759.00 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3675.90 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205152.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33083.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208752.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29483.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212352.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25883.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215952.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22283.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219552.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18683.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223152.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15083.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226752.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11483.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230352.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7883.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233952.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4283.10 sec) 11:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237552.314980 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 683.10 sec) 11:20:00 [GPUGRID] [rr_sim] 2238235.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984328677.67G/439.78G) 11:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852076.32 11:20:00 [---] pick_jobs_to_run() start 11:20:00 [---] pick_jobs_to_run() end 11:20:00 [---] [rr_sim] end 11:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:20:00 [---] [cpu_sched_debug] preliminary job list: 11:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:20:00 [---] [cpu_sched_debug] final job list: 11:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:20:00 [---] [cpu_sched_debug] enforce_run_list: end 11:20:00 [---] choose_project(): 40800.000000 11:20:00 [iThena.Measurements] doing RPC to get work 11:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394799.09 cpu 2140592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:20:00 [---] Got 0 tasks 11:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:21:00 [---] [rr_sim] doing sim: CPU sched 11:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984301087.32G/439.78G) 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238172.67 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 223817.27 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223817.267370 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014355.41 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 201435.54 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425252.808003 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812919.87 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 181291.99 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606544.794572 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631627.88 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 163162.79 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769707.582485 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468465.09 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 146846.51 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916554.091606 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321618.58 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 132161.86 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048715.949815 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189456.72 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 118945.67 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167661.622203 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070511.05 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 107051.11 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274712.727353 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963459.95 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 96345.99 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371058.721987 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867113.95 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 86711.40 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457770.117158 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780402.56 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 78040.26 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535810.372812 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702362.30 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 70236.23 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606046.602901 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632126.07 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 63212.61 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669259.209980 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568913.46 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 56891.35 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726150.556352 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512022.12 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 51202.21 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777352.768087 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460819.91 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 46081.99 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823434.758648 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414737.92 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 41473.79 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864908.550153 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373264.12 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 37326.41 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902234.962507 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335937.71 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 33593.77 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935828.733626 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302343.94 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 30234.39 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966063.127633 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272109.55 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 27210.95 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993274.082240 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244898.59 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 24489.86 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017763.941385 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220408.73 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 22040.87 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039804.814617 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198367.86 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 19836.79 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059641.600525 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178531.07 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 17853.11 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077494.707842 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160677.97 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 16067.80 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093562.504428 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144610.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 14461.02 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108023.521355 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130149.15 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 13014.92 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121038.436589 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117134.24 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 11713.42 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132751.860300 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105420.81 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 10542.08 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143293.941639 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94878.73 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 9487.87 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152781.814845 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85390.86 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 8539.09 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161320.900730 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76851.77 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 7685.18 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169006.078027 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69166.60 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 6916.66 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175922.737594 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62249.94 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 6224.99 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182147.731205 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56024.94 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 5602.49 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187750.225454 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50422.45 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 5042.24 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192792.470278 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45380.20 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 4538.02 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197330.490620 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40842.18 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 4084.22 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201414.708928 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36757.96 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3675.80 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205090.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33082.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208690.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29482.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212290.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25882.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215890.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22282.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219490.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18682.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223090.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15082.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226690.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11482.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230290.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7882.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233890.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4282.17 sec) 11:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237490.505405 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 682.17 sec) 11:21:00 [GPUGRID] [rr_sim] 2238172.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984301087.32G/439.78G) 11:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852073.59 11:21:00 [---] pick_jobs_to_run() start 11:21:00 [---] pick_jobs_to_run() end 11:21:00 [---] [rr_sim] end 11:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:21:00 [---] [cpu_sched_debug] preliminary job list: 11:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:21:00 [---] [cpu_sched_debug] final job list: 11:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:21:00 [---] [cpu_sched_debug] enforce_run_list: end 11:21:00 [---] choose_project(): 40860.000000 11:21:00 [iThena.Measurements] doing RPC to get work 11:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394739.09 cpu 2140532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:21:00 [---] Got 0 tasks 11:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:22:00 [---] [rr_sim] doing sim: CPU sched 11:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984273496.96G/439.78G) 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238109.94 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 223810.99 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223810.993682 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014298.94 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 201429.89 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425240.887995 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812869.05 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 181286.90 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606527.792877 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631582.14 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 163158.21 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769686.007271 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468423.93 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 146842.39 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916528.400225 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321581.54 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 132158.15 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048686.553884 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189423.38 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 118942.34 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167628.892177 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070481.04 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 107048.10 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274676.996641 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963432.94 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 96343.29 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1371020.290659 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867089.65 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 86708.96 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457729.255274 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780380.68 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 78038.07 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535767.323428 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702342.61 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 70234.26 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1606001.584767 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632108.35 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 63210.84 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669212.419972 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568897.52 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 56889.75 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726102.171656 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512007.77 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 51200.78 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777302.948172 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460806.99 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 46080.70 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823383.647036 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414726.29 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 41472.63 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864856.276014 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373253.66 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 37325.37 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902181.642094 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335928.29 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 33592.83 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935774.471566 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302335.47 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 30233.55 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966008.018091 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272101.92 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 27210.19 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993218.209964 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244891.73 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 24489.17 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017707.382649 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220402.55 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 22040.26 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039747.638066 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198362.30 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 19836.23 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059583.867941 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178526.07 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 17852.61 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077436.474828 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160673.46 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 16067.35 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093503.821027 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144606.12 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 14460.61 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107964.432606 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130145.50 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 13014.55 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120978.983027 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117130.95 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 11713.10 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132692.078405 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105417.86 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 10541.79 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143233.864246 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94876.07 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 9487.61 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152721.471503 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85388.47 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 8538.85 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161260.318034 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76849.62 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 7684.96 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168945.279913 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69164.66 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 6916.47 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175861.745603 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62248.19 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 6224.82 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182086.564724 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56023.37 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 5602.34 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187688.901933 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50421.03 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 5042.10 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192731.005421 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45378.93 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 4537.89 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197268.898561 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40841.04 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 4084.10 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201353.002386 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36756.93 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3675.69 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205028.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33081.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208628.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29481.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212228.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25881.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215828.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22281.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219428.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18681.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223028.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15081.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226628.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11481.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230228.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7881.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233828.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4281.24 sec) 11:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237428.695829 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 681.24 sec) 11:22:00 [GPUGRID] [rr_sim] 2238109.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984273496.96G/439.78G) 11:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852070.85 11:22:00 [---] pick_jobs_to_run() start 11:22:00 [---] pick_jobs_to_run() end 11:22:00 [---] [rr_sim] end 11:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:22:00 [---] [cpu_sched_debug] preliminary job list: 11:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:22:00 [---] [cpu_sched_debug] final job list: 11:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:22:00 [---] [cpu_sched_debug] enforce_run_list: end 11:22:00 [---] choose_project(): 40920.000000 11:22:00 [iThena.Measurements] doing RPC to get work 11:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394679.09 cpu 2140472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:22:00 [---] Got 0 tasks 11:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:23:00 [---] [rr_sim] doing sim: CPU sched 11:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984245906.60G/439.78G) 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2238047.20 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 223804.72 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223804.719993 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014242.48 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 201424.25 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425228.967987 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812818.23 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 181281.82 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606510.791182 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631536.41 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 163153.64 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769664.432057 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468382.77 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 146838.28 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916502.708845 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321544.49 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 132154.45 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048657.157953 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189390.04 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 118939.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167596.162151 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070451.04 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 107045.10 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274641.265930 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963405.93 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 96340.59 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370981.859330 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867065.34 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 86706.53 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457688.393390 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780358.81 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 78035.88 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535724.274045 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702322.93 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 70232.29 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605956.566633 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632090.63 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 63209.06 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669165.629963 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568881.57 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 56888.16 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726053.786960 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511993.41 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 51199.34 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777253.128258 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460794.07 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 46079.41 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823332.535425 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414714.66 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 41471.47 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864804.001876 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373243.20 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 37324.32 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902128.321682 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335918.88 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 33591.89 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935720.209507 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302326.99 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 30232.70 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965952.908549 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272094.29 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 27209.43 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993162.337688 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244884.86 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 24488.49 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017650.823912 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220396.38 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 22039.64 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039690.461514 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198356.74 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 19835.67 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059526.135356 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178521.06 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 17852.11 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077378.241814 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160668.96 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 16066.90 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093445.137626 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144602.06 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 14460.21 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107905.343857 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130141.86 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 13014.19 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120919.529464 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117127.67 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 11712.77 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132632.296511 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105414.90 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 10541.49 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143173.786853 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94873.41 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 9487.34 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152661.128161 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85386.07 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 8538.61 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161199.735339 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76847.46 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 7684.75 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168884.481798 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69162.72 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 6916.27 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175800.753611 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62246.45 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 6224.64 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182025.398244 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56021.80 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 5602.18 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187627.578413 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50419.62 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 5041.96 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192669.540565 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45377.66 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 4537.77 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197207.306501 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40839.89 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 4083.99 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201291.295845 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36755.90 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3675.59 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204966.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33080.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208566.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29480.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212166.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25880.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215766.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22280.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219366.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18680.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222966.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15080.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226566.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11480.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230166.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7880.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233766.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4280.31 sec) 11:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237366.886253 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 680.31 sec) 11:23:00 [GPUGRID] [rr_sim] 2238047.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984245906.60G/439.78G) 11:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852068.11 11:23:00 [---] pick_jobs_to_run() start 11:23:00 [---] pick_jobs_to_run() end 11:23:00 [---] [rr_sim] end 11:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:23:00 [---] [cpu_sched_debug] preliminary job list: 11:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:23:00 [---] [cpu_sched_debug] final job list: 11:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:23:00 [---] [cpu_sched_debug] enforce_run_list: end 11:23:00 [---] choose_project(): 40980.000000 11:23:00 [iThena.Measurements] doing RPC to get work 11:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394619.09 cpu 2140412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:23:00 [---] Got 0 tasks 11:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:24:00 [---] [rr_sim] doing sim: CPU sched 11:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984218316.25G/439.78G) 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237984.46 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 223798.45 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223798.446305 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014186.02 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 201418.60 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425217.047980 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812767.42 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 181276.74 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606493.789487 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631490.67 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 163149.07 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769642.856843 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468341.61 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 146834.16 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916477.017464 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321507.45 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 132150.74 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048627.762023 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189356.70 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 118935.67 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167563.432126 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070421.03 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 107042.10 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274605.535218 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963378.93 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 96337.89 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370943.428001 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867041.04 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 86704.10 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457647.531506 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780336.93 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 78033.69 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535681.224661 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702303.24 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 70230.32 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605911.548500 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632072.91 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 63207.29 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669118.839955 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568865.62 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 56886.56 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1726005.402265 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511979.06 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 51197.91 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777203.308343 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460781.15 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 46078.12 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823281.423814 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414703.04 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 41470.30 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864751.727738 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373232.74 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 37323.27 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902075.001269 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335909.46 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 33590.95 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935665.947447 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302318.52 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 30231.85 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965897.799008 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272086.66 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 27208.67 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993106.465412 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244878.00 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 24487.80 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017594.265176 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220390.20 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 22039.02 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039633.284963 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198351.18 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 19835.12 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059468.402772 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178516.06 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 17851.61 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077320.008800 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160664.45 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 16066.45 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093386.454225 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144598.01 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 14459.80 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107846.255108 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130138.21 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 13013.82 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120860.075902 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117124.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 11712.44 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132572.514617 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105411.95 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 10541.19 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143113.709460 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94870.75 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 9487.08 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152600.784819 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85383.68 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 8538.37 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161139.152643 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76845.31 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 7684.53 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168823.683683 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69160.78 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 6916.08 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175739.761620 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62244.70 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 6224.47 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181964.231763 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56020.23 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 5602.02 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187566.254892 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50418.21 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 5041.82 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192608.075708 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45376.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 4537.64 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197145.714442 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40838.75 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 4083.87 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201229.589303 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36754.87 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3675.49 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204905.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33079.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208505.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29479.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212105.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25879.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215705.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22279.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219305.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18679.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222905.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15079.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226505.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11479.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230105.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7879.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233705.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4279.39 sec) 11:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237305.076678 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 679.39 sec) 11:24:00 [GPUGRID] [rr_sim] 2237984.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984218316.25G/439.78G) 11:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852065.38 11:24:00 [---] pick_jobs_to_run() start 11:24:00 [---] pick_jobs_to_run() end 11:24:00 [---] [rr_sim] end 11:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:24:00 [---] [cpu_sched_debug] preliminary job list: 11:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:24:00 [---] [cpu_sched_debug] final job list: 11:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:24:00 [---] [cpu_sched_debug] enforce_run_list: end 11:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:25:00 [---] [rr_sim] doing sim: CPU sched 11:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984190725.89G/439.78G) 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237921.73 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 223792.17 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223792.172617 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014129.55 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 201412.96 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425205.127972 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812716.60 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 181271.66 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606476.787792 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631444.94 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 163144.49 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769621.281629 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468300.44 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 146830.04 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916451.326083 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321470.40 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 132147.04 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048598.366092 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189323.36 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 118932.34 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167530.702100 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070391.02 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 107039.10 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274569.804507 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963351.92 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 96335.19 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370904.996673 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 867016.73 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 86701.67 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457606.669622 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780315.06 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 78031.51 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535638.175277 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702283.55 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 70228.36 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605866.530366 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632055.20 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 63205.52 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669072.049946 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568849.68 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 56884.97 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725957.017569 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511964.71 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 51196.47 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777153.488429 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460768.24 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 46076.82 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823230.312203 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414691.41 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 41469.14 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864699.453599 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373222.27 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 37322.23 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902021.680856 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335900.05 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 33590.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935611.685388 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302310.04 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 30231.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965842.689466 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272079.04 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 27207.90 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993050.593136 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244871.13 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 24487.11 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017537.706439 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220384.02 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 22038.40 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039576.108412 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198345.62 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 19834.56 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059410.670188 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178511.06 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 17851.11 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077261.775786 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160659.95 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 16066.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093327.770824 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144593.96 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 14459.40 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107787.166359 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130134.56 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 13013.46 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120800.622340 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117121.10 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 11712.11 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132512.732723 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105408.99 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 10540.90 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143053.632067 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94868.09 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 9486.81 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152540.441477 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85381.28 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 8538.13 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161078.569947 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76843.16 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 7684.32 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168762.885569 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69158.84 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 6915.88 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175678.769629 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62242.96 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 6224.30 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181903.065283 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56018.66 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 5601.87 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187504.931371 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50416.79 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 5041.68 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192546.610851 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45375.12 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 4537.51 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197084.122383 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40837.60 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 4083.76 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201167.882761 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36753.84 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3675.38 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204843.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33078.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208443.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29478.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212043.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25878.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215643.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22278.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219243.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18678.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222843.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15078.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226443.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11478.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230043.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7878.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233643.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4278.46 sec) 11:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237243.267102 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 678.46 sec) 11:25:00 [GPUGRID] [rr_sim] 2237921.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984190725.89G/439.78G) 11:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852062.64 11:25:00 [---] pick_jobs_to_run() start 11:25:00 [---] pick_jobs_to_run() end 11:25:00 [---] [rr_sim] end 11:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:25:00 [---] [cpu_sched_debug] preliminary job list: 11:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:25:00 [---] [cpu_sched_debug] final job list: 11:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:25:00 [---] [cpu_sched_debug] enforce_run_list: end 11:25:00 [---] choose_project(): 41100.000000 11:25:00 [iThena.Measurements] doing RPC to get work 11:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394499.09 cpu 2140292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:25:00 [---] Got 0 tasks 11:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:26:00 [---] [rr_sim] doing sim: CPU sched 11:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984163135.53G/439.78G) 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237858.99 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 223785.90 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223785.898929 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014073.09 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 201407.31 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425193.207964 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812665.78 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 181266.58 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606459.786097 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631399.20 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 163139.92 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769599.706416 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468259.28 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 146825.93 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916425.634703 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321433.35 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 132143.34 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048568.970161 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189290.02 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 118929.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167497.972074 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070361.02 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 107036.10 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274534.073795 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963324.92 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 96332.49 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370866.565344 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866992.42 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 86699.24 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457565.807738 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780293.18 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 78029.32 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535595.125893 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702263.86 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 70226.39 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605821.512233 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632037.48 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 63203.75 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1669025.259938 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568833.73 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 56883.37 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725908.632873 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511950.36 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 51195.04 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777103.668514 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460755.32 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 46075.53 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823179.200591 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414679.79 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 41467.98 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864647.179461 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373211.81 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 37321.18 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901968.360444 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335890.63 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 33589.06 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935557.423328 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302301.57 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 30230.16 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965787.579924 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272071.41 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 27207.14 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992994.720860 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244864.27 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 24486.43 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017481.147703 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220377.84 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 22037.78 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039518.931861 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198340.06 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 19834.01 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059352.937604 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178506.05 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 17850.61 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077203.542772 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160655.45 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 16065.54 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093269.087423 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144589.90 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 14458.99 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107728.077610 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130130.91 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 13013.09 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120741.168777 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117117.82 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 11711.78 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132452.950828 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105406.04 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 10540.60 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142993.554674 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94865.43 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 9486.54 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152480.098135 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85378.89 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 8537.89 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161017.987251 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76841.00 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 7684.10 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168702.087454 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69156.90 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 6915.69 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175617.777637 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62241.21 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 6224.12 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181841.898802 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56017.09 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 5601.71 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187443.607851 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50415.38 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 5041.54 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192485.145994 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45373.84 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 4537.38 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197022.530324 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40836.46 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 4083.65 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201106.176220 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36752.81 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3675.28 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204781.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33077.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208381.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29477.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211981.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25877.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215581.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22277.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219181.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18677.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222781.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15077.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226381.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11477.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229981.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7877.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233581.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4277.53 sec) 11:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237181.457527 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 677.53 sec) 11:26:00 [GPUGRID] [rr_sim] 2237858.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984163135.53G/439.78G) 11:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852059.90 11:26:00 [---] pick_jobs_to_run() start 11:26:00 [---] pick_jobs_to_run() end 11:26:00 [---] [rr_sim] end 11:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:26:00 [---] [cpu_sched_debug] preliminary job list: 11:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:26:00 [---] [cpu_sched_debug] final job list: 11:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:26:00 [---] [cpu_sched_debug] enforce_run_list: end 11:26:00 [---] choose_project(): 41160.000000 11:26:00 [iThena.Measurements] doing RPC to get work 11:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394439.09 cpu 2140232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:26:00 [---] Got 0 tasks 11:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:27:00 [---] [rr_sim] doing sim: CPU sched 11:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984135545.18G/439.78G) 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237796.25 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 223779.63 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223779.625240 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2014016.63 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 201401.66 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425181.287957 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812614.96 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 181261.50 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606442.784402 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631353.47 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 163135.35 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769578.131202 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468218.12 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 146821.81 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916399.943322 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321396.31 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 132139.63 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048539.574230 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189256.68 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 118925.67 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167465.242048 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070331.01 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 107033.10 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274498.343083 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963297.91 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 96329.79 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370828.134016 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866968.12 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 86696.81 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457524.945854 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780271.31 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 78027.13 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535552.076509 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702244.18 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 70224.42 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605776.494099 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632019.76 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 63201.98 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668978.469929 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568817.78 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 56881.78 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725860.248177 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511936.00 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 51193.60 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777053.848600 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460742.40 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 46074.24 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823128.088980 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414668.16 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 41466.82 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864594.905323 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373201.35 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 37320.13 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901915.040031 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335881.21 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 33588.12 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935503.161268 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302293.09 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 30229.31 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965732.470382 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272063.78 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 27206.38 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992938.848584 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244857.40 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 24485.74 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017424.588966 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220371.66 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 22037.17 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039461.755310 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198334.50 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 19833.45 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059295.205019 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178501.05 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 17850.10 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077145.309758 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160650.94 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 16065.09 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093210.404023 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144585.85 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 14458.58 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107668.988861 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130127.26 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 13012.73 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120681.715215 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117114.54 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 11711.45 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132393.168934 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105403.08 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 10540.31 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142933.477281 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94862.78 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 9486.28 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152419.754793 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85376.50 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 8537.65 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160957.404555 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76838.85 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 7683.88 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168641.289340 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69154.96 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 6915.50 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175556.785646 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62239.47 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 6223.95 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181780.732322 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56015.52 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 5601.55 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187382.284330 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50413.97 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 5041.40 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192423.681138 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45372.57 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 4537.26 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196960.938264 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40835.31 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 4083.53 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201044.469678 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36751.78 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3675.18 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204719.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33076.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208319.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29476.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211919.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25876.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215519.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22276.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219119.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18676.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222719.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15076.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226319.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11476.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229919.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7876.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233519.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4276.60 sec) 11:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237119.647951 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 676.60 sec) 11:27:00 [GPUGRID] [rr_sim] 2237796.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984135545.18G/439.78G) 11:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852057.17 11:27:00 [---] pick_jobs_to_run() start 11:27:00 [---] pick_jobs_to_run() end 11:27:00 [---] [rr_sim] end 11:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:27:00 [---] [cpu_sched_debug] preliminary job list: 11:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:27:00 [---] [cpu_sched_debug] final job list: 11:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:27:00 [---] [cpu_sched_debug] enforce_run_list: end 11:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:28:00 [---] [rr_sim] doing sim: CPU sched 11:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984107954.82G/439.78G) 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237733.52 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 223773.35 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223773.351552 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013960.16 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 201396.02 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425169.367949 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812564.15 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 181256.41 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606425.782707 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631307.73 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 163130.77 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769556.555988 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468176.96 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 146817.70 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916374.251942 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321359.26 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 132135.93 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048510.178300 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189223.34 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 118922.33 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167432.512022 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070301.00 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 107030.10 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274462.612372 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963270.90 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 96327.09 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370789.702687 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866943.81 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 86694.38 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457484.083970 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780249.43 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 78024.94 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535509.027126 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702224.49 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 70222.45 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605731.475965 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632002.04 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 63200.20 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668931.679921 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568801.84 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 56880.18 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725811.863481 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511921.65 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 51192.17 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1777004.028685 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460729.49 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 46072.95 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823076.977369 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414656.54 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 41465.65 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864542.631184 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373190.88 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 37319.09 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901861.719618 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335871.80 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 33587.18 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935448.899209 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302284.62 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 30228.46 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965677.360840 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272056.15 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 27205.62 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992882.976308 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244850.54 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 24485.05 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017368.030230 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220365.49 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 22036.55 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039404.578759 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198328.94 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 19832.89 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059237.472435 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178496.04 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 17849.60 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077087.076744 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160646.44 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 16064.64 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093151.720622 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144581.79 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 14458.18 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107609.900112 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130123.62 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 13012.36 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120622.261653 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117111.25 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 11711.13 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132333.387040 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105400.13 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 10540.01 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142873.399888 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94860.12 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 9486.01 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152359.411451 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85374.10 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 8537.41 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160896.821859 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76836.69 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 7683.67 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168580.491225 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69153.02 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 6915.30 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175495.793655 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62237.72 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 6223.77 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181719.565841 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56013.95 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 5601.39 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187320.960810 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50412.55 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 5041.26 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192362.216281 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45371.30 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 4537.13 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196899.346205 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40834.17 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 4083.42 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200982.763137 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36750.75 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3675.08 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204657.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33075.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208257.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29475.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211857.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25875.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215457.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22275.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219057.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18675.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222657.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15075.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226257.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11475.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229857.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7875.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233457.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4275.68 sec) 11:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2237057.838375 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 675.68 sec) 11:28:00 [GPUGRID] [rr_sim] 2237733.52: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984107954.82G/439.78G) 11:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852054.43 11:28:00 [---] pick_jobs_to_run() start 11:28:00 [---] pick_jobs_to_run() end 11:28:00 [---] [rr_sim] end 11:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:28:00 [---] [cpu_sched_debug] preliminary job list: 11:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:28:00 [---] [cpu_sched_debug] final job list: 11:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:28:00 [---] [cpu_sched_debug] enforce_run_list: end 11:28:00 [---] choose_project(): 41280.000000 11:28:00 [iThena.Measurements] doing RPC to get work 11:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 567119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 394319.09 cpu 2140112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:28:00 [---] Got 0 tasks 11:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:34:00 [---] [rr_sim] doing sim: CPU sched 11:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984080364.47G/439.78G) 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237670.78 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 223767.08 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223767.077864 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013903.70 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 201390.37 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425157.447942 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812513.33 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 181251.33 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606408.781011 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631262.00 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 163126.20 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769534.980774 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468135.80 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 146813.58 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916348.560561 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321322.22 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 132132.22 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048480.782369 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189190.00 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 118919.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167399.781996 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070271.00 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 107027.10 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274426.881660 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963243.90 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 96324.39 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370751.271358 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866919.51 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 86691.95 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457443.222087 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780227.56 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 78022.76 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535465.977742 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702204.80 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 70220.48 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605686.457832 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631984.32 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 63198.43 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668884.889913 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568785.89 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 56878.59 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725763.478785 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511907.30 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 51190.73 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776954.208771 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460716.57 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 46071.66 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1823025.865758 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414644.91 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 41464.49 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864490.357046 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373180.42 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 37318.04 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901808.399205 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335862.38 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 33586.24 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935394.637149 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302276.14 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 30227.61 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965622.251298 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272048.53 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 27204.85 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992827.104032 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244843.67 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 24484.37 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017311.471493 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220359.31 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 22035.93 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039347.402208 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198323.38 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 19832.34 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059179.739851 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178491.04 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 17849.10 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077028.843730 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160641.93 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 16064.19 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093093.037221 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144577.74 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 14457.77 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107550.811363 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130119.97 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 13012.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120562.808091 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117107.97 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 11710.80 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132273.605146 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105397.17 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 10539.72 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142813.322495 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94857.46 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 9485.75 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152299.068109 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85371.71 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 8537.17 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160836.239163 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76834.54 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 7683.45 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168519.693110 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69151.09 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 6915.11 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175434.801663 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62235.98 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 6223.60 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181658.399361 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56012.38 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 5601.24 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187259.637289 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50411.14 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 5041.11 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192300.751424 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45370.03 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 4537.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196837.754146 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40833.02 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 4083.30 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200921.056595 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36749.72 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3674.97 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204596.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33074.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208196.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29474.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211796.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25874.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215396.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22274.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218996.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18674.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222596.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15074.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226196.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11474.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229796.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7874.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233396.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4274.75 sec) 11:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236996.028800 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 674.75 sec) 11:34:00 [GPUGRID] [rr_sim] 2237670.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984080364.47G/439.78G) 11:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852351.69 11:34:00 [---] pick_jobs_to_run() start 11:34:00 [---] pick_jobs_to_run() end 11:34:00 [---] [rr_sim] end 11:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:34:00 [---] [cpu_sched_debug] preliminary job list: 11:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:34:00 [---] [cpu_sched_debug] final job list: 11:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:34:00 [---] [cpu_sched_debug] enforce_run_list: end 11:34:00 [---] choose_project(): 41640.000000 11:34:00 [iThena.Measurements] doing RPC to get work 11:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393959.09 cpu 2140052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2140052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2140052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:34:00 [---] Got 0 tasks 11:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:35:00 [---] [rr_sim] doing sim: CPU sched 11:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984052774.11G/439.78G) 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237608.04 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 223760.80 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223760.804176 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013847.24 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 201384.72 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425145.527934 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812462.51 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 181246.25 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606391.779316 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631216.26 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 163121.63 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769513.405561 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468094.64 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 146809.46 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916322.869180 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321285.17 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 132128.52 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048451.386438 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189156.66 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 118915.67 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167367.051970 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070240.99 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 107024.10 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274391.150949 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963216.89 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 96321.69 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370712.840030 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866895.20 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 86689.52 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457402.360203 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780205.68 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 78020.57 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535422.928358 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702185.11 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 70218.51 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605641.439698 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631966.60 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 63196.66 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668838.099904 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568769.94 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 56876.99 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725715.094089 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511892.95 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 51189.29 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776904.388856 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460703.65 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 46070.37 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822974.754146 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414633.29 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 41463.33 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864438.082908 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373169.96 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 37317.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901755.078793 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335852.96 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 33585.30 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935340.375089 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302267.67 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 30226.77 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965567.141756 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272040.90 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 27204.09 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992771.231756 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244836.81 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 24483.68 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017254.912756 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220353.13 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 22035.31 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039290.225657 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198317.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 19831.78 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059122.007267 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178486.03 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 17848.60 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076970.610716 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160637.43 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 16063.74 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093034.353820 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144573.69 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 14457.37 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107491.722614 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130116.32 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 13011.63 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120503.354528 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117104.69 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 11710.47 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132213.823251 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105394.22 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 10539.42 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142753.245102 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94854.80 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 9485.48 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152238.724768 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85369.32 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 8536.93 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160775.656467 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76832.39 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 7683.24 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168458.894996 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69149.15 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 6914.91 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175373.809672 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62234.23 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 6223.42 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181597.232881 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56010.81 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 5601.08 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187198.313768 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50409.73 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 5040.97 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192239.286567 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45368.76 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 4536.88 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196776.162086 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40831.88 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 4083.19 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200859.350053 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36748.69 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3674.87 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204534.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33073.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208134.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29473.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211734.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25873.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215334.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22273.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218934.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18673.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222534.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15073.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226134.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11473.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229734.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7873.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233334.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4273.82 sec) 11:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236934.219224 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 673.82 sec) 11:35:00 [GPUGRID] [rr_sim] 2237608.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984052774.11G/439.78G) 11:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852348.95 11:35:00 [---] pick_jobs_to_run() start 11:35:00 [---] pick_jobs_to_run() end 11:35:00 [---] [rr_sim] end 11:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:35:00 [---] [cpu_sched_debug] preliminary job list: 11:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:35:00 [---] [cpu_sched_debug] final job list: 11:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:35:00 [---] [cpu_sched_debug] enforce_run_list: end 11:35:00 [---] choose_project(): 41700.000000 11:35:00 [iThena.Measurements] doing RPC to get work 11:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393899.09 cpu 2139992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:35:00 [---] Got 0 tasks 11:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:36:00 [---] [rr_sim] doing sim: CPU sched 11:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (984025183.75G/439.78G) 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237545.30 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 223754.53 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223754.530488 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013790.77 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 201379.08 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425133.607926 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812411.70 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 181241.17 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606374.777621 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631170.53 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 163117.05 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769491.830347 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468053.47 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 146805.35 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916297.177800 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321248.13 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 132124.81 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048421.990507 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189123.31 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 118912.33 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167334.321944 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070210.98 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 107021.10 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274355.420237 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963189.88 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 96318.99 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370674.408701 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866870.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 86687.09 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457361.498319 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780183.81 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 78018.38 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535379.878974 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702165.43 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 70216.54 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605596.421564 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631948.88 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 63194.89 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668791.309896 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568753.99 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 56875.40 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725666.709394 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511878.60 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 51187.86 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776854.568942 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460690.74 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 46069.07 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822923.642535 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414621.66 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 41462.17 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864385.808769 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373159.50 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 37315.95 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901701.758380 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335843.55 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 33584.35 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935286.113030 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302259.19 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 30225.92 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965512.032214 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272033.27 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 27203.33 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992715.359480 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244829.95 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 24482.99 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017198.354020 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220346.95 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 22034.70 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039233.049105 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198312.26 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 19831.23 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059064.274682 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178481.03 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 17848.10 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076912.377702 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160632.93 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 16063.29 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092975.670419 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144569.63 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 14456.96 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107432.633865 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130112.67 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 13011.27 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120443.900966 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117101.40 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 11710.14 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132154.041357 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105391.26 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 10539.13 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142693.167709 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94852.14 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 9485.21 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152178.381426 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85366.92 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 8536.69 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160715.073771 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76830.23 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 7683.02 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168398.096881 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69147.21 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 6914.72 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175312.817681 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62232.49 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 6223.25 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181536.066400 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56009.24 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 5600.92 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187136.990248 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50408.31 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 5040.83 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192177.821710 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45367.48 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 4536.75 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196714.570027 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40830.73 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 4083.07 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200797.643512 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36747.66 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3674.77 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204472.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33072.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208072.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29472.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211672.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25872.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215272.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22272.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218872.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18672.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222472.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15072.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226072.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11472.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229672.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7872.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233272.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4272.90 sec) 11:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236872.409648 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 672.90 sec) 11:36:00 [GPUGRID] [rr_sim] 2237545.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (984025183.75G/439.78G) 11:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852346.22 11:36:00 [---] pick_jobs_to_run() start 11:36:00 [---] pick_jobs_to_run() end 11:36:00 [---] [rr_sim] end 11:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:36:00 [---] [cpu_sched_debug] preliminary job list: 11:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:36:00 [---] [cpu_sched_debug] final job list: 11:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:36:00 [---] [cpu_sched_debug] enforce_run_list: end 11:36:00 [---] choose_project(): 41760.000000 11:36:00 [iThena.Measurements] doing RPC to get work 11:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393839.09 cpu 2139932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:36:00 [---] Got 0 tasks 11:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:37:00 [---] [rr_sim] doing sim: CPU sched 11:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983997593.40G/439.78G) 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237482.57 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 223748.26 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223748.256799 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013734.31 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 201373.43 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425121.687919 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812360.88 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 181236.09 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606357.775926 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631124.79 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 163112.48 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769470.255133 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1468012.31 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 146801.23 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916271.486419 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321211.08 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 132121.11 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048392.594577 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189089.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 118909.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167301.591918 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070180.98 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 107018.10 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274319.689526 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963162.88 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 96316.29 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370635.977373 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866846.59 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 86684.66 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457320.636435 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780161.93 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 78016.19 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535336.829591 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702145.74 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 70214.57 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605551.403431 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631931.16 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 63193.12 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668744.519887 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568738.05 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 56873.80 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725618.324698 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511864.24 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 51186.42 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776804.749027 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460677.82 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 46067.78 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822872.530924 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414610.04 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 41461.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864333.534631 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373149.03 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 37314.90 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901648.437967 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335834.13 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 33583.41 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935231.850970 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302250.72 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 30225.07 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965456.922672 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272025.65 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 27202.56 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992659.487204 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244823.08 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 24482.31 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017141.795283 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220340.77 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 22034.08 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039175.872554 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198306.70 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 19830.67 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059006.542098 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178476.03 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 17847.60 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076854.144688 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160628.42 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 16062.84 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092916.987018 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144565.58 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 14456.56 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107373.545116 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130109.02 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 13010.90 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120384.447404 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117098.12 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 11709.81 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132094.259463 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105388.31 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 10538.83 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142633.090316 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94849.48 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 9484.95 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152118.038084 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85364.53 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 8536.45 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160654.491075 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76828.08 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 7682.81 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168337.298766 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69145.27 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 6914.53 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175251.825689 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62230.74 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 6223.07 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181474.899920 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56007.67 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 5600.77 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187075.666727 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50406.90 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 5040.69 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192116.356854 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45366.21 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 4536.62 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196652.977968 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40829.59 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 4082.96 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200735.936970 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36746.63 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3674.66 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204410.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33071.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208010.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29471.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211610.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25871.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215210.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22271.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218810.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18671.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222410.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15071.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226010.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11471.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229610.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7871.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233210.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4271.97 sec) 11:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236810.600073 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 671.97 sec) 11:37:00 [GPUGRID] [rr_sim] 2237482.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983997593.40G/439.78G) 11:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852343.48 11:37:00 [---] pick_jobs_to_run() start 11:37:00 [---] pick_jobs_to_run() end 11:37:00 [---] [rr_sim] end 11:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:37:00 [---] [cpu_sched_debug] preliminary job list: 11:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:37:00 [---] [cpu_sched_debug] final job list: 11:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:37:00 [---] [cpu_sched_debug] enforce_run_list: end 11:37:00 [---] choose_project(): 41820.000000 11:37:00 [iThena.Measurements] doing RPC to get work 11:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393779.09 cpu 2139872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:37:00 [---] Got 0 tasks 11:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:38:00 [---] [rr_sim] doing sim: CPU sched 11:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983970003.04G/439.78G) 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237419.83 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 223741.98 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223741.983111 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013677.85 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 201367.78 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425109.767911 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812310.06 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 181231.01 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606340.774231 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631079.06 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 163107.91 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769448.679919 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467971.15 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 146797.12 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916245.795038 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321174.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 132117.40 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048363.198646 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189056.63 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 118905.66 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167268.861892 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070150.97 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 107015.10 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274283.958814 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963135.87 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 96313.59 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370597.546044 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866822.29 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 86682.23 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457279.774551 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780140.06 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 78014.01 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535293.780207 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702126.05 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 70212.61 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605506.385297 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631913.45 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 63191.34 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668697.729879 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568722.10 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 56872.21 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725569.940002 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511849.89 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 51184.99 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776754.929113 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460664.90 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 46066.49 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822821.419313 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414598.41 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 41459.84 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864281.260493 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373138.57 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 37313.86 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901595.117555 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335824.71 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 33582.47 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935177.588910 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302242.24 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 30224.22 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965401.813130 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272018.02 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 27201.80 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992603.614928 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244816.22 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 24481.62 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017085.236547 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220334.59 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 22033.46 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039118.696003 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198301.14 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 19830.11 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058948.809514 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178471.02 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 17847.10 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076795.911674 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160623.92 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 16062.39 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092858.303618 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144561.53 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 14456.15 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107314.456367 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130105.37 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 13010.54 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120324.993841 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117094.84 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 11709.48 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132034.477568 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105385.35 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 10538.54 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142573.012923 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94846.82 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 9484.68 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152057.694742 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85362.14 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 8536.21 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160593.908379 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76825.92 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 7682.59 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168276.500652 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69143.33 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 6914.33 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175190.833698 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62229.00 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 6222.90 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181413.733439 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56006.10 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 5600.61 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187014.343206 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50405.49 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 5040.55 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192054.891997 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45364.94 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 4536.49 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196591.385908 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40828.45 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 4082.84 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200674.230429 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36745.60 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3674.56 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204348.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33071.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207948.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29471.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211548.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25871.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215148.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22271.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218748.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18671.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222348.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15071.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225948.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11471.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229548.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7871.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233148.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4271.04 sec) 11:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236748.790497 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 671.04 sec) 11:38:00 [GPUGRID] [rr_sim] 2237419.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983970003.04G/439.78G) 11:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852340.74 11:38:00 [---] pick_jobs_to_run() start 11:38:00 [---] pick_jobs_to_run() end 11:38:00 [---] [rr_sim] end 11:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:38:00 [---] [cpu_sched_debug] preliminary job list: 11:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:38:00 [---] [cpu_sched_debug] final job list: 11:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:38:00 [---] [cpu_sched_debug] enforce_run_list: end 11:38:00 [---] choose_project(): 41880.000000 11:38:00 [iThena.Measurements] doing RPC to get work 11:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393719.09 cpu 2139812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:38:00 [---] Got 0 tasks 11:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:39:00 [---] [rr_sim] doing sim: CPU sched 11:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983942412.69G/439.78G) 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237357.09 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 223735.71 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223735.709423 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013621.38 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 201362.14 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425097.847904 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812259.25 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 181225.92 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606323.772536 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1631033.32 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 163103.33 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769427.104705 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467929.99 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 146793.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916220.103658 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321136.99 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 132113.70 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048333.802715 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1189023.29 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 118902.33 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167236.131866 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070120.96 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 107012.10 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274248.228103 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963108.87 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 96310.89 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370559.114715 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866797.98 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 86679.80 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457238.912667 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780118.18 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 78011.82 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535250.730823 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702106.36 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 70210.64 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605461.367164 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631895.73 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 63189.57 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668650.939870 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568706.15 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 56870.62 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725521.555306 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511835.54 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 51183.55 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776705.109198 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460651.99 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 46065.20 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822770.307702 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414586.79 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 41458.68 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864228.986354 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373128.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 37312.81 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901541.797142 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335815.30 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 33581.53 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935123.326851 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302233.77 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 30223.38 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965346.703588 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272010.39 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 27201.04 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992547.742653 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244809.35 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 24480.94 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017028.677810 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220328.42 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 22032.84 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039061.519452 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198295.57 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 19829.56 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058891.076930 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178466.02 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 17846.60 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076737.678660 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160619.42 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 16061.94 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092799.620217 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144557.47 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 14455.75 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107255.367618 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130101.73 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 13010.17 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120265.540279 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117091.55 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 11709.16 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131974.695674 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105382.40 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 10538.24 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142512.935530 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94844.16 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 9484.42 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151997.351400 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85359.74 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 8535.97 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160533.325683 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76823.77 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 7682.38 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168215.702537 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69141.39 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 6914.14 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175129.841706 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62227.25 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 6222.73 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181352.566959 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56004.53 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 5600.45 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186953.019686 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50404.07 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 5040.41 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191993.427140 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45363.67 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 4536.37 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196529.793849 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40827.30 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 4082.73 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200612.523887 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36744.57 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3674.46 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204286.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33070.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207886.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29470.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211486.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25870.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215086.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22270.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218686.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18670.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222286.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15070.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225886.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11470.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229486.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7870.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233086.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4270.11 sec) 11:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236686.980921 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 670.11 sec) 11:39:00 [GPUGRID] [rr_sim] 2237357.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983942412.69G/439.78G) 11:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852338.01 11:39:00 [---] pick_jobs_to_run() start 11:39:00 [---] pick_jobs_to_run() end 11:39:00 [---] [rr_sim] end 11:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:39:00 [---] [cpu_sched_debug] preliminary job list: 11:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:39:00 [---] [cpu_sched_debug] final job list: 11:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:39:00 [---] [cpu_sched_debug] enforce_run_list: end 11:39:00 [---] choose_project(): 41940.000000 11:39:00 [iThena.Measurements] doing RPC to get work 11:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393659.09 cpu 2139752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:39:00 [---] Got 0 tasks 11:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:40:00 [---] [rr_sim] doing sim: CPU sched 11:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983914822.33G/439.78G) 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237294.36 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 223729.44 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223729.435735 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013564.92 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 201356.49 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425085.927896 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812208.43 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 181220.84 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606306.770841 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630987.59 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 163098.76 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769405.529492 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467888.83 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 146788.88 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916194.412277 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321099.95 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 132109.99 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048304.406784 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188989.95 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 118899.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167203.401841 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070090.96 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 107009.10 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274212.497391 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963081.86 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 96308.19 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370520.683387 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866773.67 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 86677.37 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457198.050783 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780096.31 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 78009.63 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535207.681439 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702086.68 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 70208.67 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605416.349030 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631878.01 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 63187.80 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668604.149862 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568690.21 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 56869.02 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725473.170610 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511821.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 51182.12 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776655.289284 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460639.07 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 46063.91 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822719.196090 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414575.16 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 41457.52 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864176.712216 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373117.65 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 37311.76 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901488.476729 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335805.88 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 33580.59 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935069.064791 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302225.29 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 30222.53 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965291.594047 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272002.76 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 27200.28 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992491.870377 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244802.49 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 24480.25 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016972.119074 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220322.24 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 22032.22 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039004.342901 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198290.01 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 19829.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058833.344346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178461.01 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 17846.10 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076679.445646 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160614.91 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 16061.49 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092740.936816 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144553.42 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 14455.34 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107196.278869 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130098.08 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 13009.81 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120206.086717 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117088.27 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 11708.83 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131914.913780 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105379.44 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 10537.94 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142452.858137 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94841.50 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 9484.15 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151937.008058 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85357.35 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 8535.73 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160472.742987 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76821.61 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 7682.16 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168154.904423 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69139.45 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 6913.95 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175068.849715 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62225.51 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 6222.55 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181291.400478 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56002.96 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 5600.30 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186891.696165 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50402.66 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 5040.27 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191931.962283 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45362.40 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 4536.24 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196468.201790 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40826.16 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 4082.62 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200550.817345 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36743.54 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3674.35 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204225.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33069.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207825.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29469.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211425.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25869.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215025.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22269.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218625.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18669.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222225.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15069.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225825.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11469.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229425.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7869.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233025.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4269.19 sec) 11:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236625.171346 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 669.19 sec) 11:40:00 [GPUGRID] [rr_sim] 2237294.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983914822.33G/439.78G) 11:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852335.27 11:40:00 [---] pick_jobs_to_run() start 11:40:00 [---] pick_jobs_to_run() end 11:40:00 [---] [rr_sim] end 11:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:40:00 [---] [cpu_sched_debug] preliminary job list: 11:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:40:00 [---] [cpu_sched_debug] final job list: 11:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:40:00 [---] [cpu_sched_debug] enforce_run_list: end 11:40:00 [---] choose_project(): 42000.000000 11:40:00 [iThena.Measurements] doing RPC to get work 11:40:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393599.09 cpu 2139692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:40:00 [---] Got 0 tasks 11:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:41:00 [---] [rr_sim] doing sim: CPU sched 11:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983914822.33G/439.78G) 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237294.36 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 223729.44 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223729.435735 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013564.92 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 201356.49 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425085.927896 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812208.43 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 181220.84 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606306.770841 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630987.59 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 163098.76 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769405.529492 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467888.83 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 146788.88 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916194.412277 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321099.95 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 132109.99 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048304.406784 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188989.95 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 118899.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167203.401841 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070090.96 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 107009.10 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274212.497391 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963081.86 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 96308.19 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370520.683387 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866773.67 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 86677.37 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457198.050783 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780096.31 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 78009.63 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535207.681439 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702086.68 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 70208.67 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605416.349030 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631878.01 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 63187.80 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668604.149862 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568690.21 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 56869.02 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725473.170610 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511821.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 51182.12 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776655.289284 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460639.07 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 46063.91 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822719.196090 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414575.16 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 41457.52 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864176.712216 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373117.65 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 37311.76 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901488.476729 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335805.88 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 33580.59 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935069.064791 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302225.29 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 30222.53 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965291.594047 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272002.76 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 27200.28 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992491.870377 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244802.49 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 24480.25 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016972.119074 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220322.24 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 22032.22 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039004.342901 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198290.01 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 19829.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058833.344346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178461.01 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 17846.10 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076679.445646 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160614.91 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 16061.49 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092740.936816 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144553.42 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 14455.34 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107196.278869 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130098.08 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 13009.81 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120206.086717 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117088.27 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 11708.83 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131914.913780 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105379.44 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 10537.94 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142452.858137 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94841.50 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 9484.15 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151937.008058 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85357.35 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 8535.73 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160472.742987 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76821.61 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 7682.16 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168154.904423 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69139.45 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 6913.95 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175068.849715 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62225.51 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 6222.55 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181291.400478 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56002.96 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 5600.30 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186891.696165 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50402.66 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 5040.27 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191931.962283 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45362.40 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 4536.24 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196468.201790 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40826.16 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 4082.62 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200550.817345 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36743.54 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3674.35 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204225.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33069.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207825.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29469.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211425.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25869.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215025.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22269.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218625.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18669.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222225.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15069.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225825.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11469.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229425.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7869.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233025.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4269.19 sec) 11:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236625.171346 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 669.19 sec) 11:41:00 [GPUGRID] [rr_sim] 2237294.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983914822.33G/439.78G) 11:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852395.27 11:41:00 [---] pick_jobs_to_run() start 11:41:00 [---] pick_jobs_to_run() end 11:41:00 [---] [rr_sim] end 11:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:41:00 [---] [cpu_sched_debug] preliminary job list: 11:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:41:00 [---] [cpu_sched_debug] final job list: 11:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:41:00 [---] [cpu_sched_debug] enforce_run_list: end 11:41:00 [---] choose_project(): 42060.000000 11:41:00 [iThena.Measurements] doing RPC to get work 11:41:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393539.09 cpu 2139692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:41:00 [---] Got 0 tasks 11:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:42:00 [---] [rr_sim] doing sim: CPU sched 11:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983887231.97G/439.78G) 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237231.62 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 223723.16 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223723.162046 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013508.46 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 201350.85 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425074.007888 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812157.61 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 181215.76 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606289.769146 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630941.85 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 163094.19 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769383.954278 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467847.67 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 146784.77 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916168.720897 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321062.90 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 132106.29 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048275.010853 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188956.61 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 118895.66 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167170.671815 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070060.95 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 107006.09 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274176.766680 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963054.85 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 96305.49 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370482.252058 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866749.37 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 86674.94 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457157.188899 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780074.43 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 78007.44 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535164.632055 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702066.99 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 70206.70 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605371.330896 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631860.29 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 63186.03 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668557.359853 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568674.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 56867.43 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725424.785914 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511806.83 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 51180.68 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776605.469369 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460626.15 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 46062.62 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822668.084479 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414563.54 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 41456.35 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864124.438078 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373107.18 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 37310.72 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901435.156316 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335796.46 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 33579.65 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935014.802731 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302216.82 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 30221.68 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965236.484505 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271995.14 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 27199.51 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992435.998101 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244795.62 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 24479.56 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016915.560337 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220316.06 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 22031.61 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038947.166350 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198284.45 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 19828.45 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058775.611761 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178456.01 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 17845.60 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076621.212632 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160610.41 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 16061.04 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092682.253415 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144549.37 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 14454.94 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107137.190120 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130094.43 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 13009.44 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120146.633155 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117084.99 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 11708.50 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131855.131886 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105376.49 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 10537.65 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142392.780744 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94838.84 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 9483.88 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151876.664716 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85354.96 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 8535.50 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160412.160291 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76819.46 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 7681.95 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168094.106308 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69137.51 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 6913.75 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175007.857724 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62223.76 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 6222.38 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181230.233998 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56001.39 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 5600.14 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186830.372645 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50401.25 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 5040.12 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191870.497427 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45361.12 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 4536.11 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196406.609730 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40825.01 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 4082.50 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200489.110804 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36742.51 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3674.25 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204163.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33068.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207763.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29468.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211363.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25868.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214963.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22268.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218563.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18668.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222163.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15068.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225763.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11468.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229363.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7868.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232963.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4268.26 sec) 11:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236563.361770 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 668.26 sec) 11:42:00 [GPUGRID] [rr_sim] 2237231.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983887231.97G/439.78G) 11:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852392.53 11:42:00 [---] pick_jobs_to_run() start 11:42:00 [---] pick_jobs_to_run() end 11:42:00 [---] [rr_sim] end 11:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:42:00 [---] [cpu_sched_debug] preliminary job list: 11:42:00 [---] [cpu_sched_debug] final job list: 11:42:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 11:42:00 [---] [work_fetch] Request work fetch: CPUs idle 11:42:00 [---] [cpu_sched_debug] enforce_run_list: end 11:42:00 [---] choose_project(): 42120.000000 11:42:00 [iThena.Measurements] doing RPC to get work 11:42:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393479.09 cpu 2139632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:42:00 [---] Got 0 tasks 11:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:43:00 [---] [rr_sim] doing sim: CPU sched 11:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983887231.97G/439.78G) 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237231.62 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 223723.16 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223723.162046 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013508.46 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 201350.85 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425074.007888 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812157.61 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 181215.76 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606289.769146 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630941.85 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 163094.19 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769383.954278 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467847.67 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 146784.77 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916168.720897 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321062.90 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 132106.29 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048275.010853 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188956.61 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 118895.66 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167170.671815 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070060.95 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 107006.09 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274176.766680 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963054.85 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 96305.49 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370482.252058 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866749.37 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 86674.94 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457157.188899 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780074.43 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 78007.44 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535164.632055 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702066.99 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 70206.70 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605371.330896 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631860.29 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 63186.03 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668557.359853 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568674.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 56867.43 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725424.785914 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511806.83 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 51180.68 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776605.469369 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460626.15 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 46062.62 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822668.084479 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414563.54 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 41456.35 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864124.438078 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373107.18 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 37310.72 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901435.156316 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335796.46 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 33579.65 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935014.802731 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302216.82 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 30221.68 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965236.484505 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271995.14 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 27199.51 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992435.998101 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244795.62 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 24479.56 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016915.560337 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220316.06 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 22031.61 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038947.166350 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198284.45 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 19828.45 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058775.611761 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178456.01 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 17845.60 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076621.212632 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160610.41 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 16061.04 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092682.253415 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144549.37 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 14454.94 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107137.190120 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130094.43 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 13009.44 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120146.633155 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117084.99 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 11708.50 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131855.131886 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105376.49 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 10537.65 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142392.780744 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94838.84 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 9483.88 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151876.664716 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85354.96 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 8535.50 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160412.160291 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76819.46 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 7681.95 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168094.106308 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69137.51 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 6913.75 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175007.857724 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62223.76 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 6222.38 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181230.233998 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56001.39 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 5600.14 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186830.372645 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50401.25 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 5040.12 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191870.497427 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45361.12 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 4536.11 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196406.609730 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40825.01 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 4082.50 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200489.110804 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36742.51 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3674.25 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204163.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33068.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207763.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29468.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211363.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25868.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214963.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22268.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218563.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18668.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222163.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15068.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225763.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11468.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229363.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7868.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232963.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4268.26 sec) 11:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236563.361770 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 668.26 sec) 11:43:00 [GPUGRID] [rr_sim] 2237231.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983887231.97G/439.78G) 11:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852452.53 11:43:00 [---] pick_jobs_to_run() start 11:43:00 [---] pick_jobs_to_run() end 11:43:00 [---] [rr_sim] end 11:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:43:00 [---] [cpu_sched_debug] preliminary job list: 11:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:43:00 [---] [cpu_sched_debug] final job list: 11:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:43:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 983887231.97G 11:43:00 [---] [cpu_sched_debug] enforce_run_list: end 11:43:00 [---] choose_project(): 42180.000000 11:43:00 [iThena.Measurements] doing RPC to get work 11:43:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393419.09 cpu 2139632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:43:00 [---] Got 0 tasks 11:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:44:00 [---] [rr_sim] doing sim: CPU sched 11:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983859641.62G/439.78G) 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237168.88 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 223716.89 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223716.888358 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013452.00 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 201345.20 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425062.087881 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812106.80 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 181210.68 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606272.767451 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630896.12 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 163089.61 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769362.379064 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467806.50 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 146780.65 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916143.029516 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1321025.85 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 132102.59 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048245.614923 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188923.27 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 118892.33 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167137.941789 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070030.94 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 107003.09 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274141.035968 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963027.85 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 96302.78 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370443.820730 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866725.06 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 86672.51 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457116.327015 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780052.56 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 78005.26 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535121.582672 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702047.30 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 70204.73 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605326.312763 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631842.57 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 63184.26 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668510.569845 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568658.31 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 56865.83 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725376.401219 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511792.48 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 51179.25 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776555.649455 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460613.23 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 46061.32 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822616.972868 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414551.91 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 41455.19 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864072.163939 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373096.72 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 37309.67 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901381.835904 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335787.05 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 33578.70 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934960.540672 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302208.34 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 30220.83 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965181.374963 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271987.51 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 27198.75 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992380.125825 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244788.76 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 24478.88 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016859.001601 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220309.88 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 22030.99 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038889.989799 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198278.89 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 19827.89 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058717.879177 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178451.00 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 17845.10 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076562.979618 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160605.90 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 16060.59 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092623.570014 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144545.31 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 14454.53 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107078.101371 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130090.78 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 13009.08 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120087.179592 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117081.70 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 11708.17 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131795.349991 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105373.53 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 10537.35 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142332.703350 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94836.18 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 9483.62 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151816.321374 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85352.56 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 8535.26 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160351.577595 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76817.31 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 7681.73 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168033.308193 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69135.58 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 6913.56 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174946.865732 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62222.02 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 6222.20 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181169.067517 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55999.82 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 5599.98 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186769.049124 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50399.83 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 5039.98 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191809.032570 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45359.85 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 4535.99 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196345.017671 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40823.87 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 4082.39 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200427.404262 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36741.48 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3674.15 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204101.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33067.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207701.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29467.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211301.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25867.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214901.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22267.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218501.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18667.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222101.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15067.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225701.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11467.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229301.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7867.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232901.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4267.33 sec) 11:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236501.552194 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 667.33 sec) 11:44:00 [GPUGRID] [rr_sim] 2237168.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983859641.62G/439.78G) 11:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852449.80 11:44:00 [---] pick_jobs_to_run() start 11:44:00 [---] pick_jobs_to_run() end 11:44:00 [---] [rr_sim] end 11:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:44:00 [---] [cpu_sched_debug] preliminary job list: 11:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 11:44:00 [---] [cpu_sched_debug] final job list: 11:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:44:00 [---] [cpu_sched_debug] enforce_run_list: end 11:44:00 [---] choose_project(): 42240.000000 11:44:00 [iThena.Measurements] doing RPC to get work 11:44:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393359.09 cpu 2139572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:44:00 [---] Got 0 tasks 11:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:45:00 [---] [rr_sim] doing sim: CPU sched 11:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983832051.26G/439.78G) 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237106.15 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 223710.61 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223710.614670 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013395.53 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 201339.55 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425050.167873 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812055.98 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 181205.60 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606255.765756 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630850.38 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 163085.04 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769340.803850 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467765.34 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 146776.53 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916117.338135 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320988.81 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 132098.88 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048216.218992 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188889.93 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 118888.99 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167105.211763 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1070000.93 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 107000.09 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274105.305257 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 963000.84 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 96300.08 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370405.389401 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866700.76 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 86670.08 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457075.465131 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780030.68 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 78003.07 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535078.533288 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702027.61 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 70202.76 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605281.294629 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631824.85 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 63182.49 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668463.779836 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568642.37 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 56864.24 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725328.016523 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511778.13 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 51177.81 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776505.829541 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460600.32 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 46060.03 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822565.861257 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414540.29 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 41454.03 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864019.889801 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373086.26 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 37308.63 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901328.515491 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335777.63 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 33577.76 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934906.278612 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302199.87 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 30219.99 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965126.265421 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271979.88 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 27197.99 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992324.253549 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244781.89 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 24478.19 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016802.442864 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220303.70 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 22030.37 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038832.813248 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198273.33 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 19827.33 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058660.146593 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178446.00 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 17844.60 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076504.746604 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160601.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 16060.14 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092564.886613 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144541.26 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 14454.13 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107019.012622 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130087.13 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 13008.71 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120027.726030 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117078.42 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 11707.84 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131735.568097 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105370.58 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 10537.06 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142272.625957 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94833.52 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 9483.35 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151755.978032 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85350.17 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 8535.02 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160290.994899 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76815.15 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 7681.52 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167972.510079 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69133.64 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 6913.36 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174885.873741 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62220.27 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 6222.03 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181107.901037 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55998.25 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 5599.82 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186707.725603 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50398.42 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 5039.84 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191747.567713 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45358.58 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 4535.86 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196283.425612 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40822.72 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 4082.27 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200365.697721 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36740.45 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3674.04 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204039.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33066.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207639.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29466.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211239.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25866.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214839.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22266.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218439.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18666.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222039.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15066.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225639.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11466.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229239.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7866.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232839.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4266.40 sec) 11:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236439.742619 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 666.40 sec) 11:45:00 [GPUGRID] [rr_sim] 2237106.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983832051.26G/439.78G) 11:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852447.06 11:45:00 [---] pick_jobs_to_run() start 11:45:00 [---] pick_jobs_to_run() end 11:45:00 [---] [rr_sim] end 11:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:45:00 [---] [cpu_sched_debug] preliminary job list: 11:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:45:00 [---] [cpu_sched_debug] final job list: 11:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:45:00 [---] [cpu_sched_debug] enforce_run_list: end 11:45:00 [---] choose_project(): 42300.000000 11:45:00 [iThena.Measurements] doing RPC to get work 11:45:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393299.09 cpu 2139512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:45:00 [---] Got 0 tasks 11:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:46:00 [---] [rr_sim] doing sim: CPU sched 11:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983804460.90G/439.78G) 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2237043.41 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 223704.34 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223704.340982 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013339.07 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 201333.91 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425038.247866 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1812005.16 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 181200.52 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606238.764061 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630804.65 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 163080.46 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769319.228637 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467724.18 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 146772.42 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916091.646755 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320951.76 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 132095.18 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048186.823061 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188856.59 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 118885.66 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167072.481737 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069970.93 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 106997.09 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274069.574545 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962973.84 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 96297.38 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370366.958072 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866676.45 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 86667.65 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1457034.603247 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 780008.81 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 78000.88 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1535035.483904 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 702007.93 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 70200.79 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605236.276496 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631807.13 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 63180.71 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668416.989828 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568626.42 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 56862.64 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725279.631827 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511763.78 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 51176.38 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776456.009626 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460587.40 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 46058.74 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822514.749645 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414528.66 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 41452.87 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863967.615663 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373075.79 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 37307.58 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901275.195078 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335768.21 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 33576.82 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934852.016552 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302191.39 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 30219.14 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965071.155879 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271972.25 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 27197.23 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992268.381273 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244775.03 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 24477.50 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016745.884127 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220297.53 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 22029.75 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038775.636697 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198267.77 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 19826.78 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058602.414009 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178441.00 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 17844.10 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076446.513590 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160596.90 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 16059.69 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092506.203213 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144537.21 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 14453.72 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106959.923873 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130083.49 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 13008.35 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119968.272468 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117075.14 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 11707.51 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131675.786203 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105367.62 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 10536.76 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142212.548564 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94830.86 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 9483.09 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151695.634690 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85347.78 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 8534.78 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160230.412203 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76813.00 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 7681.30 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167911.711964 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69131.70 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 6913.17 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174824.881750 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62218.53 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 6221.85 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181046.734557 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55996.68 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 5599.67 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186646.402083 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50397.01 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 5039.70 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191686.102856 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45357.31 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 4535.73 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196221.833553 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40821.58 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 4082.16 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200303.991179 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36739.42 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3673.94 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203977.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33065.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207577.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29465.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211177.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25865.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214777.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22265.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218377.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18665.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221977.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15065.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225577.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11465.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229177.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7865.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232777.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4265.48 sec) 11:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236377.933043 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 665.48 sec) 11:46:00 [GPUGRID] [rr_sim] 2237043.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983804460.90G/439.78G) 11:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852444.32 11:46:00 [---] pick_jobs_to_run() start 11:46:00 [---] pick_jobs_to_run() end 11:46:00 [---] [rr_sim] end 11:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:46:00 [---] [cpu_sched_debug] preliminary job list: 11:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:46:00 [---] [cpu_sched_debug] final job list: 11:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:46:00 [---] [cpu_sched_debug] enforce_run_list: end 11:46:00 [---] choose_project(): 42360.000000 11:46:00 [iThena.Measurements] doing RPC to get work 11:46:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 566039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393239.09 cpu 2139452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:46:00 [---] Got 0 tasks 11:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:47:00 [---] [rr_sim] doing sim: CPU sched 11:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983776870.55G/439.78G) 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236980.67 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 223698.07 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223698.067294 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013282.61 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 201328.26 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425026.327858 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811954.35 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 181195.43 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606221.762366 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630758.91 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 163075.89 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769297.653423 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467683.02 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 146768.30 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916065.955374 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320914.72 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 132091.47 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048157.427130 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188823.25 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 118882.32 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167039.751711 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069940.92 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 106994.09 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1274033.843833 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962946.83 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 96294.68 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370328.526744 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866652.15 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 86665.21 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456993.741363 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779986.93 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 77998.69 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534992.434520 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701988.24 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 70198.82 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605191.258362 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631789.41 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 63178.94 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668370.199819 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568610.47 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 56861.05 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725231.247131 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511749.43 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 51174.94 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776406.189712 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460574.48 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 46057.45 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822463.638034 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414517.03 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 41451.70 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863915.341524 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373065.33 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 37306.53 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901221.874666 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335758.80 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 33575.88 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934797.754493 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302182.92 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 30218.29 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965016.046337 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271964.63 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 27196.46 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992212.508997 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244768.16 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 24476.82 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016689.325391 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220291.35 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 22029.13 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038718.460145 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198262.21 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 19826.22 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058544.681424 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178435.99 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 17843.60 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076388.280576 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160592.39 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 16059.24 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092447.519812 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144533.15 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 14453.32 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106900.835124 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130079.84 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 13007.98 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119908.818905 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117071.85 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 11707.19 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131616.004308 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105364.67 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 10536.47 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142152.471171 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94828.20 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 9482.82 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151635.291348 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85345.38 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 8534.54 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160169.829507 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76810.84 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 7681.08 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167850.913850 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69129.76 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 6912.98 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174763.889758 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62216.78 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 6221.68 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180985.568076 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55995.10 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 5599.51 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186585.078562 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50395.59 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 5039.56 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191624.638000 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45356.03 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 4535.60 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196160.241493 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40820.43 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 4082.04 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200242.284638 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36738.39 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3673.84 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203916.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33064.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207516.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29464.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211116.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25864.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214716.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22264.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218316.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18664.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221916.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15064.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225516.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11464.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229116.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7864.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232716.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4264.55 sec) 11:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236316.123467 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 664.55 sec) 11:47:00 [GPUGRID] [rr_sim] 2236980.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983776870.55G/439.78G) 11:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852441.59 11:47:00 [---] pick_jobs_to_run() start 11:47:00 [---] pick_jobs_to_run() end 11:47:00 [---] [rr_sim] end 11:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:47:00 [---] [cpu_sched_debug] preliminary job list: 11:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:47:00 [---] [cpu_sched_debug] final job list: 11:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:47:00 [---] [cpu_sched_debug] enforce_run_list: end 11:47:00 [---] choose_project(): 42420.000000 11:47:00 [iThena.Measurements] doing RPC to get work 11:47:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393179.09 cpu 2139392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:47:00 [---] Got 0 tasks 11:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:48:00 [---] [rr_sim] doing sim: CPU sched 11:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983749280.19G/439.78G) 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236917.94 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 223691.79 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223691.793605 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013226.14 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 201322.61 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425014.407850 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811903.53 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 181190.35 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606204.760671 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630713.18 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 163071.32 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769276.078209 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467641.86 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 146764.19 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916040.263994 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320877.67 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 132087.77 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048128.031200 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188789.90 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 118878.99 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1167007.021685 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069910.91 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 106991.09 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273998.113122 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962919.82 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 96291.98 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370290.095415 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866627.84 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 86662.78 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456952.879479 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779965.06 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 77996.51 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534949.385137 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701968.55 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 70196.86 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605146.240228 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631771.70 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 63177.17 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668323.409811 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568594.53 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 56859.45 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725182.862435 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511735.07 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 51173.51 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776356.369797 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460561.57 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 46056.16 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822412.526423 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414505.41 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 41450.54 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863863.067386 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373054.87 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 37305.49 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901168.554253 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335749.38 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 33574.94 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934743.492433 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302174.44 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 30217.44 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964960.936795 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271957.00 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 27195.70 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992156.636721 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244761.30 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 24476.13 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016632.766654 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220285.17 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 22028.52 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038661.283594 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198256.65 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 19825.67 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058486.948840 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178430.99 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 17843.10 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076330.047562 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160587.89 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 16058.79 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092388.836411 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144529.10 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 14452.91 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106841.746375 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130076.19 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 13007.62 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119849.365343 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117068.57 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 11706.86 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131556.222414 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105361.71 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 10536.17 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142092.393778 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94825.54 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 9482.55 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151574.948006 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85342.99 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 8534.30 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160109.246811 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76808.69 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 7680.87 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167790.115735 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69127.82 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 6912.78 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174702.897767 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62215.04 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 6221.50 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180924.401596 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55993.53 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 5599.35 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186523.755041 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50394.18 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 5039.42 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191563.173143 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45354.76 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 4535.48 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196098.649434 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40819.29 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 4081.93 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200180.578096 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36737.36 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3673.74 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203854.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33063.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207454.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29463.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211054.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25863.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214654.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22263.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218254.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18663.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221854.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15063.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225454.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11463.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229054.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7863.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232654.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4263.62 sec) 11:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236254.313892 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 663.62 sec) 11:48:00 [GPUGRID] [rr_sim] 2236917.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983749280.19G/439.78G) 11:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852438.85 11:48:00 [---] pick_jobs_to_run() start 11:48:00 [---] pick_jobs_to_run() end 11:48:00 [---] [rr_sim] end 11:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:48:00 [---] [cpu_sched_debug] preliminary job list: 11:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:48:00 [---] [cpu_sched_debug] final job list: 11:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:48:00 [---] [cpu_sched_debug] enforce_run_list: end 11:48:00 [---] choose_project(): 42480.000000 11:48:00 [iThena.Measurements] doing RPC to get work 11:48:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393119.09 cpu 2139332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:48:00 [---] Got 0 tasks 11:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:49:00 [---] [rr_sim] doing sim: CPU sched 11:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983721689.84G/439.78G) 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236855.20 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 223685.52 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223685.519917 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013169.68 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 201316.97 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 425002.487843 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811852.71 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 181185.27 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606187.758976 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630667.44 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 163066.74 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769254.502995 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467600.70 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 146760.07 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 916014.572613 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320840.63 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 132084.06 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048098.635269 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188756.56 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 118875.66 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166974.291659 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069880.91 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 106988.09 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273962.382410 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962892.82 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 96289.28 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370251.664087 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866603.54 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 86660.35 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456912.017595 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779943.18 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 77994.32 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534906.335753 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701948.86 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 70194.89 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605101.222095 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631753.98 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 63175.40 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668276.619802 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568578.58 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 56857.86 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725134.477739 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511720.72 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 51172.07 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776306.549883 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460548.65 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 46054.86 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822361.414812 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414493.78 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 41449.38 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863810.793248 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373044.41 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 37304.44 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901115.233840 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335739.97 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 33574.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934689.230373 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302165.97 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 30216.60 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964905.827253 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271949.37 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 27194.94 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992100.764445 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244754.43 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 24475.44 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016576.207918 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220278.99 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 22027.90 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038604.107043 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198251.09 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 19825.11 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058429.216256 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178425.98 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 17842.60 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076271.814548 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160583.38 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 16058.34 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092330.153010 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144525.05 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 14452.50 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106782.657626 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130072.54 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 13007.25 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119789.911781 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117065.29 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 11706.53 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131496.440520 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105358.76 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 10535.88 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142032.316385 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94822.88 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 9482.29 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151514.604664 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85340.59 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 8534.06 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160048.664115 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76806.54 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 7680.65 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167729.317620 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69125.88 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 6912.59 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174641.905776 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62213.29 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 6221.33 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180863.235115 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55991.96 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 5599.20 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186462.431521 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50392.77 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 5039.28 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191501.708286 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45353.49 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 4535.35 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196037.057375 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40818.14 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 4081.81 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200118.871554 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36736.33 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3673.63 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203792.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33062.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207392.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29462.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210992.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25862.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214592.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22262.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218192.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18662.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221792.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15062.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225392.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11462.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228992.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7862.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232592.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4262.69 sec) 11:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236192.504316 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 662.69 sec) 11:49:00 [GPUGRID] [rr_sim] 2236855.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983721689.84G/439.78G) 11:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852436.11 11:49:00 [---] pick_jobs_to_run() start 11:49:00 [---] pick_jobs_to_run() end 11:49:00 [---] [rr_sim] end 11:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:49:00 [---] [cpu_sched_debug] preliminary job list: 11:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:49:00 [---] [cpu_sched_debug] final job list: 11:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:49:00 [---] [cpu_sched_debug] enforce_run_list: end 11:49:00 [---] choose_project(): 42540.000000 11:49:00 [iThena.Measurements] doing RPC to get work 11:49:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 393059.09 cpu 2139272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139272.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:49:00 [---] Got 0 tasks 11:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:50:00 [---] [rr_sim] doing sim: CPU sched 11:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983694099.48G/439.78G) 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236792.46 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 223679.25 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223679.246229 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013113.22 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 201311.32 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424990.567835 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811801.89 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 181180.19 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606170.757281 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630621.71 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 163062.17 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769232.927781 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467559.53 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 146755.95 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915988.881232 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320803.58 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 132080.36 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048069.239338 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188723.22 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 118872.32 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166941.561633 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069850.90 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 106985.09 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273926.651699 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962865.81 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 96286.58 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370213.232758 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866579.23 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 86657.92 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456871.155711 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779921.31 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 77992.13 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534863.286369 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701929.18 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 70192.92 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605056.203961 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631736.26 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 63173.63 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668229.829794 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568562.63 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 56856.26 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725086.093044 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511706.37 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 51170.64 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776256.729968 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460535.73 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 46053.57 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822310.303200 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414482.16 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 41448.22 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863758.519109 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373033.94 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 37303.39 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901061.913427 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335730.55 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 33573.05 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934634.968314 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302157.49 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 30215.75 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964850.717711 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271941.74 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 27194.17 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992044.892169 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244747.57 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 24474.76 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016519.649181 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220272.81 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 22027.28 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038546.930492 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198245.53 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 19824.55 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058371.483672 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178420.98 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 17842.10 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076213.581534 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160578.88 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 16057.89 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092271.469609 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144520.99 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 14452.10 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106723.568877 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130068.89 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 13006.89 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119730.458219 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117062.00 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 11706.20 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131436.658626 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105355.80 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 10535.58 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141972.238992 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94820.22 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 9482.02 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151454.261322 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85338.20 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 8533.82 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159988.081419 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76804.38 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 7680.44 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167668.519506 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69123.94 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 6912.39 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174580.913784 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62211.55 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 6221.15 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180802.068635 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55990.39 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 5599.04 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186401.108000 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50391.35 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 5039.14 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191440.243429 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45352.22 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 4535.22 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195975.465315 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40817.00 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 4081.70 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200057.165013 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36735.30 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3673.53 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203730.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33061.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207330.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29461.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210930.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25861.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214530.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22261.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218130.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18661.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221730.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15061.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225330.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11461.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228930.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7861.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232530.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4261.77 sec) 11:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236130.694740 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 661.77 sec) 11:50:00 [GPUGRID] [rr_sim] 2236792.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983694099.48G/439.78G) 11:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852433.37 11:50:00 [---] pick_jobs_to_run() start 11:50:00 [---] pick_jobs_to_run() end 11:50:00 [---] [rr_sim] end 11:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:50:00 [---] [cpu_sched_debug] preliminary job list: 11:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:50:00 [---] [cpu_sched_debug] final job list: 11:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:50:00 [---] [cpu_sched_debug] enforce_run_list: end 11:50:00 [---] choose_project(): 42600.000000 11:50:00 [iThena.Measurements] doing RPC to get work 11:50:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565799.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392999.09 cpu 2139212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139212.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:50:00 [---] Got 0 tasks 11:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:51:00 [---] [rr_sim] doing sim: CPU sched 11:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983666509.12G/439.78G) 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236729.73 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 223672.97 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223672.972541 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013056.75 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 201305.68 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424978.647827 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811751.08 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 181175.11 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606153.755585 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630575.97 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 163057.60 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769211.352568 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467518.37 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 146751.84 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915963.189852 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320766.54 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 132076.65 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048039.843407 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188689.88 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 118868.99 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166908.831607 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069820.89 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 106982.09 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273890.920987 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962838.80 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 96283.88 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370174.801429 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866554.92 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 86655.49 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456830.293827 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779899.43 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 77989.94 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534820.236985 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701909.49 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 70190.95 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1605011.185827 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631718.54 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 63171.85 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668183.039786 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568546.69 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 56854.67 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1725037.708348 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511692.02 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 51169.20 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776206.910054 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460522.82 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 46052.28 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822259.191589 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414470.53 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 41447.05 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863706.244971 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373023.48 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 37302.35 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901008.593015 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335721.13 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 33572.11 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934580.706254 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302149.02 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 30214.90 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964795.608169 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271934.12 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 27193.41 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991989.019893 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244740.71 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 24474.07 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016463.090445 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220266.63 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 22026.66 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038489.753941 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198239.97 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 19824.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058313.751088 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178415.97 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 17841.60 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076155.348520 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160574.38 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 16057.44 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092212.786208 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144516.94 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 14451.69 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106664.480128 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130065.25 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 13006.52 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119671.004656 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117058.72 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 11705.87 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131376.876731 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105352.85 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 10535.28 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141912.161599 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94817.56 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 9481.76 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151393.917980 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85335.81 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 8533.58 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159927.498723 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76802.23 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 7680.22 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167607.721391 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69122.00 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 6912.20 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174519.921793 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62209.80 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 6220.98 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180740.902154 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55988.82 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 5598.88 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186339.784480 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50389.94 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 5038.99 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191378.778572 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45350.95 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 4535.09 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195913.873256 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40815.85 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 4081.59 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199995.458471 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36734.27 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3673.43 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203668.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33060.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207268.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29460.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210868.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25860.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214468.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22260.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218068.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18660.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221668.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15060.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225268.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11460.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228868.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7860.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232468.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4260.84 sec) 11:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236068.885165 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 660.84 sec) 11:51:00 [GPUGRID] [rr_sim] 2236729.73: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983666509.12G/439.78G) 11:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852430.64 11:51:00 [---] pick_jobs_to_run() start 11:51:00 [---] pick_jobs_to_run() end 11:51:00 [---] [rr_sim] end 11:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:51:00 [---] [cpu_sched_debug] preliminary job list: 11:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:51:00 [---] [cpu_sched_debug] final job list: 11:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:51:00 [---] [cpu_sched_debug] enforce_run_list: end 11:51:00 [---] choose_project(): 42660.000000 11:51:00 [iThena.Measurements] doing RPC to get work 11:51:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565739.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392939.09 cpu 2139152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139152.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:51:00 [---] Got 0 tasks 11:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:52:00 [---] [rr_sim] doing sim: CPU sched 11:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983638918.77G/439.78G) 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236666.99 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 223666.70 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223666.698853 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2013000.29 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 201300.03 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424966.727820 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811700.26 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 181170.03 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606136.753890 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630530.23 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 163053.02 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769189.777354 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467477.21 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 146747.72 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915937.498471 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320729.49 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 132072.95 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1048010.447477 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188656.54 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 118865.65 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166876.101581 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069790.89 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 106979.09 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273855.190276 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962811.80 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 96281.18 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370136.370101 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866530.62 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 86653.06 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456789.431943 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779877.56 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 77987.76 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534777.187601 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701889.80 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 70188.98 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604966.167694 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631700.82 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 63170.08 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668136.249777 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568530.74 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 56853.07 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724989.323652 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511677.66 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 51167.77 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776157.090139 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460509.90 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 46050.99 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822208.079978 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414458.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 41445.89 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863653.970833 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373013.02 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 37301.30 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900955.272602 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335711.72 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 33571.17 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934526.444194 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302140.54 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 30214.05 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964740.498627 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271926.49 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 27192.65 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991933.147617 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244733.84 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 24473.38 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016406.531708 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220260.46 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 22026.05 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038432.577390 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198234.41 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 19823.44 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058256.018503 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178410.97 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 17841.10 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076097.115506 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160569.87 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 16056.99 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092154.102808 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144512.89 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 14451.29 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106605.391379 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130061.60 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 13006.16 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119611.551094 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117055.44 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 11705.54 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131317.094837 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105349.89 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 10534.99 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141852.084206 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94814.90 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 9481.49 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151333.574638 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85333.41 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 8533.34 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159866.916027 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76800.07 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 7680.01 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167546.923277 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69120.07 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 6912.01 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174458.929801 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62208.06 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 6220.81 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180679.735674 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55987.25 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 5598.73 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186278.460959 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50388.53 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 5038.85 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191317.313716 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45349.67 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 4534.97 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195852.281197 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40814.71 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 4081.47 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199933.751930 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36733.24 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3673.32 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203607.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33059.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207207.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29459.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210807.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25859.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214407.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22259.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218007.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18659.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221607.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15059.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225207.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11459.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228807.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7859.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232407.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4259.91 sec) 11:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2236007.075589 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 659.91 sec) 11:52:00 [GPUGRID] [rr_sim] 2236666.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983638918.77G/439.78G) 11:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852427.90 11:52:00 [---] pick_jobs_to_run() start 11:52:00 [---] pick_jobs_to_run() end 11:52:00 [---] [rr_sim] end 11:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:52:00 [---] [cpu_sched_debug] preliminary job list: 11:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:52:00 [---] [cpu_sched_debug] final job list: 11:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:52:00 [---] [cpu_sched_debug] enforce_run_list: end 11:52:00 [---] choose_project(): 42720.000000 11:52:00 [iThena.Measurements] doing RPC to get work 11:52:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565679.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392879.09 cpu 2139092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139092.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:52:00 [---] Got 0 tasks 11:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:53:00 [---] [rr_sim] doing sim: CPU sched 11:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983611328.41G/439.78G) 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236604.25 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 223660.43 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223660.425164 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012943.83 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 201294.38 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424954.807812 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811649.44 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 181164.94 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606119.752195 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630484.50 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 163048.45 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769168.202140 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467436.05 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 146743.60 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915911.807090 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320692.44 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 132069.24 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047981.051546 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188623.20 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 118862.32 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166843.371556 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069760.88 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 106976.09 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273819.459564 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962784.79 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 96278.48 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370097.938772 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866506.31 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 86650.63 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456748.570059 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779855.68 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 77985.57 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534734.138218 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701870.11 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 70187.01 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604921.149560 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631683.10 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 63168.31 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668089.459769 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568514.79 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 56851.48 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724940.938956 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511663.31 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 51166.33 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776107.270225 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460496.98 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 46049.70 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822156.968367 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414447.28 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 41444.73 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863601.696694 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373002.55 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 37300.26 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900901.952189 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335702.30 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 33570.23 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934472.182135 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302132.07 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 30213.21 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964685.389085 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271918.86 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 27191.89 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991877.275341 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244726.98 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 24472.70 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016349.972971 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220254.28 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 22025.43 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038375.400839 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198228.85 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 19822.89 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058198.285919 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178405.97 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 17840.60 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076038.882492 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160565.37 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 16056.54 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092095.419407 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144508.83 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 14450.88 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106546.302630 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130057.95 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 13005.79 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119552.097532 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117052.15 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 11705.22 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131257.312943 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105346.94 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 10534.69 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141792.006813 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94812.24 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 9481.22 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151273.231296 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85331.02 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 8533.10 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159806.333331 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76797.92 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 7679.79 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167486.125162 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69118.13 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 6911.81 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174397.937810 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62206.31 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 6220.63 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180618.569193 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55985.68 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 5598.57 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186217.137438 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50387.11 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 5038.71 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191255.848859 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45348.40 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 4534.84 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195790.689137 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40813.56 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 4081.36 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199872.045388 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36732.21 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3673.22 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203545.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33058.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207145.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29458.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210745.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25858.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214345.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22258.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217945.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18658.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221545.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15058.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225145.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11458.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228745.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7858.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232345.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4258.99 sec) 11:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235945.266013 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 658.99 sec) 11:53:00 [GPUGRID] [rr_sim] 2236604.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983611328.41G/439.78G) 11:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852425.16 11:53:00 [---] pick_jobs_to_run() start 11:53:00 [---] pick_jobs_to_run() end 11:53:00 [---] [rr_sim] end 11:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:53:00 [---] [cpu_sched_debug] preliminary job list: 11:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:53:00 [---] [cpu_sched_debug] final job list: 11:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:53:00 [---] [cpu_sched_debug] enforce_run_list: end 11:53:00 [---] choose_project(): 42780.000000 11:53:00 [iThena.Measurements] doing RPC to get work 11:53:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565619.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392819.09 cpu 2139032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2139032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2139032.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:53:00 [---] Got 0 tasks 11:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:54:00 [---] [rr_sim] doing sim: CPU sched 11:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983583738.05G/439.78G) 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236541.51 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 223654.15 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223654.151476 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012887.36 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 201288.74 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424942.887805 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811598.63 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 181159.86 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606102.750500 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630438.76 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 163043.88 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769146.626926 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467394.89 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 146739.49 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915886.115710 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320655.40 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 132065.54 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047951.655615 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188589.86 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 118858.99 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166810.641530 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069730.87 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 106973.09 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273783.728853 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962757.79 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 96275.78 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370059.507444 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866482.01 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 86648.20 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456707.708175 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779833.81 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 77983.38 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534691.088834 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701850.43 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 70185.04 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604876.131427 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631665.38 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 63166.54 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1668042.669760 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568498.85 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 56849.88 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724892.554260 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511648.96 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 51164.90 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776057.450310 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460484.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 46048.41 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822105.856755 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414435.66 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 41443.57 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863549.422556 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372992.09 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 37299.21 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900848.631776 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335692.88 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 33569.29 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934417.920075 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302123.59 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 30212.36 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964630.279544 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271911.24 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 27191.12 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991821.403065 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244720.11 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 24472.01 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016293.414235 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220248.10 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 22024.81 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038318.224288 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198223.29 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 19822.33 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058140.553335 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178400.96 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 17840.10 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075980.649478 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160560.87 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 16056.09 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092036.736006 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144504.78 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 14450.48 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106487.213881 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130054.30 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 13005.43 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119492.643969 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117048.87 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 11704.89 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131197.531049 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105343.98 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 10534.40 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141731.929420 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94809.59 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 9480.96 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151212.887954 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85328.63 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 8532.86 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159745.750635 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76795.76 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 7679.58 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167425.327047 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69116.19 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 6911.62 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174336.945819 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62204.57 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 6220.46 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180557.402713 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55984.11 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 5598.41 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186155.813918 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50385.70 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 5038.57 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191194.384002 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45347.13 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 4534.71 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195729.097078 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40812.42 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 4081.24 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199810.338846 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36731.18 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3673.12 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203483.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33058.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207083.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29458.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210683.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25858.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214283.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22258.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217883.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18658.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221483.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15058.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225083.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11458.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228683.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7858.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232283.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4258.06 sec) 11:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235883.456438 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 658.06 sec) 11:54:00 [GPUGRID] [rr_sim] 2236541.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983583738.05G/439.78G) 11:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852422.43 11:54:00 [---] pick_jobs_to_run() start 11:54:00 [---] pick_jobs_to_run() end 11:54:00 [---] [rr_sim] end 11:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:54:00 [---] [cpu_sched_debug] preliminary job list: 11:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:54:00 [---] [cpu_sched_debug] final job list: 11:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:54:00 [---] [cpu_sched_debug] enforce_run_list: end 11:54:00 [---] choose_project(): 42840.000000 11:54:00 [iThena.Measurements] doing RPC to get work 11:54:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565559.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392759.09 cpu 2138972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138972.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:54:00 [---] Got 0 tasks 11:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:55:00 [---] [rr_sim] doing sim: CPU sched 11:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983556147.70G/439.78G) 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236478.78 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 223647.88 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223647.877788 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012830.90 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 201283.09 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424930.967797 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811547.81 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 181154.78 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606085.748805 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630393.03 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 163039.30 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769125.051713 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467353.73 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 146735.37 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915860.424329 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320618.35 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 132061.84 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047922.259684 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188556.52 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 118855.65 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166777.911504 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069700.87 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 106970.09 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273747.998141 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962730.78 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 96273.08 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1370021.076115 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866457.70 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 86645.77 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456666.846291 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779811.93 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 77981.19 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534648.039450 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701830.74 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 70183.07 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604831.113293 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631647.66 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 63164.77 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667995.879752 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568482.90 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 56848.29 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724844.169564 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511634.61 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 51163.46 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1776007.630396 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460471.15 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 46047.11 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822054.745144 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414424.03 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 41442.40 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863497.148418 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372981.63 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 37298.16 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900795.311364 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335683.47 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 33568.35 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934363.658015 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302115.12 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 30211.51 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964575.170002 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271903.61 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 27190.36 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991765.530789 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244713.25 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 24471.32 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016236.855498 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220241.92 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 22024.19 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038261.047736 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198217.73 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 19821.77 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058082.820751 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178395.96 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 17839.60 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075922.416464 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160556.36 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 16055.64 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091978.052605 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144500.73 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 14450.07 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106428.125132 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130050.65 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 13005.07 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119433.190407 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117045.59 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 11704.56 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131137.749154 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105341.03 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 10534.10 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141671.852027 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94806.93 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 9480.69 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151152.544612 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85326.23 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 8532.62 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159685.167939 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76793.61 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 7679.36 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167364.528933 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69114.25 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 6911.42 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174275.953827 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62202.82 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 6220.28 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180496.236233 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55982.54 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 5598.25 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186094.490397 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50384.29 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 5038.43 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191132.919145 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45345.86 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 4534.59 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195667.505019 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40811.27 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 4081.13 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199748.632305 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36730.15 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3673.01 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203421.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33057.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207021.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29457.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210621.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25857.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214221.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22257.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217821.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18657.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221421.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15057.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225021.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11457.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228621.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7857.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232221.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4257.13 sec) 11:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235821.646862 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 657.13 sec) 11:55:00 [GPUGRID] [rr_sim] 2236478.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983556147.70G/439.78G) 11:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852419.69 11:55:00 [---] pick_jobs_to_run() start 11:55:00 [---] pick_jobs_to_run() end 11:55:00 [---] [rr_sim] end 11:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:55:00 [---] [cpu_sched_debug] preliminary job list: 11:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:55:00 [---] [cpu_sched_debug] final job list: 11:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:55:00 [---] [cpu_sched_debug] enforce_run_list: end 11:55:00 [---] choose_project(): 42900.000000 11:55:00 [iThena.Measurements] doing RPC to get work 11:55:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565499.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392699.09 cpu 2138912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138912.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:55:00 [---] Got 0 tasks 11:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:56:00 [---] [rr_sim] doing sim: CPU sched 11:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983528557.34G/439.78G) 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236416.04 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 223641.60 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223641.604100 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012774.44 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 201277.44 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424919.047789 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811496.99 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 181149.70 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606068.747110 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630347.29 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 163034.73 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769103.476499 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467312.56 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 146731.26 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915834.732949 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320581.31 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 132058.13 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047892.863753 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188523.18 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 118852.32 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166745.181478 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069670.86 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 106967.09 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273712.267430 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962703.77 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 96270.38 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369982.644786 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866433.40 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 86643.34 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456625.984407 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779790.06 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 77979.01 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534604.990066 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701811.05 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 70181.11 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604786.095159 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631629.95 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 63162.99 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667949.089743 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568466.95 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 56846.70 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724795.784869 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511620.26 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 51162.03 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775957.810481 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460458.23 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 46045.82 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1822003.633533 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414412.41 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 41441.24 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863444.874279 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372971.17 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 37297.12 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900741.990951 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335674.05 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 33567.41 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934309.395956 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302106.65 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 30210.66 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964520.060460 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271895.98 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 27189.60 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991709.658513 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244706.38 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 24470.64 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016180.296762 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220235.74 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 22023.57 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038203.871185 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198212.17 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 19821.22 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058025.088166 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178390.95 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 17839.10 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075864.183449 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160551.86 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 16055.19 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091919.369204 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144496.67 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 14449.67 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106369.036383 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130047.00 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 13004.70 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119373.736845 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117042.30 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 11704.23 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131077.967260 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105338.07 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 10533.81 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141611.774634 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94804.27 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 9480.43 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151092.201270 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85323.84 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 8532.38 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159624.585243 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76791.46 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 7679.15 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167303.730818 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69112.31 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 6911.23 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174214.961836 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62201.08 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 6220.11 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180435.069752 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55980.97 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 5598.10 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186033.166877 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50382.87 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 5038.29 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191071.454289 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45344.59 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 4534.46 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195605.912959 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40810.13 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 4081.01 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199686.925763 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36729.12 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3672.91 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203359.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33056.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206959.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29456.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210559.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25856.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214159.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22256.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217759.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18656.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221359.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15056.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224959.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11456.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228559.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7856.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232159.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4256.20 sec) 11:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235759.837287 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 656.20 sec) 11:56:00 [GPUGRID] [rr_sim] 2236416.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983528557.34G/439.78G) 11:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852416.95 11:56:00 [---] pick_jobs_to_run() start 11:56:00 [---] pick_jobs_to_run() end 11:56:00 [---] [rr_sim] end 11:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:56:00 [---] [cpu_sched_debug] preliminary job list: 11:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:56:00 [---] [cpu_sched_debug] final job list: 11:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:56:00 [---] [cpu_sched_debug] enforce_run_list: end 11:56:00 [---] choose_project(): 42960.000000 11:56:00 [iThena.Measurements] doing RPC to get work 11:56:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565439.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392639.09 cpu 2138852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138852.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:56:00 [---] Got 0 tasks 11:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:57:00 [---] [rr_sim] doing sim: CPU sched 11:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983500966.99G/439.78G) 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236353.30 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 223635.33 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223635.330411 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012717.97 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 201271.80 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424907.127782 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811446.18 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 181144.62 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606051.745415 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630301.56 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 163030.16 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769081.901285 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467271.40 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 146727.14 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915809.041568 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320544.26 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 132054.43 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047863.467823 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188489.84 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 118848.98 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166712.451452 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069640.85 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 106964.09 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273676.536718 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962676.77 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 96267.68 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369944.213458 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866409.09 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 86640.91 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456585.122523 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779768.18 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 77976.82 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534561.940683 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701791.36 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 70179.14 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604741.077026 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631612.23 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 63161.22 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667902.299735 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568451.00 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 56845.10 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724747.400173 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511605.90 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 51160.59 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775907.990567 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460445.31 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 46044.53 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821952.521922 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414400.78 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 41440.08 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863392.600141 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372960.70 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 37296.07 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900688.670538 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335664.63 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 33566.46 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934255.133896 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302098.17 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 30209.82 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964464.950918 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271888.35 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 27188.84 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991653.786238 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244699.52 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 24469.95 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016123.738025 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220229.57 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 22022.96 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038146.694634 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198206.61 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 19820.66 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057967.355582 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178385.95 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 17838.59 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075805.950435 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160547.35 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 16054.74 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091860.685803 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144492.62 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 14449.26 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106309.947635 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130043.36 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 13004.34 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119314.283283 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117039.02 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 11703.90 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131018.185366 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105335.12 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 10533.51 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141551.697241 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94801.61 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 9480.16 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151031.857928 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85321.45 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 8532.14 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159564.002547 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76789.30 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 7678.93 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167242.932704 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69110.37 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 6911.04 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174153.969845 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62199.33 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 6219.93 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180373.903272 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55979.40 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 5597.94 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185971.843356 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50381.46 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 5038.15 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191009.989432 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45343.31 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 4534.33 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195544.320900 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40808.98 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 4080.90 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199625.219222 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36728.08 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3672.81 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203298.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33055.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206898.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29455.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210498.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25855.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214098.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22255.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217698.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18655.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221298.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15055.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224898.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11455.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228498.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7855.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232098.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4255.28 sec) 11:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235698.027711 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 655.28 sec) 11:57:00 [GPUGRID] [rr_sim] 2236353.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983500966.99G/439.78G) 11:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852414.22 11:57:00 [---] pick_jobs_to_run() start 11:57:00 [---] pick_jobs_to_run() end 11:57:00 [---] [rr_sim] end 11:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:57:00 [---] [cpu_sched_debug] preliminary job list: 11:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:57:00 [---] [cpu_sched_debug] final job list: 11:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:57:00 [---] [cpu_sched_debug] enforce_run_list: end 11:57:00 [---] choose_project(): 43020.000000 11:57:00 [iThena.Measurements] doing RPC to get work 11:57:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565379.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392579.09 cpu 2138792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138792.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:57:00 [---] Got 0 tasks 11:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:58:00 [---] [rr_sim] doing sim: CPU sched 11:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983473376.63G/439.78G) 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236290.57 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 223629.06 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223629.056723 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012661.51 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 201266.15 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424895.207774 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811395.36 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 181139.54 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606034.743720 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630255.82 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 163025.58 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769060.326071 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467230.24 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 146723.02 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915783.350187 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320507.22 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 132050.72 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047834.071892 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188456.50 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 118845.65 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166679.721426 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069610.85 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 106961.08 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273640.806007 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962649.76 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 96264.98 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369905.782129 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866384.79 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 86638.48 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456544.260640 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779746.31 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 77974.63 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534518.891299 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701771.68 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 70177.17 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604696.058892 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631594.51 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 63159.45 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667855.509726 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568435.06 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 56843.51 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724699.015477 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511591.55 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 51159.16 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775858.170652 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460432.40 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 46043.24 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821901.410310 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414389.16 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 41438.92 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863340.326003 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372950.24 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 37295.02 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900635.350126 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335655.22 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 33565.52 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934200.871836 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302089.70 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 30208.97 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964409.841376 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271880.73 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 27188.07 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991597.913962 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244692.65 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 24469.27 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016067.179289 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220223.39 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 22022.34 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038089.518083 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198201.05 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 19820.10 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057909.622998 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178380.94 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 17838.09 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075747.717421 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160542.85 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 16054.28 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091802.002403 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144488.56 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 14448.86 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106250.858886 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130039.71 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 13003.97 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119254.829720 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117035.74 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 11703.57 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130958.403471 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105332.16 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 10533.22 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141491.619848 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94798.95 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 9479.89 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150971.514586 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85319.05 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 8531.91 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159503.419851 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76787.15 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 7678.71 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167182.134589 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69108.43 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 6910.84 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174092.977853 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62197.59 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 6219.76 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180312.736791 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55977.83 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 5597.78 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185910.519835 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50380.05 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 5038.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190948.524575 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45342.04 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 4534.20 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195482.728841 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40807.84 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 4080.78 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199563.512680 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36727.05 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3672.71 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203236.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33054.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206836.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29454.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210436.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25854.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214036.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22254.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217636.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18654.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221236.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15054.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224836.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11454.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228436.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7854.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232036.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4254.35 sec) 11:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235636.218135 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 654.35 sec) 11:58:00 [GPUGRID] [rr_sim] 2236290.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983473376.63G/439.78G) 11:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852411.48 11:58:00 [---] pick_jobs_to_run() start 11:58:00 [---] pick_jobs_to_run() end 11:58:00 [---] [rr_sim] end 11:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:58:00 [---] [cpu_sched_debug] preliminary job list: 11:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:58:00 [---] [cpu_sched_debug] final job list: 11:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:58:00 [---] [cpu_sched_debug] enforce_run_list: end 11:58:00 [---] choose_project(): 43080.000000 11:58:00 [iThena.Measurements] doing RPC to get work 11:58:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565319.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392519.09 cpu 2138732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138732.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:58:00 [---] Got 0 tasks 11:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 11:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 11:59:00 [---] [rr_sim] doing sim: CPU sched 11:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983445786.27G/439.78G) 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236227.83 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 223622.78 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223622.783035 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012605.05 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 201260.50 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424883.287767 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811344.54 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 181134.45 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606017.742025 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630210.09 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 163021.01 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769038.750857 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467189.08 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 146718.91 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915757.658807 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320470.17 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 132047.02 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047804.675961 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188423.15 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 118842.32 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166646.991400 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069580.84 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 106958.08 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273605.075295 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962622.76 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 96262.28 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369867.350801 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866360.48 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 86636.05 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456503.398756 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779724.43 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 77972.44 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534475.841915 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701751.99 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 70175.20 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604651.040759 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631576.79 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 63157.68 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667808.719718 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568419.11 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 56841.91 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724650.630781 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511577.20 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 51157.72 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775808.350738 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460419.48 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 46041.95 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821850.298699 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414377.53 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 41437.75 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863288.051864 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372939.78 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 37293.98 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900582.029713 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335645.80 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 33564.58 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934146.609777 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302081.22 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 30208.12 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964354.731834 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271873.10 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 27187.31 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991542.041686 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244685.79 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 24468.58 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016010.620552 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220217.21 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 22021.72 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038032.341532 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198195.49 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 19819.55 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057851.890414 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178375.94 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 17837.59 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075689.484407 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160538.35 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 16053.83 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091743.319002 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144484.51 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 14448.45 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106191.770137 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130036.06 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 13003.61 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119195.376158 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117032.45 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 11703.25 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130898.621577 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105329.21 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 10532.92 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141431.542455 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94796.29 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 9479.63 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150911.171244 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85316.66 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 8531.67 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159442.837155 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76784.99 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 7678.50 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167121.336474 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69106.49 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 6910.65 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174031.985862 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62195.84 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 6219.58 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180251.570311 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55976.26 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 5597.63 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185849.196315 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50378.63 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 5037.86 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190887.059718 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45340.77 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 4534.08 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195421.136781 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40806.69 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 4080.67 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199501.806138 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36726.02 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3672.60 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203174.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33053.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206774.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29453.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210374.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25853.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213974.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22253.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217574.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18653.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221174.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15053.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224774.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11453.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228374.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7853.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231974.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4253.42 sec) 11:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235574.408560 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 653.42 sec) 11:59:00 [GPUGRID] [rr_sim] 2236227.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983445786.27G/439.78G) 11:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852408.74 11:59:00 [---] pick_jobs_to_run() start 11:59:00 [---] pick_jobs_to_run() end 11:59:00 [---] [rr_sim] end 11:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 11:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 11:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 11:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 11:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 11:59:00 [---] [cpu_sched_debug] preliminary job list: 11:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:59:00 [---] [cpu_sched_debug] final job list: 11:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 11:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 11:59:00 [---] [cpu_sched_debug] enforce_run_list: end 11:59:00 [---] choose_project(): 43140.000000 11:59:00 [iThena.Measurements] doing RPC to get work 11:59:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565259.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392459.09 cpu 2138672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138672.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 11:59:00 [---] Got 0 tasks 12:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:00:00 [---] [rr_sim] doing sim: CPU sched 12:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983418195.92G/439.78G) 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236165.09 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 223616.51 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223616.509347 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012548.58 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 201254.86 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424871.367759 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811293.73 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 181129.37 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 606000.740330 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630164.35 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 163016.44 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 769017.175644 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467147.92 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 146714.79 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915731.967426 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320433.13 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 132043.31 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047775.280030 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188389.81 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 118838.98 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166614.261374 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069550.83 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 106955.08 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273569.344584 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962595.75 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 96259.57 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369828.919472 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866336.17 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 86633.62 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456462.536872 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779702.56 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 77970.26 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534432.792531 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701732.30 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 70173.23 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604606.022625 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631559.07 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 63155.91 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667761.929709 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568403.16 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 56840.32 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724602.246085 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511562.85 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 51156.28 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775758.530823 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460406.56 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 46040.66 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821799.187088 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414365.91 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 41436.59 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863235.777726 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372929.32 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 37292.93 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900528.709300 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335636.38 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 33563.64 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934092.347717 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302072.75 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 30207.27 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964299.622292 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271865.47 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 27186.55 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991486.169410 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244678.92 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 24467.89 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015954.061816 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220211.03 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 22021.10 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037975.164981 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198189.93 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 19818.99 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057794.157830 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178370.94 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 17837.09 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075631.251393 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160533.84 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 16053.38 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091684.635601 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144480.46 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 14448.05 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106132.681388 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130032.41 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 13003.24 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119135.922596 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117029.17 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 11702.92 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130838.839683 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105326.25 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 10532.63 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141371.465061 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94793.63 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 9479.36 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150850.827902 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85314.27 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 8531.43 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159382.254459 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76782.84 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 7678.28 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167060.538360 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69104.56 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 6910.46 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173970.993871 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62194.10 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 6219.41 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180190.403830 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55974.69 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 5597.47 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185787.872794 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50377.22 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 5037.72 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190825.594861 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45339.50 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 4533.95 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195359.544722 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40805.55 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 4080.55 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199440.099597 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36724.99 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3672.50 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203112.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33052.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206712.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29452.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210312.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25852.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213912.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22252.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217512.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18652.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221112.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15052.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224712.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11452.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228312.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7852.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231912.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4252.49 sec) 12:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235512.598984 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 652.49 sec) 12:00:00 [GPUGRID] [rr_sim] 2236165.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983418195.92G/439.78G) 12:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852406.01 12:00:00 [---] pick_jobs_to_run() start 12:00:00 [---] pick_jobs_to_run() end 12:00:00 [---] [rr_sim] end 12:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:00:00 [---] [cpu_sched_debug] preliminary job list: 12:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:00:00 [---] [cpu_sched_debug] final job list: 12:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:00:00 [---] [cpu_sched_debug] enforce_run_list: end 12:00:00 [---] choose_project(): 43200.000000 12:00:00 [iThena.Measurements] doing RPC to get work 12:00:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565199.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392399.09 cpu 2138612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138612.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:00:00 [---] Got 0 tasks 12:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:01:00 [---] [rr_sim] doing sim: CPU sched 12:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983390605.56G/439.78G) 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236102.36 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 223610.24 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223610.235659 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012492.12 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 201249.21 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424859.447751 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811242.91 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 181124.29 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605983.738635 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630118.62 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 163011.86 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768995.600430 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467106.76 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 146710.68 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915706.276046 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320396.08 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 132039.61 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047745.884100 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188356.47 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 118835.65 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166581.531348 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069520.83 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 106952.08 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273533.613872 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962568.74 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 96256.87 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369790.488143 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866311.87 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 86631.19 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456421.674988 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779680.68 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 77968.07 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534389.743148 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701712.61 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 70171.26 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604561.004491 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631541.35 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 63154.14 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667715.139701 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568387.22 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 56838.72 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724553.861389 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511548.50 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 51154.85 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775708.710909 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460393.65 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 46039.36 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821748.075477 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414354.28 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 41435.43 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863183.503588 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372918.85 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 37291.89 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900475.388887 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335626.97 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 33562.70 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934038.085657 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302064.27 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 30206.43 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964244.512750 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271857.84 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 27185.78 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991430.297134 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244672.06 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 24467.21 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015897.503079 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220204.85 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 22020.49 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037917.988430 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198184.37 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 19818.44 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057736.425245 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178365.93 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 17836.59 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075573.018379 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160529.34 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 16052.93 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091625.952200 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144476.40 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 14447.64 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106073.592639 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130028.76 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 13002.88 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119076.469033 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117025.89 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 11702.59 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130779.057789 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105323.30 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 10532.33 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141311.387668 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94790.97 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 9479.10 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150790.484560 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85311.87 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 8531.19 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159321.671763 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76780.68 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 7678.07 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166999.740245 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69102.62 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 6910.26 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173910.001879 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62192.35 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 6219.24 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180129.237350 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55973.12 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 5597.31 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185726.549273 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50375.81 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 5037.58 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190764.130005 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45338.23 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 4533.82 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195297.952663 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40804.40 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 4080.44 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199378.393055 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36723.96 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3672.40 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203050.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33051.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206650.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29451.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210250.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25851.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213850.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22251.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217450.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18651.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221050.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15051.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224650.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11451.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228250.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7851.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231850.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4251.57 sec) 12:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235450.789408 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 651.57 sec) 12:01:00 [GPUGRID] [rr_sim] 2236102.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983390605.56G/439.78G) 12:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852403.27 12:01:00 [---] pick_jobs_to_run() start 12:01:00 [---] pick_jobs_to_run() end 12:01:00 [---] [rr_sim] end 12:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:01:00 [---] [cpu_sched_debug] preliminary job list: 12:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:01:00 [---] [cpu_sched_debug] final job list: 12:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:01:00 [---] [cpu_sched_debug] enforce_run_list: end 12:01:00 [---] choose_project(): 43260.000000 12:01:00 [iThena.Measurements] doing RPC to get work 12:01:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565139.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392339.09 cpu 2138552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138552.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:01:00 [---] Got 0 tasks 12:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:02:00 [---] [rr_sim] doing sim: CPU sched 12:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983363015.20G/439.78G) 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2236039.62 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 223603.96 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223603.961970 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012435.66 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 201243.57 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424847.527744 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811192.09 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 181119.21 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605966.736940 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630072.88 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 163007.29 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768974.025216 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467065.59 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 146706.56 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915680.584665 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320359.04 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 132035.90 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047716.488169 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188323.13 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 118832.31 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166548.801322 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069490.82 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 106949.08 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273497.883160 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962541.74 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 96254.17 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369752.056815 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866287.56 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 86628.76 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456380.813104 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779658.81 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 77965.88 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534346.693764 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701692.93 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 70169.29 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604515.986358 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631523.63 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 63152.36 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667668.349692 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568371.27 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 56837.13 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724505.476693 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511534.14 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 51153.41 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775658.890995 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460380.73 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 46038.07 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821696.963865 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414342.66 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 41434.27 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863131.229449 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372908.39 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 37290.84 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900422.068475 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335617.55 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 33561.76 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933983.823598 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302055.80 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 30205.58 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964189.403208 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271850.22 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 27185.02 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991374.424858 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244665.19 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 24466.52 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015840.944342 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220198.68 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 22019.87 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037860.811879 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198178.81 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 19817.88 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057678.692661 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178360.93 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 17836.09 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075514.785365 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160524.83 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 16052.48 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091567.268799 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144472.35 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 14447.24 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106014.503890 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130025.12 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 13002.51 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119017.015471 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117022.60 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 11702.26 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130719.275894 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105320.34 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 10532.03 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141251.310275 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94788.31 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 9478.83 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150730.141218 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85309.48 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 8530.95 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159261.089067 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76778.53 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 7677.85 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166938.942130 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69100.68 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 6910.07 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173849.009888 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62190.61 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 6219.06 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180068.070869 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55971.55 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 5597.15 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185665.225753 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50374.39 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 5037.44 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190702.665148 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45336.95 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 4533.70 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195236.360604 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40803.26 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 4080.33 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199316.686514 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36722.93 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3672.29 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202988.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33050.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206588.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29450.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210188.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25850.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213788.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22250.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217388.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18650.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220988.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15050.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224588.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11450.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228188.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7850.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231788.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4250.64 sec) 12:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235388.979833 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 650.64 sec) 12:02:00 [GPUGRID] [rr_sim] 2236039.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983363015.20G/439.78G) 12:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852400.53 12:02:00 [---] pick_jobs_to_run() start 12:02:00 [---] pick_jobs_to_run() end 12:02:00 [---] [rr_sim] end 12:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:02:00 [---] [cpu_sched_debug] preliminary job list: 12:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:02:00 [---] [cpu_sched_debug] final job list: 12:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:02:00 [---] [cpu_sched_debug] enforce_run_list: end 12:02:00 [---] choose_project(): 43320.000000 12:02:00 [iThena.Measurements] doing RPC to get work 12:02:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565079.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392279.09 cpu 2138492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138492.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:02:00 [---] Got 0 tasks 12:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:03:00 [---] [rr_sim] doing sim: CPU sched 12:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983335424.85G/439.78G) 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235976.88 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 223597.69 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223597.688282 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012379.19 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 201237.92 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424835.607736 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811141.28 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 181114.13 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605949.735245 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1630027.15 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 163002.71 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768952.450002 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1467024.43 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 146702.44 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915654.893284 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320321.99 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 132032.20 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047687.092238 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188289.79 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 118828.98 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166516.071296 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069460.81 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 106946.08 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273462.152449 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962514.73 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 96251.47 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369713.625486 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866263.26 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 86626.33 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456339.951220 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779636.93 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 77963.69 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534303.644380 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701673.24 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 70167.32 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604470.968224 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631505.91 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 63150.59 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667621.559684 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568355.32 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 56835.53 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724457.091998 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511519.79 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 51151.98 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775609.071080 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460367.81 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 46036.78 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821645.852254 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414331.03 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 41433.10 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863078.955311 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372897.93 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 37289.79 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900368.748062 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335608.13 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 33560.81 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933929.561538 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302047.32 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 30204.73 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964134.293666 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271842.59 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 27184.26 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991318.552582 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244658.33 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 24465.83 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015784.385606 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220192.50 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 22019.25 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037803.635327 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198173.25 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 19817.32 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057620.960077 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178355.92 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 17835.59 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075456.552351 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160520.33 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 16052.03 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091508.585398 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144468.30 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 14446.83 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105955.415141 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130021.47 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 13002.15 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118957.561909 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117019.32 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 11701.93 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130659.494000 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105317.39 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 10531.74 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141191.232882 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94785.65 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 9478.56 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150669.797876 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85307.08 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 8530.71 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159200.506371 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76776.38 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 7677.64 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166878.144016 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69098.74 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 6909.87 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173788.017896 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62188.86 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 6218.89 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180006.904389 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55969.98 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 5597.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185603.902232 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50372.98 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 5037.30 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190641.200291 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45335.68 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 4533.57 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195174.768544 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40802.11 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 4080.21 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199254.979972 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36721.90 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3672.19 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202927.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33049.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206527.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29449.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210127.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25849.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213727.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22249.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217327.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18649.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220927.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15049.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224527.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11449.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228127.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7849.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231727.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4249.71 sec) 12:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235327.170257 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 649.71 sec) 12:03:00 [GPUGRID] [rr_sim] 2235976.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983335424.85G/439.78G) 12:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852397.80 12:03:00 [---] pick_jobs_to_run() start 12:03:00 [---] pick_jobs_to_run() end 12:03:00 [---] [rr_sim] end 12:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:03:00 [---] [cpu_sched_debug] preliminary job list: 12:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:03:00 [---] [cpu_sched_debug] final job list: 12:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:03:00 [---] [cpu_sched_debug] enforce_run_list: end 12:03:00 [---] choose_project(): 43380.000000 12:03:00 [iThena.Measurements] doing RPC to get work 12:03:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 565019.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392219.09 cpu 2138432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138432.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:03:00 [---] Got 0 tasks 12:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:04:00 [---] [rr_sim] doing sim: CPU sched 12:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983307834.49G/439.78G) 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235914.15 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 223591.41 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223591.414594 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012322.73 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 201232.27 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424823.687729 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811090.46 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 181109.05 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605932.733550 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629981.41 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 162998.14 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768930.874789 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466983.27 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 146698.33 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915629.201904 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320284.94 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 132028.49 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047657.696307 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188256.45 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 118825.64 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166483.341270 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069430.80 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 106943.08 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273426.421737 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962487.72 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 96248.77 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369675.194158 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866238.95 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 86623.90 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456299.089336 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779615.06 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 77961.51 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534260.594996 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701653.55 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 70165.36 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604425.950091 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631488.20 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 63148.82 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667574.769675 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568339.38 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 56833.94 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724408.707302 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511505.44 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 51150.54 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775559.251166 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460354.89 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 46035.49 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821594.740643 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414319.41 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 41431.94 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863026.681173 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372887.46 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 37288.75 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900315.427649 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335598.72 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 33559.87 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933875.299478 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302038.85 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 30203.88 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964079.184124 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271834.96 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 27183.50 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991262.680306 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244651.47 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 24465.15 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015727.826869 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220186.32 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 22018.63 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037746.458776 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198167.69 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 19816.77 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057563.227493 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178350.92 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 17835.09 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075398.319337 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160515.83 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 16051.58 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091449.901998 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144464.24 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 14446.42 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105896.326392 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130017.82 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 13001.78 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118898.108347 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117016.04 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 11701.60 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130599.712106 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105314.43 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 10531.44 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141131.155489 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94782.99 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 9478.30 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150609.454534 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85304.69 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 8530.47 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159139.923675 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76774.22 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 7677.42 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166817.345901 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69096.80 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 6909.68 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173727.025905 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62187.12 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 6218.71 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179945.737909 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55968.41 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 5596.84 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185542.578712 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50371.57 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 5037.16 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190579.735434 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45334.41 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 4533.44 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195113.176485 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40800.97 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 4080.10 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199193.273430 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36720.87 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3672.09 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202865.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33048.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206465.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29448.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210065.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25848.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213665.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22248.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217265.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18648.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220865.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15048.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224465.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11448.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228065.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7848.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231665.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4248.79 sec) 12:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235265.360681 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 648.79 sec) 12:04:00 [GPUGRID] [rr_sim] 2235914.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983307834.49G/439.78G) 12:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852395.06 12:04:00 [---] pick_jobs_to_run() start 12:04:00 [---] pick_jobs_to_run() end 12:04:00 [---] [rr_sim] end 12:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:04:00 [---] [cpu_sched_debug] preliminary job list: 12:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:04:00 [---] [cpu_sched_debug] final job list: 12:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:04:00 [---] [cpu_sched_debug] enforce_run_list: end 12:04:00 [---] choose_project(): 43440.000000 12:04:00 [iThena.Measurements] doing RPC to get work 12:04:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564959.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392159.09 cpu 2138372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138372.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:04:00 [---] Got 0 tasks 12:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:05:00 [---] [rr_sim] doing sim: CPU sched 12:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983280244.14G/439.78G) 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235851.41 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 223585.14 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223585.140906 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012266.27 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 201226.63 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424811.767721 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1811039.64 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 181103.96 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605915.731855 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629935.68 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 162993.57 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768909.299575 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466942.11 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 146694.21 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915603.510523 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320247.90 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 132024.79 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047628.300377 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188223.11 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 118822.31 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166450.611245 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069400.80 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 106940.08 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273390.691026 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962460.72 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 96246.07 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369636.762829 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866214.65 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 86621.46 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456258.227452 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779593.18 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 77959.32 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534217.545612 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701633.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 70163.39 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604380.931957 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631470.48 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 63147.05 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667527.979667 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568323.43 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 56832.34 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724360.322606 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511491.09 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 51149.11 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775509.431251 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460341.98 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 46034.20 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821543.629032 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414307.78 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 41430.78 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862974.407034 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372877.00 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 37287.70 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900262.107237 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335589.30 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 33558.93 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933821.037419 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302030.37 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 30203.04 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964024.074583 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271827.33 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 27182.73 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991206.808030 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244644.60 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 24464.46 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015671.268133 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220180.14 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 22018.01 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037689.282225 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198162.13 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 19816.21 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057505.494908 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178345.91 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 17834.59 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075340.086323 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160511.32 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 16051.13 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091391.218597 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144460.19 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 14446.02 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105837.237643 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130014.17 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 13001.42 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118838.654784 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117012.75 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 11701.28 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130539.930212 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105311.48 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 10531.15 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141071.078096 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94780.33 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 9478.03 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150549.111192 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85302.30 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 8530.23 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159079.340979 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76772.07 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 7677.21 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166756.547787 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69094.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 6909.49 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173666.033914 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62185.38 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 6218.54 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179884.571428 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55966.84 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 5596.68 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185481.255191 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50370.15 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 5037.02 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190518.270578 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45333.14 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 4533.31 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195051.584426 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40799.82 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 4079.98 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199131.566889 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36719.84 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3671.98 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202803.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33047.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206403.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29447.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210003.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25847.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213603.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22247.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217203.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18647.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220803.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15047.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224403.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11447.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228003.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7847.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231603.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4247.86 sec) 12:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235203.551106 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 647.86 sec) 12:05:00 [GPUGRID] [rr_sim] 2235851.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983280244.14G/439.78G) 12:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852392.32 12:05:00 [---] pick_jobs_to_run() start 12:05:00 [---] pick_jobs_to_run() end 12:05:00 [---] [rr_sim] end 12:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:05:00 [---] [cpu_sched_debug] preliminary job list: 12:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:05:00 [---] [cpu_sched_debug] final job list: 12:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:05:00 [---] [cpu_sched_debug] enforce_run_list: end 12:05:00 [---] choose_project(): 43500.000000 12:05:00 [iThena.Measurements] doing RPC to get work 12:05:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564899.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392099.09 cpu 2138312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138312.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:05:00 [---] Got 0 tasks 12:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:06:00 [---] [rr_sim] doing sim: CPU sched 12:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983252653.78G/439.78G) 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235788.67 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 223578.87 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223578.867218 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012209.80 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 201220.98 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424799.847713 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810988.82 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 181098.88 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605898.730159 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629889.94 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 162988.99 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768887.724361 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466900.95 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 146690.09 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915577.819142 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320210.85 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 132021.09 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047598.904446 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188189.77 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 118818.98 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166417.881219 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069370.79 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 106937.08 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273354.960314 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962433.71 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 96243.37 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369598.331500 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866190.34 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 86619.03 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456217.365568 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779571.31 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 77957.13 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534174.496229 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701614.18 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 70161.42 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604335.913823 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631452.76 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 63145.28 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667481.189658 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568307.48 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 56830.75 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724311.937910 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511476.73 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 51147.67 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775459.611337 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460329.06 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 46032.91 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821492.517421 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414296.15 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 41429.62 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862922.132896 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372866.54 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 37286.65 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900208.786824 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335579.89 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 33557.99 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933766.775359 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302021.90 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 30202.19 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963968.965041 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271819.71 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 27181.97 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991150.935754 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244637.74 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 24463.77 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015614.709396 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220173.96 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 22017.40 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037632.105674 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198156.57 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 19815.66 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057447.762324 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178340.91 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 17834.09 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075281.853309 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160506.82 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 16050.68 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091332.535196 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144456.14 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 14445.61 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105778.148894 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130010.52 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 13001.05 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118779.201222 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117009.47 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 11700.95 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130480.148317 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105308.52 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 10530.85 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141011.000703 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94777.67 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 9477.77 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150488.767850 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85299.90 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 8529.99 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159018.758283 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76769.91 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 7676.99 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166695.749672 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69092.92 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 6909.29 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173605.041922 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62183.63 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 6218.36 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179823.404948 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55965.27 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 5596.53 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185419.931670 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50368.74 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 5036.87 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190456.805721 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45331.87 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 4533.19 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194989.992366 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40798.68 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 4079.87 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199069.860347 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36718.81 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3671.88 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202741.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33046.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206341.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29446.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209941.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25846.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213541.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22246.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217141.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18646.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220741.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15046.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224341.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11446.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227941.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7846.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231541.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4246.93 sec) 12:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235141.741530 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 646.93 sec) 12:06:00 [GPUGRID] [rr_sim] 2235788.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983252653.78G/439.78G) 12:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852389.58 12:06:00 [---] pick_jobs_to_run() start 12:06:00 [---] pick_jobs_to_run() end 12:06:00 [---] [rr_sim] end 12:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:06:00 [---] [cpu_sched_debug] preliminary job list: 12:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:06:00 [---] [cpu_sched_debug] final job list: 12:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:06:00 [---] [cpu_sched_debug] enforce_run_list: end 12:06:00 [---] choose_project(): 43560.000000 12:06:00 [iThena.Measurements] doing RPC to get work 12:06:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564839.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 392039.09 cpu 2138252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138252.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:06:00 [---] Got 0 tasks 12:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:07:00 [---] [rr_sim] doing sim: CPU sched 12:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983225063.42G/439.78G) 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235725.94 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 223572.59 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223572.593529 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012153.34 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 201215.33 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424787.927706 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810938.01 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 181093.80 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605881.728464 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629844.21 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 162984.42 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768866.149147 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466859.79 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 146685.98 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915552.127762 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320173.81 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 132017.38 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047569.508515 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188156.43 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 118815.64 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166385.151193 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069340.78 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 106934.08 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273319.229603 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962406.71 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 96240.67 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369559.900172 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866166.04 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 86616.60 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456176.503684 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779549.43 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 77954.94 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534131.446845 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701594.49 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 70159.45 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604290.895690 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631435.04 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 63143.50 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667434.399650 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568291.54 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 56829.15 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724263.553214 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511462.38 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 51146.24 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775409.791422 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460316.14 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 46031.61 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821441.405809 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414284.53 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 41428.45 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862869.858758 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372856.08 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 37285.61 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900155.466411 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335570.47 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 33557.05 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933712.513299 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302013.42 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 30201.34 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963913.855499 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271812.08 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 27181.21 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991095.063478 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244630.87 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 24463.09 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015558.150660 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220167.78 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 22016.78 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037574.929123 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198151.01 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 19815.10 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057390.029740 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178335.91 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 17833.59 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075223.620295 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160502.31 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 16050.23 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091273.851795 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144452.08 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 14445.21 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105719.060145 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130006.88 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 13000.69 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118719.747660 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117006.19 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 11700.62 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130420.366423 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105305.57 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 10530.56 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140950.923310 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94775.01 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 9477.50 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150428.424508 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85297.51 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 8529.75 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158958.175587 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76767.76 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 7676.78 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166634.951557 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69090.98 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 6909.10 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173544.049931 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62181.89 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 6218.19 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179762.238467 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55963.70 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 5596.37 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185358.608150 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50367.33 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 5036.73 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190395.340864 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45330.59 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 4533.06 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194928.400307 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40797.53 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 4079.75 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199008.153806 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36717.78 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3671.78 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202679.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33046.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206279.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29446.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209879.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25846.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213479.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22246.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217079.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18646.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220679.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15046.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224279.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11446.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227879.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7846.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231479.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4246.00 sec) 12:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235079.931954 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 646.00 sec) 12:07:00 [GPUGRID] [rr_sim] 2235725.94: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983225063.42G/439.78G) 12:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852386.85 12:07:00 [---] pick_jobs_to_run() start 12:07:00 [---] pick_jobs_to_run() end 12:07:00 [---] [rr_sim] end 12:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:07:00 [---] [cpu_sched_debug] preliminary job list: 12:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:07:00 [---] [cpu_sched_debug] final job list: 12:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:07:00 [---] [cpu_sched_debug] enforce_run_list: end 12:07:00 [---] choose_project(): 43620.000000 12:07:00 [iThena.Measurements] doing RPC to get work 12:07:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564779.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391979.09 cpu 2138192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138192.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:07:00 [---] Got 0 tasks 12:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:08:00 [---] [rr_sim] doing sim: CPU sched 12:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983197473.07G/439.78G) 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235663.20 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 223566.32 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223566.319841 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012096.88 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 201209.69 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424776.007698 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810887.19 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 181088.72 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605864.726769 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629798.47 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 162979.85 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768844.573933 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466818.62 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 146681.86 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915526.436381 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320136.76 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 132013.68 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047540.112584 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188123.09 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 118812.31 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166352.421167 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069310.78 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 106931.08 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273283.498891 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962379.70 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 96237.97 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369521.468843 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866141.73 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 86614.17 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456135.641800 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779527.56 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 77952.76 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534088.397461 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701574.80 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 70157.48 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604245.877556 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631417.32 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 63141.73 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667387.609642 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568275.59 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 56827.56 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724215.168518 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511448.03 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 51144.80 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775359.971508 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460303.23 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 46030.32 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821390.294198 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414272.90 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 41427.29 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862817.584619 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372845.61 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 37284.56 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900102.145998 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335561.05 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 33556.11 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933658.251240 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302004.95 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 30200.49 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963858.745957 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271804.45 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 27180.45 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991039.191202 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244624.01 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 24462.40 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015501.591923 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220161.61 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 22016.16 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037517.752572 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198145.45 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 19814.54 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057332.297156 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178330.90 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 17833.09 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075165.387281 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160497.81 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 16049.78 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091215.168394 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144448.03 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 14444.80 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105659.971396 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130003.23 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 13000.32 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118660.294097 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117002.90 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 11700.29 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130360.584529 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105302.61 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 10530.26 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140890.845917 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94772.35 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 9477.24 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150368.081166 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85295.12 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 8529.51 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158897.592891 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76765.61 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 7676.56 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166574.153443 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69089.04 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 6908.90 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173483.057940 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62180.14 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 6218.01 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179701.071987 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55962.13 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 5596.21 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185297.284629 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50365.91 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 5036.59 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190333.876007 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45329.32 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 4532.93 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194866.808248 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40796.39 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 4079.64 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198946.447264 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36716.75 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3671.68 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202618.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33045.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206218.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29445.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209818.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25845.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213418.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22245.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217018.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18645.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220618.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15045.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224218.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11445.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227818.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7845.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231418.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4245.08 sec) 12:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2235018.122379 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 645.08 sec) 12:08:00 [GPUGRID] [rr_sim] 2235663.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983197473.07G/439.78G) 12:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852384.11 12:08:00 [---] pick_jobs_to_run() start 12:08:00 [---] pick_jobs_to_run() end 12:08:00 [---] [rr_sim] end 12:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:08:00 [---] [cpu_sched_debug] preliminary job list: 12:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:08:00 [---] [cpu_sched_debug] final job list: 12:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:08:00 [---] [cpu_sched_debug] enforce_run_list: end 12:08:00 [---] choose_project(): 43680.000000 12:08:00 [iThena.Measurements] doing RPC to get work 12:08:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564719.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391919.09 cpu 2138132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138132.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:08:00 [---] Got 0 tasks 12:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:09:00 [---] [rr_sim] doing sim: CPU sched 12:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983169882.71G/439.78G) 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235600.46 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 223560.05 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223560.046153 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2012040.42 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 201204.04 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424764.087690 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810836.37 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 181083.64 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605847.725074 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629752.74 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 162975.27 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768822.998720 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466777.46 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 146677.75 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915500.745001 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320099.72 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 132009.97 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047510.716653 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188089.74 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 118808.97 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166319.691141 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069280.77 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 106928.08 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273247.768180 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962352.69 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 96235.27 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369483.037515 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866117.42 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 86611.74 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456094.779916 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779505.68 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 77950.57 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534045.348077 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701555.11 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 70155.51 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604200.859422 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631399.60 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 63139.96 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667340.819633 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568259.64 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 56825.96 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724166.783823 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511433.68 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 51143.37 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775310.151593 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460290.31 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 46029.03 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821339.182587 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414261.28 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 41426.13 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862765.310481 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372835.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 37283.52 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900048.825586 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335551.64 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 33555.16 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933603.989180 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301996.47 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 30199.65 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963803.636415 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271796.83 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 27179.68 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990983.318926 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244617.14 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 24461.71 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015445.033187 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220155.43 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 22015.54 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037460.576021 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198139.89 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 19813.99 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057274.564572 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178325.90 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 17832.59 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075107.154267 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160493.31 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 16049.33 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091156.484993 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144443.98 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 14444.40 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105600.882647 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129999.58 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 12999.96 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118600.840535 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116999.62 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 11699.96 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130300.802634 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105299.66 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 10529.97 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140830.768524 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94769.69 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 9476.97 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150307.737824 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85292.72 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 8529.27 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158837.010195 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76763.45 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 7676.35 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166513.355328 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69087.11 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 6908.71 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173422.065948 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62178.40 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 6217.84 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179639.905506 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55960.56 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 5596.06 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185235.961109 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50364.50 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 5036.45 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190272.411151 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45328.05 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 4532.81 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194805.216188 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40795.25 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 4079.52 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198884.740722 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36715.72 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3671.57 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202556.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33044.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206156.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29444.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209756.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25844.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213356.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22244.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216956.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18644.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220556.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15044.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224156.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11444.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227756.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7844.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231356.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4244.15 sec) 12:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234956.312803 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 644.15 sec) 12:09:00 [GPUGRID] [rr_sim] 2235600.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983169882.71G/439.78G) 12:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852381.37 12:09:00 [---] pick_jobs_to_run() start 12:09:00 [---] pick_jobs_to_run() end 12:09:00 [---] [rr_sim] end 12:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:09:00 [---] [cpu_sched_debug] preliminary job list: 12:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:09:00 [---] [cpu_sched_debug] final job list: 12:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:09:00 [---] [cpu_sched_debug] enforce_run_list: end 12:09:00 [---] choose_project(): 43740.000000 12:09:00 [iThena.Measurements] doing RPC to get work 12:09:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564659.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391859.09 cpu 2138072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138072.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:09:00 [---] Got 0 tasks 12:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:10:00 [---] [rr_sim] doing sim: CPU sched 12:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983142292.35G/439.78G) 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235537.72 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 223553.77 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223553.772465 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011983.95 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 201198.40 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424752.167683 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810785.56 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 181078.56 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605830.723379 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629707.00 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 162970.70 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768801.423506 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466736.30 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 146673.63 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915475.053620 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320062.67 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 132006.27 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047481.320723 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188056.40 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 118805.64 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166286.961115 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069250.76 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 106925.08 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273212.037468 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962325.69 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 96232.57 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369444.606186 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866093.12 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 86609.31 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456053.918032 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779483.81 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 77948.38 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1534002.298694 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701535.43 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 70153.54 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604155.841289 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631381.88 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 63138.19 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667294.029625 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568243.70 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 56824.37 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724118.399127 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511419.33 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 51141.93 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775260.331679 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460277.39 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 46027.74 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821288.070976 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414249.65 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 41424.97 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862713.036343 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372824.69 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 37282.47 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899995.505173 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335542.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 33554.22 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933549.727120 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301988.00 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 30198.80 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963748.526873 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271789.20 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 27178.92 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990927.446650 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244610.28 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 24461.03 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015388.474450 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220149.25 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 22014.93 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037403.399470 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198134.33 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 19813.43 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057216.831987 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178320.89 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 17832.09 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075048.921253 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160488.80 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 16048.88 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091097.801593 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144439.92 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 14443.99 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105541.793898 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129995.93 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 12999.59 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118541.386973 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116996.34 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 11699.63 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130241.020740 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105296.70 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 10529.67 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140770.691131 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94767.03 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 9476.70 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150247.394482 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85290.33 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 8529.03 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158776.427499 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76761.30 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 7676.13 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166452.557214 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69085.17 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 6908.52 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173361.073957 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62176.65 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 6217.67 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179578.739026 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55958.99 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 5595.90 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185174.637588 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50363.09 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 5036.31 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190210.946294 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45326.78 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 4532.68 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194743.624129 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40794.10 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 4079.41 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198823.034181 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36714.69 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3671.47 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202494.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33043.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206094.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29443.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209694.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25843.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213294.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22243.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216894.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18643.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220494.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15043.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224094.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11443.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227694.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7843.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231294.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4243.22 sec) 12:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234894.503227 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 643.22 sec) 12:10:00 [GPUGRID] [rr_sim] 2235537.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983142292.35G/439.78G) 12:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852378.64 12:10:00 [---] pick_jobs_to_run() start 12:10:00 [---] pick_jobs_to_run() end 12:10:00 [---] [rr_sim] end 12:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:10:00 [---] [cpu_sched_debug] preliminary job list: 12:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:10:00 [---] [cpu_sched_debug] final job list: 12:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:10:00 [---] [cpu_sched_debug] enforce_run_list: end 12:10:00 [---] choose_project(): 43800.000000 12:10:00 [iThena.Measurements] doing RPC to get work 12:10:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564599.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391799.09 cpu 2138012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2138012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2138012.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:10:00 [---] Got 0 tasks 12:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:11:00 [---] [rr_sim] doing sim: CPU sched 12:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983114702.00G/439.78G) 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235474.99 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 223547.50 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223547.498776 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011927.49 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 201192.75 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424740.247675 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810734.74 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 181073.47 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605813.721684 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629661.27 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 162966.13 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768779.848292 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466695.14 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 146669.51 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915449.362239 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1320025.63 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 132002.56 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047451.924792 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1188023.06 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 118802.31 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166254.231089 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069220.76 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 106922.08 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273176.306757 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962298.68 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 96229.87 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369406.174857 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866068.81 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 86606.88 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1456013.056148 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779461.93 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 77946.19 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533959.249310 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701515.74 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 70151.57 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604110.823155 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631364.16 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 63136.42 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667247.239616 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568227.75 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 56822.77 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724070.014431 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511404.97 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 51140.50 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775210.511764 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460264.48 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 46026.45 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821236.959364 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414238.03 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 41423.80 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862660.762204 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372814.23 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 37281.42 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899942.184760 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335532.80 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 33553.28 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933495.465061 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301979.52 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 30197.95 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963693.417331 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271781.57 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 27178.16 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990871.574374 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244603.41 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 24460.34 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015331.915713 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220143.07 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 22014.31 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037346.222919 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198128.76 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 19812.88 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057159.099403 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178315.89 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 17831.59 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074990.688239 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160484.30 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 16048.43 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091039.118192 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144435.87 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 14443.59 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105482.705149 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129992.28 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 12999.23 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118481.933411 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116993.05 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 11699.31 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130181.238846 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105293.75 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 10529.37 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140710.613738 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94764.37 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 9476.44 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150187.051140 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85287.94 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 8528.79 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158715.844803 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76759.14 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 7675.91 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166391.759099 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69083.23 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 6908.32 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173300.081966 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62174.91 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 6217.49 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179517.572545 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55957.42 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 5595.74 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185113.314067 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50361.67 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 5036.17 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190149.481437 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45325.51 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 4532.55 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194682.032070 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40792.96 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 4079.30 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198761.327639 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36713.66 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3671.37 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202432.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33042.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206032.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29442.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209632.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25842.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213232.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22242.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216832.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18642.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220432.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15042.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224032.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11442.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227632.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7842.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231232.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4242.29 sec) 12:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234832.693652 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 642.29 sec) 12:11:00 [GPUGRID] [rr_sim] 2235474.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983114702.00G/439.78G) 12:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852375.90 12:11:00 [---] pick_jobs_to_run() start 12:11:00 [---] pick_jobs_to_run() end 12:11:00 [---] [rr_sim] end 12:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:11:00 [---] [cpu_sched_debug] preliminary job list: 12:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:11:00 [---] [cpu_sched_debug] final job list: 12:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:11:00 [---] [cpu_sched_debug] enforce_run_list: end 12:11:00 [---] choose_project(): 43860.000000 12:11:00 [iThena.Measurements] doing RPC to get work 12:11:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564539.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391739.09 cpu 2137952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137952.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:11:00 [---] Got 0 tasks 12:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:12:00 [---] [rr_sim] doing sim: CPU sched 12:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983087111.64G/439.78G) 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235412.25 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 223541.23 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223541.225088 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011871.03 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 201187.10 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424728.327668 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810683.92 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 181068.39 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605796.719989 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629615.53 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 162961.55 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768758.273078 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466653.98 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 146665.40 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915423.670859 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319988.58 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 131998.86 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047422.528861 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187989.72 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 118798.97 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166221.501063 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069190.75 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 106919.07 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273140.576045 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962271.67 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 96227.17 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369367.743529 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866044.51 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 86604.45 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455972.194264 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779440.06 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 77944.01 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533916.199926 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701496.05 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 70149.61 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604065.805022 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631346.45 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 63134.64 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667200.449608 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568211.80 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 56821.18 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1724021.629735 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511390.62 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 51139.06 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775160.691850 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460251.56 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 46025.16 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821185.847753 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414226.40 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 41422.64 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862608.488066 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372803.76 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 37280.38 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899888.864348 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335523.39 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 33552.34 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933441.203001 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301971.05 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 30197.10 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963638.307789 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271773.94 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 27177.39 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990815.702098 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244596.55 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 24459.65 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015275.356977 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220136.89 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 22013.69 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037289.046367 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198123.20 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 19812.32 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057101.366819 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178310.88 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 17831.09 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074932.455225 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160479.80 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 16047.98 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090980.434791 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144431.82 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 14443.18 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105423.616400 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129988.63 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 12998.86 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118422.479848 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116989.77 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 11698.98 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130121.456952 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105290.79 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 10529.08 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140650.536345 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94761.71 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 9476.17 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150126.707798 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85285.54 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 8528.55 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158655.262107 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76756.99 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 7675.70 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166330.960984 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69081.29 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 6908.13 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173239.089974 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62173.16 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 6217.32 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179456.406065 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55955.84 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 5595.58 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185051.990547 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50360.26 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 5036.03 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190088.016580 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45324.23 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 4532.42 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194620.440010 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40791.81 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 4079.18 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198699.621098 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36712.63 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3671.26 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202370.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33041.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205970.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29441.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209570.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25841.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213170.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22241.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216770.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18641.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220370.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15041.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223970.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11441.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227570.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7841.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231170.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4241.37 sec) 12:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234770.884076 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 641.37 sec) 12:12:00 [GPUGRID] [rr_sim] 2235412.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983087111.64G/439.78G) 12:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852373.16 12:12:00 [---] pick_jobs_to_run() start 12:12:00 [---] pick_jobs_to_run() end 12:12:00 [---] [rr_sim] end 12:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:12:00 [---] [cpu_sched_debug] preliminary job list: 12:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:12:00 [---] [cpu_sched_debug] final job list: 12:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:12:00 [---] [cpu_sched_debug] enforce_run_list: end 12:12:00 [---] choose_project(): 43920.000000 12:12:00 [iThena.Measurements] doing RPC to get work 12:12:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564479.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391679.09 cpu 2137892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137892.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:12:00 [---] Got 0 tasks 12:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:13:00 [---] [rr_sim] doing sim: CPU sched 12:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983059521.29G/439.78G) 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235349.51 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 223534.95 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223534.951400 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011814.56 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 201181.46 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424716.407660 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810633.11 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 181063.31 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605779.718294 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629569.80 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 162956.98 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768736.697865 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466612.82 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 146661.28 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915397.979478 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319951.53 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 131995.15 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047393.132930 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187956.38 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 118795.64 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166188.771037 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069160.74 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 106916.07 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273104.845334 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962244.67 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 96224.47 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369329.312200 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 866020.20 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 86602.02 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455931.332380 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779418.18 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 77941.82 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533873.150542 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701476.36 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 70147.64 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1604020.786888 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631328.73 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 63132.87 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667153.659599 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568195.85 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 56819.59 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723973.245039 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511376.27 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 51137.63 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775110.871935 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460238.64 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 46023.86 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821134.736142 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414214.78 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 41421.48 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862556.213928 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372793.30 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 37279.33 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899835.543935 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335513.97 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 33551.40 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933386.940941 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301962.57 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 30196.26 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963583.198247 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271766.32 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 27176.63 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990759.829823 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244589.68 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 24458.97 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015218.798240 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220130.72 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 22013.07 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037231.869816 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198117.64 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 19811.76 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057043.634235 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178305.88 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 17830.59 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074874.222211 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160475.29 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 16047.53 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090921.751390 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144427.76 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 14442.78 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105364.527651 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129984.99 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 12998.50 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118363.026286 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116986.49 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 11698.65 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130061.675057 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105287.84 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 10528.78 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140590.458952 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94759.06 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 9475.91 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150066.364456 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85283.15 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 8528.31 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158594.679411 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76754.83 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 7675.48 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166270.162870 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69079.35 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 6907.94 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173178.097983 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62171.42 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 6217.14 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179395.239585 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55954.27 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 5595.43 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184990.667026 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50358.85 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 5035.88 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190026.551723 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45322.96 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 4532.30 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194558.847951 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40790.67 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 4079.07 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198637.914556 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36711.60 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3671.16 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202309.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33040.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205909.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29440.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209509.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25840.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213109.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22240.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216709.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18640.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220309.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15040.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223909.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11440.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227509.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7840.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231109.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4240.44 sec) 12:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234709.074500 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 640.44 sec) 12:13:00 [GPUGRID] [rr_sim] 2235349.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983059521.29G/439.78G) 12:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852370.43 12:13:00 [---] pick_jobs_to_run() start 12:13:00 [---] pick_jobs_to_run() end 12:13:00 [---] [rr_sim] end 12:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:13:00 [---] [cpu_sched_debug] preliminary job list: 12:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:13:00 [---] [cpu_sched_debug] final job list: 12:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:13:00 [---] [cpu_sched_debug] enforce_run_list: end 12:13:00 [---] choose_project(): 43980.000000 12:13:00 [iThena.Measurements] doing RPC to get work 12:13:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564419.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391619.09 cpu 2137832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137832.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:13:00 [---] Got 0 tasks 12:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:14:00 [---] [rr_sim] doing sim: CPU sched 12:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983031930.93G/439.78G) 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235286.78 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 223528.68 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223528.677712 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011758.10 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 201175.81 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424704.487652 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810582.29 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 181058.23 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605762.716599 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629524.06 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 162952.41 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768715.122651 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466571.65 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 146657.17 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915372.288098 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319914.49 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 131991.45 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047363.737000 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187923.04 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 118792.30 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166156.041011 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069130.74 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 106913.07 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273069.114622 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962217.66 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 96221.77 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369290.880872 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865995.90 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 86599.59 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455890.470496 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779396.31 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 77939.63 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533830.101158 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701456.68 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 70145.67 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603975.768754 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631311.01 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 63131.10 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667106.869591 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568179.91 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 56817.99 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723924.860343 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511361.92 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 51136.19 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775061.052021 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460225.73 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 46022.57 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821083.624531 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414203.15 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 41420.32 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862503.939789 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372782.84 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 37278.28 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899782.223522 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335504.55 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 33550.46 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933332.678882 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301954.10 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 30195.41 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963528.088705 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271758.69 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 27175.87 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990703.957547 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244582.82 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 24458.28 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015162.239504 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220124.54 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 22012.45 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037174.693265 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198112.08 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 19811.21 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056985.901650 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178300.88 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 17830.09 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074815.989197 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160470.79 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 16047.08 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090863.067989 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144423.71 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 14442.37 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105305.438902 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129981.34 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 12998.13 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118303.572724 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116983.20 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 11698.32 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130001.893163 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105284.88 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 10528.49 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140530.381559 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94756.40 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 9475.64 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150006.021115 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85280.76 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 8528.08 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158534.096715 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76752.68 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 7675.27 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166209.364755 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69077.41 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 6907.74 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173117.105991 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62169.67 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 6216.97 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179334.073104 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55952.70 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 5595.27 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184929.343505 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50357.43 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 5035.74 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189965.086867 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45321.69 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 4532.17 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194497.255892 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40789.52 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 4078.95 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198576.208014 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36710.57 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3671.06 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202247.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33039.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205847.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29439.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209447.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25839.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213047.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22239.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216647.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18639.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220247.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15039.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223847.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11439.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227447.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7839.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231047.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4239.51 sec) 12:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234647.264925 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 639.51 sec) 12:14:00 [GPUGRID] [rr_sim] 2235286.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983031930.93G/439.78G) 12:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852367.69 12:14:00 [---] pick_jobs_to_run() start 12:14:00 [---] pick_jobs_to_run() end 12:14:00 [---] [rr_sim] end 12:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:14:00 [---] [cpu_sched_debug] preliminary job list: 12:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:14:00 [---] [cpu_sched_debug] final job list: 12:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:14:00 [---] [cpu_sched_debug] enforce_run_list: end 12:14:00 [---] choose_project(): 44040.000000 12:14:00 [iThena.Measurements] doing RPC to get work 12:14:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564359.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391559.09 cpu 2137772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137772.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:14:00 [---] Got 0 tasks 12:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:15:00 [---] [rr_sim] doing sim: CPU sched 12:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (983004340.57G/439.78G) 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235224.04 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 223522.40 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223522.404024 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011701.64 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 201170.16 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424692.567645 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810531.47 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 181053.15 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605745.714904 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629478.33 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 162947.83 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768693.547437 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466530.49 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 146653.05 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915346.596717 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319877.44 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 131987.74 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047334.341069 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187889.70 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 118788.97 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166123.310985 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069100.73 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 106910.07 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1273033.383911 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962190.66 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 96219.07 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369252.449543 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865971.59 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 86597.16 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455849.608612 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779374.43 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 77937.44 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533787.051775 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701436.99 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 70143.70 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603930.750621 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631293.29 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 63129.33 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667060.079582 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568163.96 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 56816.40 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723876.475648 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511347.56 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 51134.76 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1775011.232106 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460212.81 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 46021.28 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1821032.512919 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414191.53 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 41419.15 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862451.665651 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372772.37 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 37277.24 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899728.903109 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335495.14 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 33549.51 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933278.416822 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301945.62 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 30194.56 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963472.979163 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271751.06 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 27175.11 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990648.085271 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244575.95 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 24457.60 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015105.680767 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220118.36 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 22011.84 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037117.516714 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198106.52 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 19810.65 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056928.169066 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178295.87 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 17829.59 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074757.756183 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160466.28 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 16046.63 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090804.384588 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144419.66 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 14441.97 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105246.350153 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129977.69 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 12997.77 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118244.119161 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116979.92 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 11697.99 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129942.111269 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105281.93 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 10528.19 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140470.304166 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94753.74 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 9475.37 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149945.677773 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85278.36 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 8527.84 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158473.514019 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76750.53 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 7675.05 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166148.566641 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69075.47 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 6907.55 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173056.114000 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62167.93 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 6216.79 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179272.906624 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55951.13 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 5595.11 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184868.019985 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50356.02 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 5035.60 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189903.622010 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45320.42 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 4532.04 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194435.663832 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40788.38 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 4078.84 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198514.501473 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36709.54 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3670.95 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202185.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33038.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205785.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29438.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209385.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25838.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212985.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22238.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216585.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18638.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220185.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15038.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223785.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11438.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227385.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7838.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230985.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4238.58 sec) 12:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234585.455349 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 638.58 sec) 12:15:00 [GPUGRID] [rr_sim] 2235224.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (983004340.57G/439.78G) 12:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852364.95 12:15:00 [---] pick_jobs_to_run() start 12:15:00 [---] pick_jobs_to_run() end 12:15:00 [---] [rr_sim] end 12:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:15:00 [---] [cpu_sched_debug] preliminary job list: 12:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:15:00 [---] [cpu_sched_debug] final job list: 12:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:15:00 [---] [cpu_sched_debug] enforce_run_list: end 12:15:00 [---] choose_project(): 44100.000000 12:15:00 [iThena.Measurements] doing RPC to get work 12:15:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564299.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391499.09 cpu 2137712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137712.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:15:00 [---] Got 0 tasks 12:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:16:00 [---] [rr_sim] doing sim: CPU sched 12:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982976750.22G/439.78G) 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235161.30 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 223516.13 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223516.130335 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011645.17 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 201164.52 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424680.647637 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810480.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 181048.07 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605728.713209 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629432.59 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 162943.26 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768671.972223 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466489.33 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 146648.93 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915320.905336 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319840.40 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 131984.04 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047304.945138 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187856.36 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 118785.64 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166090.580960 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069070.72 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 106907.07 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272997.653199 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962163.65 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 96216.37 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369214.018214 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865947.29 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 86594.73 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455808.746728 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779352.56 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 77935.26 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533744.002391 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701417.30 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 70141.73 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603885.732487 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631275.57 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 63127.56 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1667013.289574 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568148.01 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 56814.80 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723828.090952 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511333.21 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 51133.32 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774961.412192 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460199.89 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 46019.99 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820981.401308 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414179.90 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 41417.99 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862399.391513 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372761.91 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 37276.19 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899675.582697 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335485.72 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 33548.57 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933224.154762 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301937.15 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 30193.71 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963417.869622 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271743.43 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 27174.34 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990592.212995 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244569.09 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 24456.91 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015049.122031 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220112.18 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 22011.22 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037060.340163 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198100.96 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 19810.10 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056870.436482 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178290.87 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 17829.09 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074699.523169 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160461.78 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 16046.18 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090745.701188 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144415.60 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 14441.56 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105187.261404 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129974.04 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 12997.40 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118184.665599 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116976.64 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 11697.66 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129882.329375 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105278.97 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 10527.90 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140410.226772 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94751.08 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 9475.11 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149885.334431 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85275.97 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 8527.60 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158412.931323 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76748.37 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 7674.84 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166087.768526 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69073.53 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 6907.35 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172995.122009 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62166.18 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 6216.62 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179211.740143 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55949.56 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 5594.96 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184806.696464 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50354.61 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 5035.46 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189842.157153 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45319.15 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 4531.91 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194374.071773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40787.23 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 4078.72 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198452.794931 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36708.51 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3670.85 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202123.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33037.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205723.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29437.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209323.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25837.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212923.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22237.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216523.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18637.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220123.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15037.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223723.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11437.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227323.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7837.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230923.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4237.66 sec) 12:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234523.645773 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 637.66 sec) 12:16:00 [GPUGRID] [rr_sim] 2235161.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982976750.22G/439.78G) 12:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852362.22 12:16:00 [---] pick_jobs_to_run() start 12:16:00 [---] pick_jobs_to_run() end 12:16:00 [---] [rr_sim] end 12:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:16:00 [---] [cpu_sched_debug] preliminary job list: 12:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:16:00 [---] [cpu_sched_debug] final job list: 12:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:16:00 [---] [cpu_sched_debug] enforce_run_list: end 12:16:00 [---] choose_project(): 44160.000000 12:16:00 [iThena.Measurements] doing RPC to get work 12:16:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564239.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391439.09 cpu 2137652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137652.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:16:00 [---] Got 0 tasks 12:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:17:00 [---] [rr_sim] doing sim: CPU sched 12:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982949159.86G/439.78G) 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235098.57 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 223509.86 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223509.856647 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011588.71 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 201158.87 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424668.727630 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810429.84 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 181042.98 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605711.711514 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629386.85 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 162938.69 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768650.397010 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466448.17 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 146644.82 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915295.213956 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319803.35 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 131980.34 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047275.549207 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187823.02 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 118782.30 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166057.850934 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069040.72 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 106904.07 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272961.922487 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962136.64 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 96213.66 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369175.586886 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865922.98 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 86592.30 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455767.884844 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779330.68 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 77933.07 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533700.953007 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701397.61 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 70139.76 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603840.714354 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631257.85 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 63125.79 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666966.499565 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568132.07 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 56813.21 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723779.706256 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511318.86 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 51131.89 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774911.592277 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460186.97 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 46018.70 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820930.289697 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414168.28 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 41416.83 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862347.117374 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372751.45 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 37275.14 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899622.262284 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335476.30 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 33547.63 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933169.892703 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301928.67 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 30192.87 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963362.760080 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271735.81 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 27173.58 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990536.340719 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244562.23 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 24456.22 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014992.563294 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220106.00 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 22010.60 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037003.163612 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198095.40 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 19809.54 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056812.703898 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178285.86 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 17828.59 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074641.290155 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160457.28 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 16045.73 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090687.017787 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144411.55 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 14441.15 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105128.172655 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129970.39 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 12997.04 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118125.212037 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116973.35 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 11697.34 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129822.547480 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105276.02 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 10527.60 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140350.149379 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94748.42 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 9474.84 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149824.991089 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85273.58 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 8527.36 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158352.348627 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76746.22 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 7674.62 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166026.970411 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69071.60 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 6907.16 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172934.130017 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62164.44 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 6216.44 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179150.573663 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55947.99 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 5594.80 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184745.372944 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50353.19 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 5035.32 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189780.692296 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45317.87 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 4531.79 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194312.479714 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40786.09 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 4078.61 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198391.088390 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36707.48 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3670.75 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202061.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33036.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205661.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29436.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209261.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25836.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212861.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22236.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216461.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18636.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220061.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15036.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223661.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11436.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227261.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7836.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230861.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4236.73 sec) 12:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234461.836198 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 636.73 sec) 12:17:00 [GPUGRID] [rr_sim] 2235098.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982949159.86G/439.78G) 12:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852359.48 12:17:00 [---] pick_jobs_to_run() start 12:17:00 [---] pick_jobs_to_run() end 12:17:00 [---] [rr_sim] end 12:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:17:00 [---] [cpu_sched_debug] preliminary job list: 12:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:17:00 [---] [cpu_sched_debug] final job list: 12:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:17:00 [---] [cpu_sched_debug] enforce_run_list: end 12:17:00 [---] choose_project(): 44220.000000 12:17:00 [iThena.Measurements] doing RPC to get work 12:17:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564179.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391379.09 cpu 2137592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137592.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:17:00 [---] Got 0 tasks 12:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:18:00 [---] [rr_sim] doing sim: CPU sched 12:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982921569.51G/439.78G) 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2235035.83 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 223503.58 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223503.582959 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011532.25 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 201153.22 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424656.807622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810379.02 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 181037.90 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605694.709819 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629341.12 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 162934.11 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768628.821796 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466407.01 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 146640.70 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915269.522575 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319766.31 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 131976.63 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047246.153276 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187789.68 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 118778.97 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1166025.120908 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1069010.71 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 106901.07 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272926.191776 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962109.64 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 96210.96 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369137.155557 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865898.67 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 86589.87 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455727.022960 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779308.81 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 77930.88 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533657.903623 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701377.93 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 70137.79 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603795.696220 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631240.13 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 63124.01 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666919.709557 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568116.12 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 56811.61 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723731.321560 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511304.51 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 51130.45 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774861.772363 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460174.06 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 46017.41 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820879.178086 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414156.65 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 41415.67 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862294.843236 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372740.99 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 37274.10 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899568.941871 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335466.89 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 33546.69 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933115.630643 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301920.20 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 30192.02 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963307.650538 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271728.18 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 27172.82 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990480.468443 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244555.36 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 24455.54 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014936.004557 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220099.83 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 22009.98 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036945.987061 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198089.84 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 19808.98 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056754.971314 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178280.86 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 17828.09 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074583.057141 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160452.77 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 16045.28 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090628.334386 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144407.50 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 14440.75 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105069.083906 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129966.75 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 12996.67 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118065.758475 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116970.07 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 11697.01 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129762.765586 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105273.06 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 10527.31 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140290.071986 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94745.76 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 9474.58 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149764.647747 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85271.18 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 8527.12 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158291.765931 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76744.06 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 7674.41 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165966.172297 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69069.66 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 6906.97 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172873.138026 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62162.69 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 6216.27 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179089.407182 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55946.42 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 5594.64 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184684.049423 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50351.78 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 5035.18 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189719.227440 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45316.60 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 4531.66 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194250.887655 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40784.94 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 4078.49 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198329.381848 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36706.45 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3670.64 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202000.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33035.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205600.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29435.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209200.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25835.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212800.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22235.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216400.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18635.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220000.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15035.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223600.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11435.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227200.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7835.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230800.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4235.80 sec) 12:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234400.026622 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 635.80 sec) 12:18:00 [GPUGRID] [rr_sim] 2235035.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982921569.51G/439.78G) 12:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852356.74 12:18:00 [---] pick_jobs_to_run() start 12:18:00 [---] pick_jobs_to_run() end 12:18:00 [---] [rr_sim] end 12:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:18:00 [---] [cpu_sched_debug] preliminary job list: 12:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:18:00 [---] [cpu_sched_debug] final job list: 12:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:18:00 [---] [cpu_sched_debug] enforce_run_list: end 12:18:00 [---] choose_project(): 44280.000000 12:18:00 [iThena.Measurements] doing RPC to get work 12:18:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564119.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391319.09 cpu 2137532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137532.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:18:00 [---] Got 0 tasks 12:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:19:00 [---] [rr_sim] doing sim: CPU sched 12:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982893979.15G/439.78G) 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234973.09 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 223497.31 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223497.309271 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011475.78 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 201147.58 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424644.887614 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810328.21 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 181032.82 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605677.708124 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629295.38 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 162929.54 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768607.246582 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466365.85 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 146636.58 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915243.831194 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319729.26 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 131972.93 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047216.757346 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187756.34 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 118775.63 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165992.390882 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068980.70 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 106898.07 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272890.461064 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962082.63 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 96208.26 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369098.724229 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865874.37 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 86587.44 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455686.161076 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779286.93 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 77928.69 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533614.854240 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701358.24 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 70135.82 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603750.678086 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631222.41 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 63122.24 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666872.919548 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568100.17 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 56810.02 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723682.936864 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511290.16 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 51129.02 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774811.952449 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460161.14 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 46016.11 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820828.066474 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414145.03 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 41414.50 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862242.569098 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372730.52 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 37273.05 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899515.621459 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335457.47 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 33545.75 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933061.368583 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301911.72 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 30191.17 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963252.540996 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271720.55 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 27172.06 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990424.596167 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244548.50 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 24454.85 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014879.445821 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220093.65 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 22009.36 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036888.810510 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198084.28 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 19808.43 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056697.238729 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178275.85 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 17827.59 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074524.824127 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160448.27 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 16044.83 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090569.650985 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144403.44 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 14440.34 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105009.995157 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129963.10 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 12996.31 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118006.304912 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116966.79 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 11696.68 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129702.983692 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105270.11 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 10527.01 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140229.994593 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94743.10 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 9474.31 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149704.304405 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85268.79 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 8526.88 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158231.183235 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76741.91 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 7674.19 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165905.374182 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69067.72 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 6906.77 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172812.146035 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62160.95 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 6216.09 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179028.240702 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55944.85 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 5594.49 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184622.725902 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50350.37 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 5035.04 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189657.762583 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45315.33 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 4531.53 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194189.295595 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40783.80 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 4078.38 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198267.675306 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36705.42 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3670.54 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201938.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33034.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205538.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29434.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209138.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25834.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212738.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22234.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216338.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18634.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219938.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15034.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223538.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11434.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227138.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7834.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230738.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4234.88 sec) 12:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234338.217046 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 634.88 sec) 12:19:00 [GPUGRID] [rr_sim] 2234973.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982893979.15G/439.78G) 12:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852354.01 12:19:00 [---] pick_jobs_to_run() start 12:19:00 [---] pick_jobs_to_run() end 12:19:00 [---] [rr_sim] end 12:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:19:00 [---] [cpu_sched_debug] preliminary job list: 12:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:19:00 [---] [cpu_sched_debug] final job list: 12:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:19:00 [---] [cpu_sched_debug] enforce_run_list: end 12:19:00 [---] choose_project(): 44340.000000 12:19:00 [iThena.Measurements] doing RPC to get work 12:19:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 564059.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391259.09 cpu 2137472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137472.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:19:00 [---] Got 0 tasks 12:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:20:00 [---] [rr_sim] doing sim: CPU sched 12:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982866388.79G/439.78G) 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234910.36 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 223491.04 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223491.035582 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011419.32 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 201141.93 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424632.967607 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810277.39 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 181027.74 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605660.706429 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629249.65 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 162924.96 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768585.671368 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466324.68 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 146632.47 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915218.139814 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319692.22 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 131969.22 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047187.361415 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187722.99 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 118772.30 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165959.660856 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068950.69 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 106895.07 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272854.730353 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962055.63 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 96205.56 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369060.292900 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865850.06 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 86585.01 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455645.299193 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779265.06 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 77926.51 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533571.804856 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701338.55 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 70133.86 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603705.659953 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631204.70 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 63120.47 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666826.129540 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568084.23 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 56808.42 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723634.552168 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511275.80 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 51127.58 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774762.132534 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460148.22 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 46014.82 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820776.954863 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414133.40 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 41413.34 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862190.294959 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372720.06 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 37272.01 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899462.301046 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335448.05 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 33544.81 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933007.106524 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301903.25 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 30190.32 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963197.431454 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271712.92 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 27171.29 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990368.723891 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244541.63 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 24454.16 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014822.887084 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220087.47 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 22008.75 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036831.633958 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198078.72 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 19807.87 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056639.506145 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178270.85 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 17827.08 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074466.591113 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160443.76 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 16044.38 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090510.967584 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144399.39 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 14439.94 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104950.906408 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129959.45 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 12995.94 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117946.851350 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116963.50 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 11696.35 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129643.201797 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105267.15 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 10526.72 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140169.917200 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94740.44 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 9474.04 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149643.961063 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85266.39 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 8526.64 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158170.600539 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76739.76 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 7673.98 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165844.576067 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69065.78 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 6906.58 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172751.154043 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62159.20 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 6215.92 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178967.074221 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55943.28 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 5594.33 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184561.402382 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50348.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 5034.90 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189596.297726 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45314.06 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 4531.41 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194127.703536 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40782.65 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 4078.27 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198205.968765 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36704.39 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3670.44 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201876.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33033.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205476.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29433.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209076.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25833.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212676.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22233.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216276.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18633.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219876.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15033.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223476.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11433.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227076.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7833.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230676.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4233.95 sec) 12:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234276.407471 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633.95 sec) 12:20:00 [GPUGRID] [rr_sim] 2234910.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982866388.79G/439.78G) 12:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852351.27 12:20:00 [---] pick_jobs_to_run() start 12:20:00 [---] pick_jobs_to_run() end 12:20:00 [---] [rr_sim] end 12:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:20:00 [---] [cpu_sched_debug] preliminary job list: 12:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:20:00 [---] [cpu_sched_debug] final job list: 12:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:20:00 [---] [cpu_sched_debug] enforce_run_list: end 12:20:00 [---] choose_project(): 44400.000000 12:20:00 [iThena.Measurements] doing RPC to get work 12:20:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563999.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391199.09 cpu 2137412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137412.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:20:00 [---] Got 0 tasks 12:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:21:00 [---] [rr_sim] doing sim: CPU sched 12:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982838798.44G/439.78G) 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234847.62 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 223484.76 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223484.761894 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011362.86 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 201136.29 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424621.047599 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810226.57 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 181022.66 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605643.704733 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629203.91 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 162920.39 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768564.096154 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466283.52 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 146628.35 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915192.448433 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319655.17 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 131965.52 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047157.965484 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187689.65 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 118768.97 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165926.930830 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068920.69 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 106892.07 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272818.999641 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962028.62 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 96202.86 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1369021.861571 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865825.76 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 86582.58 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455604.437309 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779243.18 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 77924.32 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533528.755472 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701318.86 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 70131.89 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603660.641819 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631186.98 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 63118.70 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666779.339531 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568068.28 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 56806.83 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723586.167473 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511261.45 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 51126.15 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774712.312620 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460135.31 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 46013.53 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820725.843252 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414121.78 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 41412.18 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862138.020821 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372709.60 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 37270.96 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899408.980633 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335438.64 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 33543.86 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932952.844464 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301894.77 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 30189.48 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963142.321912 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271705.30 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 27170.53 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990312.851615 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244534.77 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 24453.48 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014766.328348 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220081.29 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 22008.13 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036774.457407 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198073.16 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 19807.32 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056581.773561 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178265.85 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 17826.58 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074408.358099 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160439.26 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 16043.93 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090452.284183 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144395.33 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 14439.53 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104891.817659 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129955.80 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 12995.58 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117887.397788 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116960.22 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 11696.02 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129583.419903 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105264.20 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 10526.42 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140109.839807 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94737.78 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 9473.78 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149583.617721 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85264.00 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 8526.40 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158110.017843 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76737.60 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 7673.76 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165783.777953 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69063.84 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 6906.38 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172690.162052 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62157.46 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 6215.75 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178905.907741 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55941.71 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 5594.17 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184500.078861 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50347.54 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 5034.75 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189534.832869 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45312.79 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 4531.28 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194066.111477 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40781.51 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 4078.15 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198144.262223 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36703.36 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3670.34 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201814.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33033.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205414.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29433.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209014.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25833.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212614.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22233.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216214.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18633.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219814.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15033.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223414.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11433.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227014.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7833.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230614.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4233.02 sec) 12:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234214.597895 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 633.02 sec) 12:21:00 [GPUGRID] [rr_sim] 2234847.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982838798.44G/439.78G) 12:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852348.53 12:21:00 [---] pick_jobs_to_run() start 12:21:00 [---] pick_jobs_to_run() end 12:21:00 [---] [rr_sim] end 12:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:21:00 [---] [cpu_sched_debug] preliminary job list: 12:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:21:00 [---] [cpu_sched_debug] final job list: 12:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:21:00 [---] [cpu_sched_debug] enforce_run_list: end 12:21:00 [---] choose_project(): 44460.000000 12:21:00 [iThena.Measurements] doing RPC to get work 12:21:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563939.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391139.09 cpu 2137352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137352.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:21:00 [---] Got 0 tasks 12:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:22:00 [---] [rr_sim] doing sim: CPU sched 12:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982811208.08G/439.78G) 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234784.88 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 223478.49 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223478.488206 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011306.39 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 201130.64 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424609.127592 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810175.75 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 181017.58 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605626.703038 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629158.18 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 162915.82 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768542.520941 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466242.36 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 146624.24 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915166.757053 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319618.13 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 131961.81 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047128.569553 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187656.31 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 118765.63 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165894.200804 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068890.68 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 106889.07 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272783.268930 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 962001.61 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 96200.16 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368983.430243 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865801.45 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 86580.15 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455563.575425 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779221.31 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 77922.13 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533485.706088 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701299.18 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 70129.92 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603615.623685 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631169.26 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 63116.93 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666732.549523 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568052.33 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 56805.23 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723537.782777 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511247.10 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 51124.71 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774662.492705 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460122.39 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 46012.24 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820674.731641 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414110.15 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 41411.02 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862085.746683 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372699.14 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 37269.91 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899355.660220 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335429.22 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 33542.92 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932898.582404 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301886.30 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 30188.63 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963087.212370 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271697.67 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 27169.77 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990256.979339 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244527.90 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 24452.79 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014709.769611 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220075.11 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 22007.51 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036717.280856 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198067.60 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 19806.76 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056524.040977 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178260.84 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 17826.08 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074350.125085 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160434.76 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 16043.48 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090393.600783 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144391.28 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 14439.13 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104832.728910 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129952.15 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 12995.22 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117827.944225 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116956.94 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 11695.69 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129523.638009 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105261.24 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 10526.12 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140049.762414 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94735.12 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 9473.51 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149523.274379 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85261.61 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 8526.16 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158049.435147 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76735.45 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 7673.54 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165722.979838 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69061.90 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 6906.19 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172629.170060 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62155.71 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 6215.57 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178844.741260 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55940.14 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 5594.01 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184438.755341 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50346.13 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 5034.61 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189473.368013 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45311.51 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 4531.15 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194004.519417 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40780.36 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 4078.04 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198082.555682 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36702.33 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3670.23 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201752.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33032.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205352.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29432.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208952.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25832.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212552.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22232.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216152.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18632.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219752.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15032.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223352.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11432.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226952.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7832.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230552.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4232.09 sec) 12:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234152.788320 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 632.09 sec) 12:22:00 [GPUGRID] [rr_sim] 2234784.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982811208.08G/439.78G) 12:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852345.79 12:22:00 [---] pick_jobs_to_run() start 12:22:00 [---] pick_jobs_to_run() end 12:22:00 [---] [rr_sim] end 12:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:22:00 [---] [cpu_sched_debug] preliminary job list: 12:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:22:00 [---] [cpu_sched_debug] final job list: 12:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:22:00 [---] [cpu_sched_debug] enforce_run_list: end 12:22:00 [---] choose_project(): 44520.000000 12:22:00 [iThena.Measurements] doing RPC to get work 12:22:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563879.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391079.09 cpu 2137292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137292.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:22:00 [---] Got 0 tasks 12:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:23:00 [---] [rr_sim] doing sim: CPU sched 12:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982783617.72G/439.78G) 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234722.15 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 223472.21 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223472.214518 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011249.93 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 201124.99 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424597.207584 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810124.94 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 181012.49 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605609.701343 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629112.44 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 162911.24 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768520.945727 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466201.20 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 146620.12 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915141.065672 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319581.08 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 131958.11 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047099.173623 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187622.97 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 118762.30 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165861.470778 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068860.67 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 106886.07 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272747.538218 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961974.61 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 96197.46 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368944.998914 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865777.15 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 86577.71 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455522.713541 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779199.43 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 77919.94 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533442.656704 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701279.49 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 70127.95 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603570.605552 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631151.54 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 63115.15 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666685.759514 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568036.39 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 56803.64 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723489.398081 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511232.75 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 51123.27 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774612.672791 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460109.47 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 46010.95 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820623.620029 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414098.53 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 41409.85 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862033.472544 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372688.67 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 37268.87 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899302.339808 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335419.81 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 33541.98 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932844.320345 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301877.82 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 30187.78 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963032.102828 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271690.04 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 27169.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990201.107063 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244521.04 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 24452.10 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014653.210875 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220068.93 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 22006.89 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036660.104305 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198062.04 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 19806.20 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056466.308392 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178255.84 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 17825.58 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074291.892071 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160430.25 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 16043.03 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090334.917382 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144387.23 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 14438.72 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104773.640161 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129948.51 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 12994.85 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117768.490663 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116953.65 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 11695.37 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129463.856115 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105258.29 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 10525.83 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139989.685021 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94732.46 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 9473.25 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149462.931037 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85259.21 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 8525.92 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157988.852451 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76733.29 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 7673.33 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165662.181724 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69059.96 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 6906.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172568.178069 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62153.97 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 6215.40 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178783.574780 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55938.57 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 5593.86 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184377.431820 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50344.71 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 5034.47 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189411.903156 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45310.24 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 4531.02 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193942.927358 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40779.22 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 4077.92 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198020.849140 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36701.30 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3670.13 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201690.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33031.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205290.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29431.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208890.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25831.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212490.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22231.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216090.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18631.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219690.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15031.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223290.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11431.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226890.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7831.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230490.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4231.17 sec) 12:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234090.978744 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631.17 sec) 12:23:00 [GPUGRID] [rr_sim] 2234722.15: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982783617.72G/439.78G) 12:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852343.06 12:23:00 [---] pick_jobs_to_run() start 12:23:00 [---] pick_jobs_to_run() end 12:23:00 [---] [rr_sim] end 12:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:23:00 [---] [cpu_sched_debug] preliminary job list: 12:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:23:00 [---] [cpu_sched_debug] final job list: 12:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:23:00 [---] [cpu_sched_debug] enforce_run_list: end 12:23:00 [---] choose_project(): 44580.000000 12:23:00 [iThena.Measurements] doing RPC to get work 12:23:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563819.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 391019.09 cpu 2137232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137232.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:23:00 [---] Got 0 tasks 12:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:24:00 [---] [rr_sim] doing sim: CPU sched 12:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982756027.37G/439.78G) 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234659.41 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 223465.94 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223465.940830 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011193.47 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 201119.35 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424585.287576 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810074.12 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 181007.41 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605592.699648 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629066.71 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 162906.67 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768499.370513 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466160.04 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 146616.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915115.374291 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319544.03 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 131954.40 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047069.777692 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187589.63 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 118758.96 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165828.740752 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068830.67 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 106883.07 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272711.807507 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961947.60 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 96194.76 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368906.567586 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865752.84 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 86575.28 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455481.851657 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779177.56 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 77917.76 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533399.607321 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701259.80 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 70125.98 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603525.587418 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631133.82 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 63113.38 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666638.969506 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568020.44 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 56802.04 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723441.013385 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511218.39 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 51121.84 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774562.852876 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460096.56 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 46009.66 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820572.508418 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414086.90 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 41408.69 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861981.198406 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372678.21 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 37267.82 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899249.019395 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335410.39 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 33541.04 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932790.058285 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301869.35 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 30186.94 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962976.993286 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271682.42 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 27168.24 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990145.234787 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244514.17 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 24451.42 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014596.652138 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220062.76 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 22006.28 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036602.927754 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198056.48 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 19805.65 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056408.575808 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178250.83 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 17825.08 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074233.659057 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160425.75 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 16042.57 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090276.233981 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144383.17 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 14438.32 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104714.551412 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129944.86 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 12994.49 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117709.037101 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116950.37 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 11695.04 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129404.074220 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105255.33 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 10525.53 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139929.607628 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94729.80 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 9472.98 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149402.587695 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85256.82 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 8525.68 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157928.269755 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76731.14 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 7673.11 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165601.383609 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69058.02 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 6905.80 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172507.186078 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62152.22 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 6215.22 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178722.408300 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55937.00 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 5593.70 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184316.108299 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50343.30 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 5034.33 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189350.438299 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45308.97 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 4530.90 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193881.335299 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40778.07 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 4077.81 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197959.142598 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36700.27 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3670.03 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201629.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33030.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205229.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29430.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208829.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25830.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212429.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22230.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216029.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18630.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219629.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15030.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223229.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11430.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226829.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7830.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230429.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4230.24 sec) 12:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2234029.169168 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630.24 sec) 12:24:00 [GPUGRID] [rr_sim] 2234659.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982756027.37G/439.78G) 12:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852340.32 12:24:00 [---] pick_jobs_to_run() start 12:24:00 [---] pick_jobs_to_run() end 12:24:00 [---] [rr_sim] end 12:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:24:00 [---] [cpu_sched_debug] preliminary job list: 12:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:24:00 [---] [cpu_sched_debug] final job list: 12:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:24:00 [---] [cpu_sched_debug] enforce_run_list: end 12:24:00 [---] choose_project(): 44640.000000 12:24:00 [iThena.Measurements] doing RPC to get work 12:24:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563759.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390959.09 cpu 2137172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137172.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:24:00 [---] Got 0 tasks 12:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:25:00 [---] [rr_sim] doing sim: CPU sched 12:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982728437.01G/439.78G) 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234596.67 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 223459.67 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223459.667141 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011137.00 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 201113.70 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424573.367569 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1810023.30 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 181002.33 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605575.697953 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1629020.97 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 162902.10 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768477.795299 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466118.88 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 146611.89 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915089.682911 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319506.99 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 131950.70 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047040.381761 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187556.29 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 118755.63 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165796.010726 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068800.66 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 106880.07 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272676.076795 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961920.59 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 96192.06 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368868.136257 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865728.54 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 86572.85 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455440.989773 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779155.68 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 77915.57 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533356.557937 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701240.11 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 70124.01 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603480.569285 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631116.10 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 63111.61 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666592.179498 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568004.49 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 56800.45 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723392.628689 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511204.04 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 51120.40 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774513.032962 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460083.64 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 46008.36 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820521.396807 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414075.27 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 41407.53 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861928.924268 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372667.75 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 37266.77 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899195.698982 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335400.97 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 33540.10 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932735.796225 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301860.88 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 30186.09 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962921.883744 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271674.79 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 27167.48 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990089.362511 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244507.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 24450.73 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014540.093402 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220056.58 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 22005.66 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036545.751203 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198050.92 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 19805.09 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056350.843224 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178245.83 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 17824.58 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074175.426043 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160421.25 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 16042.12 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090217.550580 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144379.12 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 14437.91 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104655.462663 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129941.21 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 12994.12 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117649.583539 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116947.09 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 11694.71 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129344.292326 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105252.38 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 10525.24 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139869.530235 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94727.14 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 9472.71 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149342.244353 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85254.43 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 8525.44 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157867.687059 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76728.98 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 7672.90 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165540.585494 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69056.09 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 6905.61 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172446.194086 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62150.48 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 6215.05 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178661.241819 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55935.43 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 5593.54 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184254.784779 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50341.89 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 5034.19 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189288.973442 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45307.70 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 4530.77 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193819.743239 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40776.93 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 4077.69 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197897.436057 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36699.24 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3669.92 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201567.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33029.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205167.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29429.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208767.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25829.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212367.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22229.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215967.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18629.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219567.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15029.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223167.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11429.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226767.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7829.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230367.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4229.31 sec) 12:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233967.359593 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629.31 sec) 12:25:00 [GPUGRID] [rr_sim] 2234596.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982728437.01G/439.78G) 12:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852337.58 12:25:00 [---] pick_jobs_to_run() start 12:25:00 [---] pick_jobs_to_run() end 12:25:00 [---] [rr_sim] end 12:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:25:00 [---] [cpu_sched_debug] preliminary job list: 12:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:25:00 [---] [cpu_sched_debug] final job list: 12:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:25:00 [---] [cpu_sched_debug] enforce_run_list: end 12:25:00 [---] choose_project(): 44700.000000 12:25:00 [iThena.Measurements] doing RPC to get work 12:25:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563699.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390899.09 cpu 2137112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137112.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:25:00 [---] Got 0 tasks 12:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:26:00 [---] [rr_sim] doing sim: CPU sched 12:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982700846.66G/439.78G) 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234533.93 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 223453.39 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223453.393453 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011080.54 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 201108.05 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424561.447561 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809972.49 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 180997.25 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605558.696258 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628975.24 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 162897.52 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768456.220086 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466077.71 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 146607.77 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915063.991530 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319469.94 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 131946.99 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1047010.985830 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187522.95 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 118752.29 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165763.280700 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068770.65 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 106877.07 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272640.346084 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961893.59 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 96189.36 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368829.704928 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865704.23 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 86570.42 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455400.127889 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779133.81 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 77913.38 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533313.508553 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701220.43 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 70122.04 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603435.551151 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631098.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 63109.84 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666545.389489 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567988.55 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 56798.85 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723344.243993 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511189.69 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 51118.97 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774463.213047 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460070.72 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 46007.07 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820470.285196 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414063.65 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 41406.36 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861876.650129 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372657.28 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 37265.73 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899142.378570 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335391.56 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 33539.16 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932681.534166 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301852.40 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 30185.24 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962866.774202 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271667.16 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 27166.72 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990033.490235 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244500.44 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 24450.04 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014483.534665 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220050.40 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 22005.04 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036488.574652 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198045.36 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 19804.54 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056293.110640 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178240.82 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 17824.08 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074117.193029 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160416.74 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 16041.67 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090158.867179 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144375.07 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 14437.51 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104596.373914 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129937.56 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 12993.76 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117590.129976 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116943.80 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 11694.38 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129284.510432 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105249.42 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 10524.94 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139809.452842 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94724.48 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 9472.45 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149281.901011 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85252.03 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 8525.20 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157807.104363 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76726.83 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 7672.68 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165479.787380 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69054.15 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 6905.41 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172385.202095 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62148.73 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 6214.87 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178600.075339 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55933.86 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 5593.39 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184193.461258 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50340.47 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 5034.05 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189227.508585 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45306.43 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 4530.64 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193758.151180 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40775.78 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 4077.58 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197835.729515 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36698.21 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3669.82 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201505.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33028.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205105.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29428.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208705.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25828.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212305.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22228.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215905.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18628.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219505.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15028.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223105.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11428.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226705.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7828.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230305.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4228.38 sec) 12:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233905.550017 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628.38 sec) 12:26:00 [GPUGRID] [rr_sim] 2234533.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982700846.66G/439.78G) 12:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852334.85 12:26:00 [---] pick_jobs_to_run() start 12:26:00 [---] pick_jobs_to_run() end 12:26:00 [---] [rr_sim] end 12:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:26:00 [---] [cpu_sched_debug] preliminary job list: 12:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:26:00 [---] [cpu_sched_debug] final job list: 12:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:26:00 [---] [cpu_sched_debug] enforce_run_list: end 12:26:00 [---] choose_project(): 44760.000000 12:26:00 [iThena.Measurements] doing RPC to get work 12:26:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563639.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390839.09 cpu 2137052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2137052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2137052.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:26:00 [---] Got 0 tasks 12:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:27:00 [---] [rr_sim] doing sim: CPU sched 12:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982673256.30G/439.78G) 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234471.20 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 223447.12 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223447.119765 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2011024.08 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 201102.41 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424549.527553 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809921.67 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 180992.17 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605541.694563 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628929.50 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 162892.95 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768434.644872 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1466036.55 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 146603.66 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915038.300150 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319432.90 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 131943.29 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046981.589900 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187489.61 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 118748.96 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165730.550675 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068740.65 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 106874.06 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272604.615372 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961866.58 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 96186.66 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368791.273600 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865679.92 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 86567.99 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455359.266005 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779111.93 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 77911.19 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533270.459169 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701200.74 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 70120.07 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603390.533017 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631080.66 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 63108.07 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666498.599481 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567972.60 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 56797.26 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723295.859298 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511175.34 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 51117.53 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774413.393133 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460057.80 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 46005.78 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820419.173584 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414052.02 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 41405.20 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861824.375991 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372646.82 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 37264.68 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899089.058157 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335382.14 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 33538.21 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932627.272106 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301843.93 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 30184.39 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962811.664660 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271659.53 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 27165.95 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989977.617959 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244493.58 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 24449.36 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014426.975928 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220044.22 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 22004.42 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036431.398101 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198039.80 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 19803.98 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056235.378055 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178235.82 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 17823.58 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074058.960015 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160412.24 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 16041.22 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090100.183778 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144371.01 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 14437.10 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104537.285166 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129933.91 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 12993.39 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117530.676414 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116940.52 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 11694.05 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129224.728538 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105246.47 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 10524.65 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139749.375449 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94721.82 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 9472.18 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149221.557669 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85249.64 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 8524.96 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157746.521667 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76724.68 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 7672.47 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165418.989265 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69052.21 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 6905.22 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172324.210104 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62146.99 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 6214.70 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178538.908858 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55932.29 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 5593.23 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184132.137737 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50339.06 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 5033.91 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189166.043729 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45305.15 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 4530.52 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193696.559121 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40774.64 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 4077.46 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197774.022974 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36697.17 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3669.72 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201443.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33027.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205043.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29427.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208643.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25827.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212243.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22227.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215843.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18627.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219443.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15027.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223043.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11427.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226643.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7827.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230243.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4227.46 sec) 12:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233843.740441 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627.46 sec) 12:27:00 [GPUGRID] [rr_sim] 2234471.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982673256.30G/439.78G) 12:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852332.11 12:27:00 [---] pick_jobs_to_run() start 12:27:00 [---] pick_jobs_to_run() end 12:27:00 [---] [rr_sim] end 12:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:27:00 [---] [cpu_sched_debug] preliminary job list: 12:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:27:00 [---] [cpu_sched_debug] final job list: 12:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:27:00 [---] [cpu_sched_debug] enforce_run_list: end 12:27:00 [---] choose_project(): 44820.000000 12:27:00 [iThena.Measurements] doing RPC to get work 12:27:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563579.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390779.09 cpu 2136992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136992.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:27:00 [---] Got 0 tasks 12:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:28:00 [---] [rr_sim] doing sim: CPU sched 12:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982645665.94G/439.78G) 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234408.46 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 223440.85 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223440.846077 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010967.61 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 201096.76 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424537.607546 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809870.85 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 180987.09 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605524.692868 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628883.77 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 162888.38 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768413.069658 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465995.39 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 146599.54 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 915012.608769 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319395.85 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 131939.59 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046952.193969 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187456.27 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 118745.63 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165697.820649 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068710.64 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 106871.06 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272568.884661 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961839.58 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 96183.96 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368752.842271 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865655.62 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 86565.56 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455318.404121 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779090.06 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 77909.01 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533227.409786 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701181.05 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 70118.11 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603345.514884 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631062.95 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 63106.29 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666451.809472 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567956.65 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 56795.67 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723247.474602 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511160.99 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 51116.10 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774363.573218 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460044.89 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 46004.49 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820368.061973 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414040.40 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 41404.04 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861772.101853 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372636.36 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 37263.64 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899035.737744 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335372.72 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 33537.27 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932573.010046 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301835.45 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 30183.55 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962756.555119 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271651.91 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 27165.19 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989921.745683 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244486.72 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 24448.67 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014370.417192 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220038.04 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 22003.80 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036374.221549 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198034.24 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 19803.42 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056177.645471 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178230.82 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 17823.08 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074000.727001 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160407.73 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 16040.77 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090041.500378 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144366.96 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 14436.70 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104478.196417 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129930.26 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 12993.03 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117471.222852 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116937.24 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 11693.72 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129164.946643 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105243.51 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 10524.35 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139689.298056 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94719.16 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 9471.92 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149161.214327 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85247.25 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 8524.72 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157685.938971 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76722.52 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 7672.25 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165358.191151 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69050.27 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 6905.03 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172263.218112 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62145.24 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 6214.52 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178477.742378 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55930.72 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 5593.07 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184070.814217 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50337.65 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 5033.76 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189104.578872 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45303.88 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 4530.39 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193634.967061 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40773.49 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 4077.35 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197712.316432 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36696.14 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3669.61 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201381.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33026.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204981.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29426.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208581.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25826.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212181.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22226.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215781.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18626.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219381.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15026.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222981.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11426.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226581.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7826.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230181.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4226.53 sec) 12:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233781.930866 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626.53 sec) 12:28:00 [GPUGRID] [rr_sim] 2234408.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982645665.94G/439.78G) 12:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852329.37 12:28:00 [---] pick_jobs_to_run() start 12:28:00 [---] pick_jobs_to_run() end 12:28:00 [---] [rr_sim] end 12:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:28:00 [---] [cpu_sched_debug] preliminary job list: 12:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:28:00 [---] [cpu_sched_debug] final job list: 12:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:28:00 [---] [cpu_sched_debug] enforce_run_list: end 12:28:00 [---] choose_project(): 44880.000000 12:28:00 [iThena.Measurements] doing RPC to get work 12:28:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563519.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390719.09 cpu 2136932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136932.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:28:00 [---] Got 0 tasks 12:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:29:00 [---] [rr_sim] doing sim: CPU sched 12:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982618075.59G/439.78G) 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234345.72 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 223434.57 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223434.572389 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010911.15 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 201091.12 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424525.687538 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809820.04 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 180982.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605507.691173 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628838.03 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 162883.80 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768391.494444 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465954.23 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 146595.42 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914986.917388 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319358.81 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 131935.88 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046922.798038 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187422.93 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 118742.29 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165665.090623 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068680.63 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 106868.06 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272533.153949 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961812.57 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 96181.26 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368714.410943 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865631.31 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 86563.13 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455277.542237 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779068.18 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 77906.82 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533184.360402 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701161.36 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 70116.14 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603300.496750 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631045.23 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 63104.52 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666405.019464 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567940.70 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 56794.07 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723199.089906 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511146.63 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 51114.66 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774313.753304 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460031.97 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 46003.20 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820316.950362 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414028.77 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 41402.88 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861719.827714 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372625.90 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 37262.59 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898982.417331 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335363.31 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 33536.33 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932518.747987 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301826.98 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 30182.70 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962701.445577 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271644.28 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 27164.43 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989865.873408 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244479.85 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 24447.99 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014313.858455 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220031.87 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 22003.19 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036317.044998 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198028.68 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 19802.87 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056119.912887 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178225.81 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 17822.58 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073942.493987 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160403.23 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 16040.32 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089982.816977 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144362.91 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 14436.29 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104419.107668 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129926.62 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 12992.66 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117411.769289 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116933.95 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 11693.40 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129105.164749 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105240.56 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 10524.06 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139629.220663 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94716.50 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 9471.65 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149100.870985 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85244.85 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 8524.49 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157625.356275 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76720.37 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 7672.04 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165297.393036 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69048.33 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 6904.83 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172202.226121 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62143.50 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 6214.35 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178416.575897 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55929.15 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 5592.91 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184009.490696 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50336.23 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 5033.62 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189043.114015 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45302.61 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 4530.26 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193573.375002 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40772.35 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 4077.23 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197650.609890 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36695.11 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3669.51 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201320.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33025.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204920.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29425.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208520.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25825.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212120.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22225.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215720.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18625.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219320.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15025.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222920.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11425.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226520.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7825.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230120.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4225.60 sec) 12:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233720.121290 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625.60 sec) 12:29:00 [GPUGRID] [rr_sim] 2234345.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982618075.59G/439.78G) 12:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852326.64 12:29:00 [---] pick_jobs_to_run() start 12:29:00 [---] pick_jobs_to_run() end 12:29:00 [---] [rr_sim] end 12:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:29:00 [---] [cpu_sched_debug] preliminary job list: 12:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:29:00 [---] [cpu_sched_debug] final job list: 12:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:29:00 [---] [cpu_sched_debug] enforce_run_list: end 12:29:00 [---] choose_project(): 44940.000000 12:29:00 [iThena.Measurements] doing RPC to get work 12:29:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563459.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390659.09 cpu 2136872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136872.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:29:00 [---] Got 0 tasks 12:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:30:00 [---] [rr_sim] doing sim: CPU sched 12:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982590485.23G/439.78G) 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234282.99 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 223428.30 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223428.298700 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010854.69 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 201085.47 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424513.767531 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809769.22 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 180976.92 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605490.689478 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628792.30 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 162879.23 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768369.919230 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465913.07 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 146591.31 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914961.226008 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319321.76 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 131932.18 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046893.402107 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187389.58 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 118738.96 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165632.360597 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068650.63 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 106865.06 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272497.423237 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961785.56 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 96178.56 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368675.979614 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865607.01 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 86560.70 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455236.680353 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779046.31 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 77904.63 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533141.311018 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701141.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 70114.17 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603255.478617 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631027.51 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 63102.75 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666358.229455 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567924.76 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 56792.48 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723150.705210 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511132.28 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 51113.23 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774263.933389 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460019.05 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 46001.91 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820265.838751 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414017.15 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 41401.71 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861667.553576 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372615.43 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 37261.54 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898929.096919 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335353.89 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 33535.39 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932464.485927 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301818.50 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 30181.85 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962646.336035 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271636.65 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 27163.67 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989810.001132 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244472.99 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 24447.30 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014257.299719 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220025.69 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 22002.57 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036259.868447 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198023.12 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 19802.31 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056062.180303 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178220.81 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 17822.08 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073884.260973 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160398.73 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 16039.87 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089924.133576 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144358.85 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 14435.89 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104360.018919 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129922.97 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 12992.30 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117352.315727 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116930.67 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 11693.07 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129045.382855 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105237.60 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 10523.76 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139569.143270 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94713.84 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 9471.38 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149040.527643 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85242.46 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 8524.25 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157564.773579 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76718.21 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 7671.82 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165236.594921 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69046.39 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 6904.64 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172141.234130 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62141.75 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 6214.18 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178355.409417 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55927.58 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 5592.76 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183948.167176 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50334.82 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 5033.48 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188981.649158 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45301.34 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 4530.13 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193511.782943 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40771.20 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 4077.12 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197588.903349 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36694.08 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3669.41 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201258.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33024.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204858.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29424.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208458.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25824.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212058.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22224.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215658.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18624.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219258.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15024.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222858.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11424.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226458.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7824.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230058.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4224.68 sec) 12:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233658.311714 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624.68 sec) 12:30:00 [GPUGRID] [rr_sim] 2234282.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982590485.23G/439.78G) 12:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852323.90 12:30:00 [---] pick_jobs_to_run() start 12:30:00 [---] pick_jobs_to_run() end 12:30:00 [---] [rr_sim] end 12:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:30:00 [---] [cpu_sched_debug] preliminary job list: 12:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:30:00 [---] [cpu_sched_debug] final job list: 12:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:30:00 [---] [cpu_sched_debug] enforce_run_list: end 12:30:00 [---] choose_project(): 45000.000000 12:30:00 [iThena.Measurements] doing RPC to get work 12:30:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563399.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390599.09 cpu 2136812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136812.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:30:00 [---] Got 0 tasks 12:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:31:00 [---] [rr_sim] doing sim: CPU sched 12:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982562894.87G/439.78G) 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234220.25 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 223422.03 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223422.025012 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010798.23 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 201079.82 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424501.847523 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809718.40 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 180971.84 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605473.687783 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628746.56 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 162874.66 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768348.344017 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465871.91 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 146587.19 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914935.534627 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319284.72 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 131928.47 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046864.006176 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187356.24 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 118735.62 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165599.630571 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068620.62 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 106862.06 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272461.692526 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961758.56 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 96175.86 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368637.548285 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865582.70 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 86558.27 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455195.818469 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779024.43 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 77902.44 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533098.261634 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701121.99 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 70112.20 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603210.460483 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 631009.79 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 63100.98 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666311.439447 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567908.81 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 56790.88 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723102.320514 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511117.93 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 51111.79 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774214.113475 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460006.14 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 46000.61 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820214.727139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414005.52 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 41400.55 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861615.279438 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372604.97 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 37260.50 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898875.776506 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335344.47 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 33534.45 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932410.223867 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301810.03 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 30181.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962591.226493 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271629.02 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 27162.90 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989754.128856 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244466.12 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 24446.61 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014200.740982 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220019.51 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 22001.95 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036202.691896 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198017.56 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 19801.76 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056004.447719 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178215.80 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 17821.58 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073826.027959 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160394.22 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 16039.42 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089865.450175 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144354.80 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 14435.48 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104300.930170 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129919.32 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 12991.93 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117292.862165 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116927.39 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 11692.74 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128985.600960 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105234.65 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 10523.46 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139509.065876 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94711.18 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 9471.12 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148980.184301 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85240.07 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 8524.01 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157504.190883 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76716.06 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 7671.61 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165175.796807 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69044.45 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 6904.45 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172080.242138 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62140.01 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 6214.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178294.242936 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55926.01 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 5592.60 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183886.843655 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50333.41 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 5033.34 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188920.184302 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45300.07 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 4530.01 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193450.190884 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40770.06 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 4077.01 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197527.196807 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36693.05 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3669.31 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201196.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33023.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204796.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29423.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208396.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25823.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211996.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22223.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215596.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18623.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219196.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15023.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222796.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11423.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226396.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7823.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229996.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4223.75 sec) 12:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233596.502139 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623.75 sec) 12:31:00 [GPUGRID] [rr_sim] 2234220.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982562894.87G/439.78G) 12:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852321.16 12:31:00 [---] pick_jobs_to_run() start 12:31:00 [---] pick_jobs_to_run() end 12:31:00 [---] [rr_sim] end 12:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:31:00 [---] [cpu_sched_debug] preliminary job list: 12:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:31:00 [---] [cpu_sched_debug] final job list: 12:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:31:00 [---] [cpu_sched_debug] enforce_run_list: end 12:31:00 [---] choose_project(): 45060.000000 12:31:00 [iThena.Measurements] doing RPC to get work 12:31:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563339.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390539.09 cpu 2136752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136752.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:31:00 [---] Got 0 tasks 12:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:32:00 [---] [rr_sim] doing sim: CPU sched 12:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982535304.52G/439.78G) 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234157.51 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 223415.75 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223415.751324 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010741.76 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 201074.18 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424489.927515 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809667.59 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 180966.76 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605456.686088 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628700.83 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 162870.08 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768326.768803 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465830.74 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 146583.07 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914909.843246 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319247.67 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 131924.77 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046834.610246 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187322.90 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 118732.29 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165566.900545 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068590.61 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 106859.06 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272425.961814 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961731.55 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 96173.16 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368599.116957 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865558.40 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 86555.84 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455154.956585 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 779002.56 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 77900.26 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533055.212250 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701102.30 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 70110.23 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603165.442349 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630992.07 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 63099.21 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666264.649438 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567892.86 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 56789.29 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723053.935818 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511103.58 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 51110.36 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774164.293560 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459993.22 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 45999.32 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820163.615528 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413993.90 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 41399.39 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861563.005299 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372594.51 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 37259.45 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898822.456093 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335335.06 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 33533.51 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932355.961808 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301801.55 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 30180.16 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962536.116951 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271621.40 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 27162.14 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989698.256580 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244459.26 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 24445.93 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014144.182246 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220013.33 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 22001.33 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036145.515345 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198012.00 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 19801.20 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055946.715134 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178210.80 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 17821.08 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073767.794945 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160389.72 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 16038.97 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089806.766774 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144350.75 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 14435.07 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104241.841421 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129915.67 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 12991.57 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117233.408603 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116924.10 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 11692.41 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128925.819066 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105231.69 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 10523.17 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139448.988483 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94708.52 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 9470.85 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148919.840959 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85237.67 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 8523.77 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157443.608187 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76713.91 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 7671.39 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165114.998692 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69042.51 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 6904.25 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172019.250147 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62138.26 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 6213.83 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178233.076456 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55924.44 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 5592.44 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183825.520134 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50331.99 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 5033.20 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188858.719445 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45298.79 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 4529.88 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193388.598824 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40768.91 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 4076.89 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197465.490266 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36692.02 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3669.20 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201134.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33022.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204734.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29422.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208334.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25822.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211934.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22222.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215534.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18622.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219134.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15022.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222734.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11422.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226334.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7822.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229934.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4222.82 sec) 12:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233534.692563 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622.82 sec) 12:32:00 [GPUGRID] [rr_sim] 2234157.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982535304.52G/439.78G) 12:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852318.43 12:32:00 [---] pick_jobs_to_run() start 12:32:00 [---] pick_jobs_to_run() end 12:32:00 [---] [rr_sim] end 12:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:32:00 [---] [cpu_sched_debug] preliminary job list: 12:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:32:00 [---] [cpu_sched_debug] final job list: 12:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:32:00 [---] [cpu_sched_debug] enforce_run_list: end 12:32:00 [---] choose_project(): 45120.000000 12:32:00 [iThena.Measurements] doing RPC to get work 12:32:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563279.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390479.09 cpu 2136692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136692.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:32:00 [---] Got 0 tasks 12:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:33:00 [---] [rr_sim] doing sim: CPU sched 12:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982507714.16G/439.78G) 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234094.78 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 223409.48 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223409.477636 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010685.30 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 201068.53 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424478.007508 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809616.77 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 180961.68 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605439.684393 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628655.09 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 162865.51 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768305.193589 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465789.58 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 146578.96 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914884.151866 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319210.62 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 131921.06 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046805.214315 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187289.56 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 118728.96 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165534.170519 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068560.61 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 106856.06 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272390.231103 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961704.55 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 96170.45 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368560.685628 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865534.09 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 86553.41 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455114.094701 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778980.68 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 77898.07 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1533012.162867 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701082.61 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 70108.26 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603120.424216 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630974.35 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 63097.44 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666217.859430 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567876.92 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 56787.69 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1723005.551122 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511089.23 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 51108.92 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774114.473646 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459980.30 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 45998.03 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820112.503917 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413982.27 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 41398.23 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861510.731161 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372584.05 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 37258.40 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898769.135681 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335325.64 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 33532.56 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932301.699748 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301793.08 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 30179.31 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962481.007409 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271613.77 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 27161.38 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989642.384304 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244452.39 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 24445.24 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014087.623509 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220007.15 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 22000.72 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036088.338794 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198006.44 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 19800.64 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055888.982550 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178205.79 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 17820.58 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073709.561931 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160385.21 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 16038.52 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089748.083373 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144346.69 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 14434.67 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104182.752672 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129912.02 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 12991.20 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117173.955040 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116920.82 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 11692.08 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128866.037172 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105228.74 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 10522.87 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139388.911090 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94705.87 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 9470.59 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148859.497617 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85235.28 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 8523.53 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157383.025491 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76711.75 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 7671.18 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165054.200578 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69040.58 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 6904.06 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171958.258155 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62136.52 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 6213.65 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178171.909976 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55922.87 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 5592.29 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183764.196614 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50330.58 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 5033.06 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188797.254588 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45297.52 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 4529.75 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193327.006765 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40767.77 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 4076.78 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197403.783724 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36690.99 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3669.10 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201072.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33021.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204672.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29421.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208272.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25821.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211872.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22221.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215472.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18621.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219072.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15021.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222672.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11421.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226272.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7821.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229872.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4221.89 sec) 12:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233472.882987 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621.89 sec) 12:33:00 [GPUGRID] [rr_sim] 2234094.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982507714.16G/439.78G) 12:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852315.69 12:33:00 [---] pick_jobs_to_run() start 12:33:00 [---] pick_jobs_to_run() end 12:33:00 [---] [rr_sim] end 12:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:33:00 [---] [cpu_sched_debug] preliminary job list: 12:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:33:00 [---] [cpu_sched_debug] final job list: 12:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:33:00 [---] [cpu_sched_debug] enforce_run_list: end 12:33:00 [---] choose_project(): 45180.000000 12:33:00 [iThena.Measurements] doing RPC to get work 12:33:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563219.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390419.09 cpu 2136632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136632.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:33:00 [---] Got 0 tasks 12:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:34:00 [---] [rr_sim] doing sim: CPU sched 12:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982480123.81G/439.78G) 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2234032.04 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 223403.20 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223403.203947 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010628.84 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 201062.88 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424466.087500 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809565.95 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 180956.60 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605422.682698 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628609.36 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 162860.94 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768283.618375 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465748.42 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 146574.84 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914858.460485 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319173.58 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 131917.36 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046775.818384 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187256.22 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 118725.62 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165501.440493 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068530.60 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 106853.06 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272354.500391 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961677.54 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 96167.75 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368522.254300 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865509.79 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 86550.98 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455073.232817 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778958.81 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 77895.88 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532969.113483 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701062.93 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 70106.29 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603075.406082 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630956.63 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 63095.66 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666171.069421 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567860.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 56786.10 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722957.166427 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511074.87 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 51107.49 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774064.653731 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459967.39 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 45996.74 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820061.392306 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413970.65 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 41397.06 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861458.457023 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372573.58 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 37257.36 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898715.815268 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335316.22 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 33531.62 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932247.437689 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301784.60 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 30178.46 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962425.897867 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271606.14 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 27160.61 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989586.512028 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244445.53 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 24444.55 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014031.064773 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220000.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 22000.10 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036031.162243 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198000.88 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 19800.09 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055831.249966 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178200.79 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 17820.08 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073651.328917 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160380.71 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 16038.07 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089689.399973 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144342.64 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 14434.26 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104123.663923 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129908.38 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 12990.84 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117114.501478 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116917.54 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 11691.75 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128806.255278 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105225.78 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 10522.58 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139328.833697 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94703.21 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 9470.32 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148799.154275 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85232.89 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 8523.29 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157322.442795 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76709.60 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 7670.96 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164993.402463 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69038.64 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 6903.86 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171897.266164 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62134.77 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 6213.48 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178110.743495 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55921.30 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 5592.13 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183702.873093 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50329.17 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 5032.92 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188735.789731 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45296.25 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 4529.62 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193265.414706 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40766.62 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 4076.66 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197342.077182 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36689.96 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3669.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201011.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33020.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204611.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29420.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208211.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25820.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211811.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22220.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215411.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18620.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219011.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15020.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222611.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11420.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226211.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7820.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229811.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4220.97 sec) 12:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233411.073412 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620.97 sec) 12:34:00 [GPUGRID] [rr_sim] 2234032.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982480123.81G/439.78G) 12:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852312.95 12:34:00 [---] pick_jobs_to_run() start 12:34:00 [---] pick_jobs_to_run() end 12:34:00 [---] [rr_sim] end 12:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:34:00 [---] [cpu_sched_debug] preliminary job list: 12:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:34:00 [---] [cpu_sched_debug] final job list: 12:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:34:00 [---] [cpu_sched_debug] enforce_run_list: end 12:34:00 [---] choose_project(): 45240.000000 12:34:00 [iThena.Measurements] doing RPC to get work 12:34:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563159.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390359.09 cpu 2136572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136572.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:34:00 [---] Got 0 tasks 12:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:35:00 [---] [rr_sim] doing sim: CPU sched 12:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982452533.45G/439.78G) 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233969.30 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 223396.93 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223396.930259 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010572.37 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 201057.24 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424454.167493 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809515.14 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 180951.51 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605405.681003 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628563.62 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 162856.36 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768262.043162 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465707.26 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 146570.73 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914832.769105 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319136.53 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 131913.65 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046746.422453 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187222.88 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 118722.29 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165468.710467 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068500.59 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 106850.06 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272318.769680 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961650.53 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 96165.05 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368483.822971 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865485.48 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 86548.55 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1455032.370933 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778936.93 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 77893.69 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532926.064099 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701043.24 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 70104.32 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1603030.387948 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630938.91 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 63093.89 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666124.279413 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567845.02 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 56784.50 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722908.781731 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511060.52 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 51106.05 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1774014.833817 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459954.47 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 45995.45 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1820010.280695 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413959.02 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 41395.90 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861406.182884 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372563.12 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 37256.31 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898662.494855 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335306.81 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 33530.68 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932193.175629 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301776.13 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 30177.61 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962370.788325 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271598.51 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 27159.85 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989530.639752 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244438.66 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 24443.87 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013974.506036 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219994.80 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 21999.48 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035973.985692 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197995.32 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 19799.53 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055773.517382 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178195.79 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 17819.58 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073593.095903 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160376.21 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 16037.62 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089630.716572 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144338.59 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 14433.86 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104064.575174 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129904.73 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 12990.47 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117055.047916 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116914.25 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 11691.43 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128746.473383 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105222.83 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 10522.28 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139268.756304 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94700.55 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 9470.05 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148738.810933 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85230.49 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 8523.05 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157261.860099 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76707.44 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 7670.74 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164932.604348 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69036.70 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 6903.67 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171836.274173 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62133.03 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 6213.30 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178049.577015 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55919.73 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 5591.97 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183641.549572 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50327.75 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 5032.78 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188674.324874 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45294.98 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 4529.50 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193203.822646 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40765.48 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 4076.55 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197280.370641 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36688.93 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3668.89 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200949.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33020.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204549.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29420.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208149.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25820.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211749.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22220.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215349.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18620.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218949.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15020.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222549.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11420.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226149.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7820.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229749.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4220.04 sec) 12:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233349.263836 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620.04 sec) 12:35:00 [GPUGRID] [rr_sim] 2233969.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982452533.45G/439.78G) 12:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852310.22 12:35:00 [---] pick_jobs_to_run() start 12:35:00 [---] pick_jobs_to_run() end 12:35:00 [---] [rr_sim] end 12:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:35:00 [---] [cpu_sched_debug] preliminary job list: 12:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:35:00 [---] [cpu_sched_debug] final job list: 12:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:35:00 [---] [cpu_sched_debug] enforce_run_list: end 12:35:00 [---] choose_project(): 45300.000000 12:35:00 [iThena.Measurements] doing RPC to get work 12:35:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563099.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390299.09 cpu 2136512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136512.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:35:00 [---] Got 0 tasks 12:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:36:00 [---] [rr_sim] doing sim: CPU sched 12:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982424943.09G/439.78G) 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233906.57 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 223390.66 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223390.656571 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010515.91 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 201051.59 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424442.247485 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809464.32 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 180946.43 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605388.679307 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628517.89 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 162851.79 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768240.467948 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465666.10 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 146566.61 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914807.077724 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319099.49 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 131909.95 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046717.026523 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187189.54 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 118718.95 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165435.980441 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068470.59 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 106847.06 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272283.038968 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961623.53 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 96162.35 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368445.391642 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865461.17 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 86546.12 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454991.509049 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778915.06 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 77891.51 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532883.014715 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701023.55 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 70102.36 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602985.369815 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630921.20 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 63092.12 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666077.489404 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567829.08 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 56782.91 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722860.397035 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511046.17 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 51104.62 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773965.013902 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459941.55 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 45994.16 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819959.169083 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413947.40 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 41394.74 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861353.908746 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372552.66 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 37255.27 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898609.174442 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335297.39 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 33529.74 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932138.913569 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301767.65 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 30176.77 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962315.678783 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271590.89 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 27159.09 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989474.767476 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244431.80 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 24443.18 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013917.947299 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219988.62 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 21998.86 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035916.809140 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197989.76 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 19798.98 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055715.784797 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178190.78 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 17819.08 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073534.862889 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160371.70 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 16037.17 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089572.033171 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144334.53 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 14433.45 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104005.486425 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129901.08 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 12990.11 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116995.594353 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116910.97 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 11691.10 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128686.691489 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105219.87 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 10521.99 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139208.678911 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94697.89 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 9469.79 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148678.467591 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85228.10 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 8522.81 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157201.277403 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76705.29 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 7670.53 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164871.806234 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69034.76 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 6903.48 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171775.282181 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62131.28 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 6213.13 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177988.410534 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55918.16 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 5591.82 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183580.226052 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50326.34 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 5032.63 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188612.860018 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45293.71 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 4529.37 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193142.230587 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40764.34 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 4076.43 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197218.664099 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36687.90 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3668.79 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200887.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33019.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204487.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29419.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208087.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25819.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211687.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22219.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215287.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18619.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218887.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15019.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222487.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11419.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226087.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7819.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229687.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4219.11 sec) 12:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233287.454260 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619.11 sec) 12:36:00 [GPUGRID] [rr_sim] 2233906.57: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982424943.09G/439.78G) 12:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852307.48 12:36:00 [---] pick_jobs_to_run() start 12:36:00 [---] pick_jobs_to_run() end 12:36:00 [---] [rr_sim] end 12:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:36:00 [---] [cpu_sched_debug] preliminary job list: 12:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:36:00 [---] [cpu_sched_debug] final job list: 12:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:36:00 [---] [cpu_sched_debug] enforce_run_list: end 12:36:00 [---] choose_project(): 45360.000000 12:36:00 [iThena.Measurements] doing RPC to get work 12:36:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 563039.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390239.09 cpu 2136452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136452.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:36:00 [---] Got 0 tasks 12:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:37:00 [---] [rr_sim] doing sim: CPU sched 12:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982397352.74G/439.78G) 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233843.83 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 223384.38 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223384.382883 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010459.45 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 201045.94 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424430.327477 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809413.50 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 180941.35 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605371.677612 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628472.15 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 162847.22 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768218.892734 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465624.94 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 146562.49 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914781.386343 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319062.44 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 131906.24 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046687.630592 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187156.20 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 118715.62 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165403.250415 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068440.58 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 106844.06 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272247.308257 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961596.52 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 96159.65 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368406.960314 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865436.87 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 86543.69 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454950.647165 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778893.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 77889.32 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532839.965332 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701003.86 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 70100.39 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602940.351681 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630903.48 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 63090.35 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666030.699396 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567813.13 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 56781.31 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722812.012339 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511031.82 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 51103.18 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773915.193988 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459928.63 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 45992.86 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819908.057472 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413935.77 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 41393.58 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861301.634608 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372542.19 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 37254.22 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898555.854030 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335287.97 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 33528.80 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932084.651510 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301759.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 30175.92 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962260.569241 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271583.26 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 27158.33 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989418.895200 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244424.93 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 24442.49 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013861.388563 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219982.44 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 21998.24 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035859.632589 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197984.20 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 19798.42 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055658.052213 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178185.78 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 17818.58 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073476.629875 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160367.20 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 16036.72 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089513.349770 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144330.48 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 14433.05 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103946.397676 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129897.43 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 12989.74 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116936.140791 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116907.69 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 11690.77 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128626.909595 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105216.92 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 10521.69 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139148.601518 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94695.23 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 9469.52 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148618.124249 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85225.70 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 8522.57 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157140.694707 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76703.13 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 7670.31 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164811.008119 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69032.82 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 6903.28 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171714.290190 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62129.54 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 6212.95 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177927.244054 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55916.58 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 5591.66 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183518.902531 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50324.93 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 5032.49 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188551.395161 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45292.43 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 4529.24 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193080.638528 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40763.19 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 4076.32 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197156.957558 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36686.87 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3668.69 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200825.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33018.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204425.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29418.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208025.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25818.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211625.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22218.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215225.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18618.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218825.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15018.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222425.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11418.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226025.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7818.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229625.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4218.18 sec) 12:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233225.644685 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618.18 sec) 12:37:00 [GPUGRID] [rr_sim] 2233843.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982397352.74G/439.78G) 12:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852304.74 12:37:00 [---] pick_jobs_to_run() start 12:37:00 [---] pick_jobs_to_run() end 12:37:00 [---] [rr_sim] end 12:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:37:00 [---] [cpu_sched_debug] preliminary job list: 12:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:37:00 [---] [cpu_sched_debug] final job list: 12:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:37:00 [---] [cpu_sched_debug] enforce_run_list: end 12:37:00 [---] choose_project(): 45420.000000 12:37:00 [iThena.Measurements] doing RPC to get work 12:37:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 562979.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390179.09 cpu 2136392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:37:00 [---] Got 0 tasks 12:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:38:00 [---] [rr_sim] doing sim: CPU sched 12:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982397352.74G/439.78G) 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233843.83 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 223384.38 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223384.382883 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010459.45 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 201045.94 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424430.327477 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809413.50 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 180941.35 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605371.677612 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628472.15 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 162847.22 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768218.892734 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465624.94 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 146562.49 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914781.386343 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319062.44 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 131906.24 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046687.630592 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187156.20 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 118715.62 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165403.250415 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068440.58 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 106844.06 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272247.308257 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961596.52 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 96159.65 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368406.960314 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865436.87 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 86543.69 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454950.647165 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778893.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 77889.32 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532839.965332 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 701003.86 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 70100.39 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602940.351681 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630903.48 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 63090.35 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1666030.699396 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567813.13 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 56781.31 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722812.012339 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511031.82 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 51103.18 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773915.193988 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459928.63 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 45992.86 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819908.057472 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413935.77 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 41393.58 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861301.634608 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372542.19 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 37254.22 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898555.854030 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335287.97 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 33528.80 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932084.651510 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301759.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 30175.92 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962260.569241 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271583.26 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 27158.33 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989418.895200 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244424.93 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 24442.49 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013861.388563 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219982.44 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 21998.24 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035859.632589 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197984.20 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 19798.42 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055658.052213 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178185.78 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 17818.58 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073476.629875 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160367.20 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 16036.72 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089513.349770 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144330.48 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 14433.05 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103946.397676 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129897.43 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 12989.74 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116936.140791 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116907.69 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 11690.77 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128626.909595 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105216.92 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 10521.69 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139148.601518 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94695.23 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 9469.52 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148618.124249 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85225.70 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 8522.57 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157140.694707 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76703.13 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 7670.31 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164811.008119 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69032.82 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 6903.28 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171714.290190 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62129.54 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 6212.95 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177927.244054 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55916.58 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 5591.66 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183518.902531 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50324.93 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 5032.49 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188551.395161 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45292.43 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 4529.24 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193080.638528 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40763.19 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 4076.32 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197156.957558 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36686.87 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3668.69 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200825.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33018.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204425.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29418.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208025.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25818.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211625.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22218.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215225.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18618.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218825.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15018.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222425.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11418.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226025.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7818.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229625.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4218.18 sec) 12:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233225.644685 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618.18 sec) 12:38:00 [GPUGRID] [rr_sim] 2233843.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982397352.74G/439.78G) 12:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852364.74 12:38:00 [---] pick_jobs_to_run() start 12:38:00 [---] pick_jobs_to_run() end 12:38:00 [---] [rr_sim] end 12:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:38:00 [---] [cpu_sched_debug] preliminary job list: 12:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:38:00 [---] [cpu_sched_debug] final job list: 12:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:38:00 [---] [cpu_sched_debug] enforce_run_list: end 12:38:00 [---] choose_project(): 45480.000000 12:38:00 [iThena.Measurements] doing RPC to get work 12:38:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 562919.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390119.09 cpu 2136392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136392.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:38:00 [---] Got 0 tasks 12:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:39:00 [---] [rr_sim] doing sim: CPU sched 12:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982369762.38G/439.78G) 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233781.09 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 223378.11 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223378.109195 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010402.98 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 201040.30 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424418.407470 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809362.68 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 180936.27 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605354.675917 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628426.42 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 162842.64 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768197.317520 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465583.77 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 146558.38 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914755.694963 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319025.40 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 131902.54 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046658.234661 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187122.86 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 118712.29 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165370.520390 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068410.57 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 106841.06 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272211.577545 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961569.51 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 96156.95 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368368.528985 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865412.56 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 86541.26 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454909.785281 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778871.31 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 77887.13 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532796.915948 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700984.18 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 70098.42 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602895.333548 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630885.76 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 63088.58 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665983.909387 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567797.18 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 56779.72 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722763.627643 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511017.46 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 51101.75 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773865.374074 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459915.72 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 45991.57 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819856.945861 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413924.15 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 41392.41 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861249.360469 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372531.73 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 37253.17 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898502.533617 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335278.56 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 33527.86 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932030.389450 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301750.70 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 30175.07 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962205.459699 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271575.63 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 27157.56 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989363.022924 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244418.07 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 24441.81 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013804.829826 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219976.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 21997.63 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035802.456038 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197978.64 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 19797.86 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055600.319629 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178180.77 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 17818.08 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073418.396861 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160362.70 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 16036.27 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089454.666369 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144326.43 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 14432.64 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103887.308927 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129893.78 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 12989.38 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116876.687229 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116904.40 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 11690.44 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128567.127701 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105213.96 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 10521.40 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139088.524125 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94692.57 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 9469.26 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148557.780907 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85223.31 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 8522.33 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157080.112011 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76700.98 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 7670.10 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164750.210005 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69030.88 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 6903.09 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171653.298199 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62127.79 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 6212.78 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177866.077573 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55915.01 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 5591.50 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183457.579011 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50323.51 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 5032.35 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188489.930304 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45291.16 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 4529.12 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193019.046468 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40762.05 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 4076.20 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197095.251016 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36685.84 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3668.58 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200763.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33017.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204363.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29417.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207963.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25817.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211563.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22217.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215163.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18617.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218763.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15017.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222363.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11417.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225963.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7817.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229563.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4217.26 sec) 12:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233163.835109 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 617.26 sec) 12:39:00 [GPUGRID] [rr_sim] 2233781.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982369762.38G/439.78G) 12:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852362.00 12:39:00 [---] pick_jobs_to_run() start 12:39:00 [---] pick_jobs_to_run() end 12:39:00 [---] [rr_sim] end 12:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:39:00 [---] [cpu_sched_debug] preliminary job list: 12:39:00 [---] [cpu_sched_debug] final job list: 12:39:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 12:39:00 [---] [work_fetch] Request work fetch: CPUs idle 12:39:00 [---] [cpu_sched_debug] enforce_run_list: end 12:39:00 [---] choose_project(): 45540.000000 12:39:00 [iThena.Measurements] doing RPC to get work 12:39:00 [iThena.Measurements] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] data_collect_v4_1665176701_461720_0: deadline 562859.09 cpu 1.00 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 390059.09 cpu 2136332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136332.90 [edf_detail] running data_collect_v4_1665176701_461720_0 on cpu 1; finishes at 1.00 [edf_detail] data_collect_v4_1665176701_461720_0 makes deadline; est completion 1.00 12:39:00 [---] Got 0 tasks 12:40:00 [---] [cpu_sched_debug] Request CPU reschedule: job finished 12:40:00 [---] [work_fetch] Request work fetch: job finished 12:40:00 [WUProp@Home] Computation for task data_collect_v4_1665176701_461720_0 finished 12:40:00 [---] [cpu_sched_debug] Request CPU reschedule: handle_finished_apps 12:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:40:00 [---] [rr_sim] doing sim: CPU sched 12:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982369762.38G/439.78G) 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233781.09 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 223378.11 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223378.109195 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010402.98 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 201040.30 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424418.407470 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809362.68 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 180936.27 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605354.675917 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628426.42 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 162842.64 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768197.317520 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465583.77 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 146558.38 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914755.694963 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1319025.40 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 131902.54 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046658.234661 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187122.86 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 118712.29 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165370.520390 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068410.57 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 106841.06 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272211.577545 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961569.51 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 96156.95 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368368.528985 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865412.56 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 86541.26 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454909.785281 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778871.31 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 77887.13 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532796.915948 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700984.18 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 70098.42 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602895.333548 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630885.76 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 63088.58 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665983.909387 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567797.18 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 56779.72 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722763.627643 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511017.46 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 51101.75 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773865.374074 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459915.72 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 45991.57 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819856.945861 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413924.15 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 41392.41 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861249.360469 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372531.73 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 37253.17 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898502.533617 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335278.56 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 33527.86 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932030.389450 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301750.70 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 30175.07 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962205.459699 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271575.63 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 27157.56 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989363.022924 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244418.07 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 24441.81 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013804.829826 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219976.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 21997.63 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035802.456038 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197978.64 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 19797.86 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055600.319629 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178180.77 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 17818.08 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073418.396861 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160362.70 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 16036.27 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089454.666369 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144326.43 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 14432.64 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103887.308927 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129893.78 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 12989.38 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116876.687229 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116904.40 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 11690.44 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128567.127701 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105213.96 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 10521.40 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139088.524125 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94692.57 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 9469.26 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148557.780907 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85223.31 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 8522.33 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157080.112011 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76700.98 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 7670.10 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164750.210005 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69030.88 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 6903.09 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171653.298199 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62127.79 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 6212.78 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177866.077573 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55915.01 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 5591.50 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183457.579011 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50323.51 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 5032.35 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188489.930304 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45291.16 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 4529.12 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193019.046468 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40762.05 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 4076.20 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197095.251016 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36685.84 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3668.58 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200763.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33017.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204363.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29417.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207963.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25817.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211563.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22217.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215163.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18617.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218763.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15017.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222363.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11417.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225963.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7817.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229563.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4217.26 sec) 12:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233163.835109 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 617.26 sec) 12:40:00 [GPUGRID] [rr_sim] 2233781.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982369762.38G/439.78G) 12:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852422.00 12:40:00 [---] pick_jobs_to_run() start 12:40:00 [---] pick_jobs_to_run() end 12:40:00 [---] [rr_sim] end 12:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:40:00 [---] [cpu_sched_debug] preliminary job list: 12:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:40:00 [---] [cpu_sched_debug] final job list: 12:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 12:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:40:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 982369762.38G 12:40:00 [---] [cpu_sched_debug] enforce_run_list: end 12:40:00 [---] choose_project(): 45600.000000 12:40:00 [WUProp@Home] doing RPC to get work 12:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389999.09 cpu 2136332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136332.90 12:40:00 [---] Got 0 tasks 12:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:41:00 [---] [rr_sim] doing sim: CPU sched 12:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982342172.02G/439.78G) 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233718.36 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 223371.84 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223371.835506 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010346.52 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 201034.65 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424406.487462 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809311.87 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 180931.19 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605337.674222 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628380.68 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 162838.07 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768175.742306 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465542.61 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 146554.26 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914730.003582 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318988.35 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 131898.84 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046628.838730 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187089.52 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 118708.95 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165337.790364 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068380.56 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 106838.06 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272175.846834 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961542.51 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 96154.25 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368330.097657 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865388.26 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 86538.83 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454868.923397 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778849.43 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 77884.94 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532753.866564 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700964.49 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 70096.45 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602850.315414 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630868.04 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 63086.80 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665937.119379 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567781.24 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 56778.12 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722715.242947 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511003.11 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 51100.31 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773815.554159 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459902.80 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 45990.28 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819805.834250 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413912.52 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 41391.25 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861197.086331 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372521.27 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 37252.13 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898449.213204 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335269.14 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 33526.91 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931976.127390 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301742.23 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 30174.22 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962150.350158 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271568.00 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 27156.80 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989307.150648 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244411.20 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 24441.12 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013748.271090 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219970.08 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 21997.01 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035745.279487 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197973.08 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 19797.31 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055542.587045 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178175.77 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 17817.58 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073360.163847 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160358.19 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 16035.82 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089395.982968 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144322.37 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 14432.24 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103828.220178 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129890.13 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 12989.01 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116817.233667 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116901.12 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 11690.11 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128507.345806 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105211.01 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 10521.10 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139028.446732 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94689.91 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 9468.99 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148497.437565 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85220.92 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 8522.09 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157019.529315 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76698.83 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 7669.88 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164689.411890 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69028.94 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 6902.89 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171592.306207 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62126.05 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 6212.60 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177804.911093 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55913.44 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 5591.34 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183396.255490 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50322.10 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 5032.21 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188428.465447 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45289.89 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 4528.99 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192957.454409 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40760.90 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 4076.09 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197033.544474 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36684.81 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3668.48 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200702.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33016.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204302.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29416.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207902.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25816.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211502.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22216.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215102.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18616.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218702.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15016.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222302.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11416.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225902.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7816.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229502.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4216.33 sec) 12:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233102.025533 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 616.33 sec) 12:41:00 [GPUGRID] [rr_sim] 2233718.36: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982342172.02G/439.78G) 12:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852419.27 12:41:00 [---] pick_jobs_to_run() start 12:41:00 [---] pick_jobs_to_run() end 12:41:00 [---] [rr_sim] end 12:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:41:00 [---] [cpu_sched_debug] preliminary job list: 12:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 12:41:00 [---] [cpu_sched_debug] final job list: 12:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:41:00 [---] [cpu_sched_debug] enforce_run_list: end 12:41:00 [---] choose_project(): 45660.000000 12:41:00 [WUProp@Home] doing RPC to get work 12:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389939.09 cpu 2136272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136272.90 12:41:00 [---] Got 0 tasks 12:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:42:00 [---] [rr_sim] doing sim: CPU sched 12:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982314581.67G/439.78G) 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233655.62 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 223365.56 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223365.561818 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010290.06 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 201029.01 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424394.567455 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809261.05 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 180926.11 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605320.672527 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628334.95 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 162833.49 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768154.167093 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465501.45 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 146550.15 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914704.312202 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318951.31 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 131895.13 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046599.442800 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187056.18 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 118705.62 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165305.060338 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068350.56 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 106835.06 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272140.116122 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961515.50 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 96151.55 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368291.666328 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865363.95 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 86536.40 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454828.061513 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778827.56 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 77882.76 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532710.817180 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700944.80 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 70094.48 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602805.297280 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630850.32 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 63085.03 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665890.329371 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567765.29 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 56776.53 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722666.858252 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510988.76 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 51098.88 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773765.734245 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459889.88 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 45988.99 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819754.722638 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413900.90 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 41390.09 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861144.812193 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372510.81 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 37251.08 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898395.892792 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335259.73 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 33525.97 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931921.865331 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301733.75 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 30173.38 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962095.240616 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271560.38 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 27156.04 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989251.278372 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244404.34 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 24440.43 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013691.712353 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219963.91 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 21996.39 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035688.102936 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197967.52 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 19796.75 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055484.854461 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178170.76 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 17817.08 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073301.930833 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160353.69 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 16035.37 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089337.299568 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144318.32 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 14431.83 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103769.131429 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129886.49 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 12988.65 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116757.780104 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116897.84 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 11689.78 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128447.563912 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105208.05 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 10520.81 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138968.369339 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94687.25 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 9468.72 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148437.094223 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85218.52 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 8521.85 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156958.946619 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76696.67 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 7669.67 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164628.613775 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69027.00 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 6902.70 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171531.314216 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62124.30 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 6212.43 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177743.744612 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55911.87 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 5591.19 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183334.931969 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50320.69 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 5032.07 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188367.000591 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45288.62 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 4528.86 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192895.862350 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40759.76 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 4075.98 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196971.837933 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36683.78 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3668.38 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200640.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33015.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204240.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29415.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207840.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25815.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211440.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22215.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215040.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18615.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218640.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15015.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222240.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11415.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225840.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7815.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229440.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4215.40 sec) 12:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2233040.215958 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 615.40 sec) 12:42:00 [GPUGRID] [rr_sim] 2233655.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982314581.67G/439.78G) 12:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852416.53 12:42:00 [---] pick_jobs_to_run() start 12:42:00 [---] pick_jobs_to_run() end 12:42:00 [---] [rr_sim] end 12:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:42:00 [---] [cpu_sched_debug] preliminary job list: 12:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:42:00 [---] [cpu_sched_debug] final job list: 12:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:42:00 [---] [cpu_sched_debug] enforce_run_list: end 12:42:00 [---] choose_project(): 45720.000000 12:42:00 [WUProp@Home] doing RPC to get work 12:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389879.09 cpu 2136212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136212.90 12:42:00 [---] Got 0 tasks 12:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:43:00 [---] [rr_sim] doing sim: CPU sched 12:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982286991.31G/439.78G) 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233592.88 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 223359.29 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223359.288130 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010233.59 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 201023.36 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424382.647447 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809210.23 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 180921.02 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605303.670832 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628289.21 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 162828.92 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768132.591879 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465460.29 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 146546.03 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914678.620821 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318914.26 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 131891.43 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046570.046869 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1187022.83 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 118702.28 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165272.330312 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068320.55 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 106832.06 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272104.385411 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961488.50 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 96148.85 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368253.234999 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865339.65 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 86533.96 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454787.199629 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778805.68 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 77880.57 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532667.767796 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700925.11 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 70092.51 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602760.279147 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630832.60 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 63083.26 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665843.539362 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567749.34 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 56774.93 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722618.473556 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510974.41 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 51097.44 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773715.914330 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459876.97 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 45987.70 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819703.611027 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413889.27 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 41388.93 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861092.538054 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372500.34 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 37250.03 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898342.572379 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335250.31 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 33525.03 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931867.603271 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301725.28 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 30172.53 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962040.131074 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271552.75 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 27155.28 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989195.406096 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244397.48 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 24439.75 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013635.153617 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219957.73 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 21995.77 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035630.926385 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197961.95 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 19796.20 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055427.121876 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178165.76 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 17816.58 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073243.697819 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160349.18 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 16034.92 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089278.616167 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144314.27 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 14431.43 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103710.042680 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129882.84 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 12988.28 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116698.326542 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116894.55 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 11689.46 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128387.782018 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105205.10 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 10520.51 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138908.291946 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94684.59 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 9468.46 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148376.750881 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85216.13 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 8521.61 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156898.363923 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76694.52 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 7669.45 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164567.815661 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69025.07 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 6902.51 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171470.322225 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62122.56 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 6212.26 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177682.578132 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55910.30 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 5591.03 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183273.608449 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50319.27 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 5031.93 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188305.535734 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45287.35 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 4528.73 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192834.270290 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40758.61 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 4075.86 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196910.131391 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36682.75 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3668.27 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200578.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33014.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204178.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29414.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207778.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25814.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211378.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22214.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214978.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18614.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218578.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15014.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222178.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11414.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225778.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7814.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229378.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4214.47 sec) 12:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232978.406382 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 614.47 sec) 12:43:00 [GPUGRID] [rr_sim] 2233592.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982286991.31G/439.78G) 12:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852413.79 12:43:00 [---] pick_jobs_to_run() start 12:43:00 [---] pick_jobs_to_run() end 12:43:00 [---] [rr_sim] end 12:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:43:00 [---] [cpu_sched_debug] preliminary job list: 12:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:43:00 [---] [cpu_sched_debug] final job list: 12:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:43:00 [---] [cpu_sched_debug] enforce_run_list: end 12:43:00 [---] choose_project(): 45780.000000 12:43:00 [WUProp@Home] doing RPC to get work 12:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389819.09 cpu 2136152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136152.90 12:43:00 [---] Got 0 tasks 12:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:44:00 [---] [rr_sim] doing sim: CPU sched 12:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982259400.96G/439.78G) 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233530.14 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 223353.01 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223353.014442 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010177.13 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 201017.71 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424370.727439 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809159.42 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 180915.94 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605286.669137 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628243.48 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 162824.35 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768111.016665 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465419.13 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 146541.91 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914652.929440 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318877.21 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 131887.72 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046540.650938 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186989.49 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 118698.95 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165239.600286 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068290.54 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 106829.05 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272068.654699 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961461.49 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 96146.15 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368214.803671 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865315.34 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 86531.53 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454746.337746 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778783.81 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 77878.38 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532624.718413 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700905.43 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 70090.54 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602715.261013 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630814.88 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 63081.49 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665796.749354 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567733.40 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 56773.34 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722570.088860 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510960.06 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 51096.01 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773666.094416 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459864.05 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 45986.41 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819652.499416 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413877.65 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 41387.76 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1861040.263916 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372489.88 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 37248.99 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898289.251966 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335240.89 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 33524.09 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931813.341211 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301716.80 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 30171.68 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961985.021532 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271545.12 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 27154.51 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989139.533820 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244390.61 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 24439.06 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013578.594880 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219951.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 21995.15 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035573.749834 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197956.39 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 19795.64 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055369.389292 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178160.76 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 17816.08 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073185.464805 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160344.68 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 16034.47 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089219.932766 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144310.21 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 14431.02 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103650.953931 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129879.19 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 12987.92 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116638.872980 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116891.27 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 11689.13 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128328.000123 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105202.14 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 10520.21 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138848.214553 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94681.93 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 9468.19 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148316.407539 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85213.74 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 8521.37 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156837.781227 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76692.36 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 7669.24 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164507.017546 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69023.13 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 6902.31 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171409.330233 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62120.81 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 6212.08 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177621.411652 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55908.73 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 5590.87 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183212.284928 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50317.86 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 5031.79 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188244.070877 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45286.07 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 4528.61 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192772.678231 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40757.47 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 4075.75 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196848.424850 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36681.72 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3668.17 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200516.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33013.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204116.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29413.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207716.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25813.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211316.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22213.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214916.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18613.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218516.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15013.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222116.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11413.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225716.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7813.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229316.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4213.55 sec) 12:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232916.596806 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 613.55 sec) 12:44:00 [GPUGRID] [rr_sim] 2233530.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982259400.96G/439.78G) 12:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852411.06 12:44:00 [---] pick_jobs_to_run() start 12:44:00 [---] pick_jobs_to_run() end 12:44:00 [---] [rr_sim] end 12:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:44:00 [---] [cpu_sched_debug] preliminary job list: 12:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:44:00 [---] [cpu_sched_debug] final job list: 12:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:44:00 [---] [cpu_sched_debug] enforce_run_list: end 12:44:00 [---] choose_project(): 45840.000000 12:44:00 [WUProp@Home] doing RPC to get work 12:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389759.09 cpu 2136092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136092.90 12:44:00 [---] Got 0 tasks 12:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:45:00 [---] [rr_sim] doing sim: CPU sched 12:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982231810.60G/439.78G) 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233467.41 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 223346.74 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223346.740754 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010120.67 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 201012.07 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424358.807432 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809108.60 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 180910.86 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605269.667442 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628197.74 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 162819.77 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768089.441451 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465377.97 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 146537.80 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914627.238060 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318840.17 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 131884.02 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046511.255007 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186956.15 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 118695.62 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165206.870260 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068260.54 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 106826.05 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1272032.923988 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961434.48 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 96143.45 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368176.372342 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865291.04 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 86529.10 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454705.475862 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778761.93 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 77876.19 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532581.669029 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700885.74 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 70088.57 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602670.242880 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630797.16 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 63079.72 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665749.959345 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567717.45 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 56771.74 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722521.704164 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510945.70 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 51094.57 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773616.274501 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459851.13 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 45985.11 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819601.387805 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413866.02 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 41386.60 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860987.989778 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372479.42 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 37247.94 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898235.931553 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335231.48 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 33523.15 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931759.079152 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301708.33 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 30170.83 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961929.911990 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271537.50 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 27153.75 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989083.661544 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244383.75 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 24438.37 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013522.036143 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219945.37 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 21994.54 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035516.573283 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197950.83 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 19795.08 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055311.656708 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178155.75 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 17815.58 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073127.231791 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160340.18 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 16034.02 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089161.249365 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144306.16 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 14430.62 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103591.865182 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129875.54 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 12987.55 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116579.419417 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116887.99 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 11688.80 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128268.218229 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105199.19 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 10519.92 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138788.137160 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94679.27 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 9467.93 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148256.064197 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85211.34 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 8521.13 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156777.198531 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76690.21 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 7669.02 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164446.219431 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69021.19 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 6902.12 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171348.338242 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62119.07 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 6211.91 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177560.245171 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55907.16 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 5590.72 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183150.961408 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50316.45 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 5031.64 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188182.606020 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45284.80 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 4528.48 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192711.086172 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40756.32 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 4075.63 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196786.718308 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36680.69 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3668.07 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200454.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33012.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204054.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29412.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207654.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25812.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211254.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22212.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214854.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18612.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218454.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15012.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222054.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11412.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225654.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7812.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229254.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4212.62 sec) 12:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232854.787231 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 612.62 sec) 12:45:00 [GPUGRID] [rr_sim] 2233467.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982231810.60G/439.78G) 12:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852408.32 12:45:00 [---] pick_jobs_to_run() start 12:45:00 [---] pick_jobs_to_run() end 12:45:00 [---] [rr_sim] end 12:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:45:00 [---] [cpu_sched_debug] preliminary job list: 12:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:45:00 [---] [cpu_sched_debug] final job list: 12:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:45:00 [---] [cpu_sched_debug] enforce_run_list: end 12:45:00 [---] choose_project(): 45900.000000 12:45:00 [WUProp@Home] doing RPC to get work 12:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389699.09 cpu 2136032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2136032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2136032.90 12:45:00 [---] Got 0 tasks 12:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:46:00 [---] [rr_sim] doing sim: CPU sched 12:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982204220.24G/439.78G) 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233404.67 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 223340.47 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223340.467065 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010064.20 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 201006.42 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424346.887424 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809057.78 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 180905.78 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605252.665747 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628152.00 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 162815.20 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768067.866238 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465336.80 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 146533.68 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914601.546679 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318803.12 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 131880.31 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046481.859076 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186922.81 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 118692.28 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165174.140234 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068230.53 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 106823.05 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271997.193276 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961407.48 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 96140.75 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368137.941014 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865266.73 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 86526.67 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454664.613978 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778740.06 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 77874.01 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532538.619645 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700866.05 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 70086.61 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602625.224746 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630779.45 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 63077.94 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665703.169337 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567701.50 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 56770.15 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722473.319468 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510931.35 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 51093.14 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773566.454587 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459838.22 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 45983.82 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819550.276193 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413854.39 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 41385.44 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860935.715639 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372468.96 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 37246.90 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898182.611141 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335222.06 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 33522.21 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931704.817092 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301699.85 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 30169.99 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961874.802448 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271529.87 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 27152.99 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989027.789268 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244376.88 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 24437.69 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013465.477407 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219939.19 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 21993.92 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035459.396732 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197945.27 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 19794.53 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055253.924124 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178150.75 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 17815.07 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073068.998777 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160335.67 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 16033.57 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089102.565964 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144302.10 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 14430.21 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103532.776433 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129871.89 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 12987.19 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116519.965855 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116884.70 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 11688.47 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128208.436335 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105196.23 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 10519.62 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138728.059767 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94676.61 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 9467.66 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148195.720855 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85208.95 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 8520.89 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156716.615835 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76688.05 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 7668.81 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164385.421317 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69019.25 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 6901.92 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171287.346250 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62117.32 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 6211.73 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177499.078691 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55905.59 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 5590.56 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183089.637887 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50315.03 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 5031.50 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188121.141164 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45283.53 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 4528.35 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192649.494112 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40755.18 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 4075.52 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196725.011767 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36679.66 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3667.97 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200392.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33011.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203992.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29411.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207592.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25811.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211192.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22211.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214792.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18611.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218392.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15011.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221992.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11411.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225592.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7811.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229192.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4211.69 sec) 12:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232792.977655 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 611.69 sec) 12:46:00 [GPUGRID] [rr_sim] 2233404.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982204220.24G/439.78G) 12:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852405.58 12:46:00 [---] pick_jobs_to_run() start 12:46:00 [---] pick_jobs_to_run() end 12:46:00 [---] [rr_sim] end 12:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:46:00 [---] [cpu_sched_debug] preliminary job list: 12:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:46:00 [---] [cpu_sched_debug] final job list: 12:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:46:00 [---] [cpu_sched_debug] enforce_run_list: end 12:46:00 [---] choose_project(): 45960.000000 12:46:00 [WUProp@Home] doing RPC to get work 12:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389639.09 cpu 2135972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135972.90 12:46:00 [---] Got 0 tasks 12:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:47:00 [---] [rr_sim] doing sim: CPU sched 12:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982176629.89G/439.78G) 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233341.93 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 223334.19 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223334.193377 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2010007.74 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 201000.77 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424334.967416 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1809006.97 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 180900.70 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605235.664052 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628106.27 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 162810.63 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768046.291024 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465295.64 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 146529.56 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914575.855298 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318766.08 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 131876.61 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046452.463146 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186889.47 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 118688.95 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165141.410208 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068200.52 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 106820.05 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271961.462564 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961380.47 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 96138.05 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368099.509685 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865242.42 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 86524.24 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454623.752094 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778718.18 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 77871.82 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532495.570261 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700846.36 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 70084.64 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602580.206612 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630761.73 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 63076.17 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665656.379328 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567685.55 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 56768.56 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722424.934772 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510917.00 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 51091.70 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773516.634672 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459825.30 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 45982.53 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819499.164582 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413842.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 41384.28 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860883.441501 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372458.49 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 37245.85 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898129.290728 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335212.64 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 33521.26 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931650.555032 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301691.38 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 30169.14 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961819.692906 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271522.24 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 27152.22 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988971.916993 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244370.02 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 24437.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013408.918670 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219933.02 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 21993.30 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035402.220180 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197939.71 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 19793.97 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055196.191539 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178145.74 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 17814.57 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073010.765763 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160331.17 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 16033.12 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089043.882563 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144298.05 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 14429.81 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103473.687684 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129868.25 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 12986.82 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116460.512293 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116881.42 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 11688.14 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128148.654441 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105193.28 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 10519.33 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138667.982374 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94673.95 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 9467.40 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148135.377513 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85206.56 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 8520.66 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156656.033139 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76685.90 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 7668.59 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164324.623202 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69017.31 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 6901.73 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171226.354259 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62115.58 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 6211.56 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177437.912210 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55904.02 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 5590.40 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183028.314366 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50313.62 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 5031.36 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188059.676307 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45282.26 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 4528.23 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192587.902053 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40754.03 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 4075.40 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196663.305225 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36678.63 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3667.86 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200331.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33010.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203931.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29410.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207531.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25810.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211131.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22210.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214731.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18610.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218331.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15010.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221931.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11410.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225531.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7810.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229131.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4210.77 sec) 12:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232731.168080 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 610.77 sec) 12:47:00 [GPUGRID] [rr_sim] 2233341.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982176629.89G/439.78G) 12:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852402.85 12:47:00 [---] pick_jobs_to_run() start 12:47:00 [---] pick_jobs_to_run() end 12:47:00 [---] [rr_sim] end 12:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:47:00 [---] [cpu_sched_debug] preliminary job list: 12:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:47:00 [---] [cpu_sched_debug] final job list: 12:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:47:00 [---] [cpu_sched_debug] enforce_run_list: end 12:47:00 [---] choose_project(): 46020.000000 12:47:00 [WUProp@Home] doing RPC to get work 12:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389579.09 cpu 2135912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135912.90 12:47:00 [---] Got 0 tasks 12:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:48:00 [---] [rr_sim] doing sim: CPU sched 12:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982149039.53G/439.78G) 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233279.20 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 223327.92 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223327.919689 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009951.28 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 200995.13 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424323.047409 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808956.15 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 180895.61 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605218.662357 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628060.53 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 162806.05 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768024.715810 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465254.48 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 146525.45 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914550.163918 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318729.03 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 131872.90 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046423.067215 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186856.13 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 118685.61 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165108.680182 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068170.52 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 106817.05 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271925.731853 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961353.47 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 96135.35 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368061.078356 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865218.12 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 86521.81 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454582.890210 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778696.31 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 77869.63 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532452.520878 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700826.68 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 70082.67 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602535.188479 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630744.01 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 63074.40 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665609.589320 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567669.61 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 56766.96 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722376.550077 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510902.65 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 51090.26 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773466.814758 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459812.38 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 45981.24 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819448.052971 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413831.14 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 41383.11 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860831.167363 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372448.03 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 37244.80 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898075.970315 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335203.23 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 33520.32 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931596.292973 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301682.90 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 30168.29 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961764.583364 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271514.61 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 27151.46 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988916.044717 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244363.15 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 24436.32 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013352.359934 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219926.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 21992.68 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035345.043629 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197934.15 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 19793.42 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055138.458955 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178140.74 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 17814.07 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072952.532749 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160326.66 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 16032.67 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088985.199163 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144294.00 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 14429.40 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103414.598935 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129864.60 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 12986.46 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116401.058731 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116878.14 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 11687.81 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128088.872546 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105190.32 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 10519.03 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138607.904981 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94671.29 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 9467.13 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148075.034171 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85204.16 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 8520.42 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156595.450443 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76683.75 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 7668.37 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164263.825088 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69015.37 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 6901.54 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171165.362268 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62113.83 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 6211.38 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177376.745730 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55902.45 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 5590.25 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182966.990846 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50312.21 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 5031.22 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187998.211450 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45280.99 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 4528.10 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192526.309994 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40752.89 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 4075.29 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196601.598683 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36677.60 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3667.76 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200269.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33009.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203869.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29409.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207469.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25809.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211069.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22209.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214669.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18609.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218269.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15009.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221869.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11409.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225469.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7809.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229069.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4209.84 sec) 12:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232669.358504 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 609.84 sec) 12:48:00 [GPUGRID] [rr_sim] 2233279.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982149039.53G/439.78G) 12:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852400.11 12:48:00 [---] pick_jobs_to_run() start 12:48:00 [---] pick_jobs_to_run() end 12:48:00 [---] [rr_sim] end 12:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:48:00 [---] [cpu_sched_debug] preliminary job list: 12:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:48:00 [---] [cpu_sched_debug] final job list: 12:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:48:00 [---] [cpu_sched_debug] enforce_run_list: end 12:48:00 [---] choose_project(): 46080.000000 12:48:00 [WUProp@Home] doing RPC to get work 12:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389519.09 cpu 2135852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135852.90 12:48:00 [---] Got 0 tasks 12:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:49:00 [---] [rr_sim] doing sim: CPU sched 12:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982121449.18G/439.78G) 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233216.46 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 223321.65 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223321.646001 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009894.81 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 200989.48 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424311.127401 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808905.33 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 180890.53 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605201.660662 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1628014.80 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 162801.48 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 768003.140596 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465213.32 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 146521.33 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914524.472537 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318691.99 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 131869.20 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046393.671284 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186822.79 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 118682.28 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165075.950156 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068140.51 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 106814.05 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271890.001141 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961326.46 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 96132.65 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1368022.647028 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865193.81 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 86519.38 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454542.028326 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778674.43 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 77867.44 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532409.471494 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700806.99 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 70080.70 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602490.170345 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630726.29 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 63072.63 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665562.799311 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567653.66 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 56765.37 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722328.165381 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510888.29 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 51088.83 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773416.994843 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459799.47 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 45979.95 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819396.941360 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413819.52 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 41381.95 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860778.893224 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372437.57 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 37243.76 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898022.649903 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335193.81 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 33519.38 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931542.030913 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301674.43 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 30167.44 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961709.473822 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271506.99 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 27150.70 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988860.172441 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244356.29 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 24435.63 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013295.801197 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219920.66 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 21992.07 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035287.867078 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197928.59 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 19792.86 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055080.726371 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178135.73 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 17813.57 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072894.299735 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160322.16 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 16032.22 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088926.515762 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144289.94 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 14428.99 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103355.510186 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129860.95 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 12986.09 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116341.605168 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116874.85 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 11687.49 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128029.090652 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105187.37 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 10518.74 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138547.827587 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94668.63 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 9466.86 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148014.690829 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85201.77 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 8520.18 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156534.867747 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76681.59 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 7668.16 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164203.026973 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69013.43 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 6901.34 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171104.370276 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62112.09 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 6211.21 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177315.579249 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55900.88 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 5590.09 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182905.667325 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50310.79 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 5031.08 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187936.746593 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45279.71 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 4527.97 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192464.717935 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40751.74 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 4075.17 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196539.892142 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36676.57 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3667.66 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200207.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33008.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203807.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29408.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207407.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25808.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211007.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22208.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214607.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18608.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218207.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15008.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221807.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11408.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225407.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7808.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229007.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4208.91 sec) 12:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232607.548928 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 608.91 sec) 12:49:00 [GPUGRID] [rr_sim] 2233216.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982121449.18G/439.78G) 12:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852397.37 12:49:00 [---] pick_jobs_to_run() start 12:49:00 [---] pick_jobs_to_run() end 12:49:00 [---] [rr_sim] end 12:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:49:00 [---] [cpu_sched_debug] preliminary job list: 12:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:49:00 [---] [cpu_sched_debug] final job list: 12:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:49:00 [---] [cpu_sched_debug] enforce_run_list: end 12:49:00 [---] choose_project(): 46140.000000 12:49:00 [WUProp@Home] doing RPC to get work 12:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389459.09 cpu 2135792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135792.90 12:49:00 [---] Got 0 tasks 12:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:50:00 [---] [rr_sim] doing sim: CPU sched 12:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982093858.82G/439.78G) 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233153.72 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 223315.37 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223315.372312 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009838.35 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 200983.84 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424299.207394 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808854.52 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 180885.45 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605184.658967 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627969.06 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 162796.91 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767981.565382 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465172.16 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 146517.22 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914498.781157 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318654.94 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 131865.49 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046364.275353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186789.45 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 118678.94 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165043.220130 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068110.50 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 106811.05 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271854.270430 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961299.45 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 96129.95 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367984.215699 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865169.51 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 86516.95 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454501.166442 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778652.56 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 77865.26 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532366.422110 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700787.30 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 70078.73 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602445.152211 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630708.57 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 63070.86 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665516.009303 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567637.71 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 56763.77 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722279.780685 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510873.94 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 51087.39 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773367.174929 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459786.55 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 45978.65 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819345.829748 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413807.89 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 41380.79 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860726.619086 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372427.10 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 37242.71 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897969.329490 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335184.39 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 33518.44 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931487.768853 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301665.95 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 30166.60 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961654.364280 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271499.36 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 27149.94 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988804.300165 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244349.42 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 24434.94 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013239.242461 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219914.48 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 21991.45 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035230.690527 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197923.03 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 19792.30 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055022.993787 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178130.73 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 17813.07 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072836.066721 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160317.66 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 16031.77 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088867.832361 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144285.89 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 14428.59 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103296.421437 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129857.30 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 12985.73 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116282.151606 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116871.57 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 11687.16 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127969.308758 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105184.41 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 10518.44 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138487.750194 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94665.97 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 9466.60 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147954.347487 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85199.38 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 8519.94 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156474.285051 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76679.44 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 7667.94 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164142.228858 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69011.49 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 6901.15 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171043.378285 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62110.34 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 6211.03 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177254.412769 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55899.31 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 5589.93 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182844.343804 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50309.38 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 5030.94 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187875.281736 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45278.44 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 4527.84 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192403.125875 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40750.60 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 4075.06 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196478.185600 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36675.54 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3667.55 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200145.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33007.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203745.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29407.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207345.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25807.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210945.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22207.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214545.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18607.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218145.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15007.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221745.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11407.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225345.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7807.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228945.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4207.98 sec) 12:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232545.739353 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 607.98 sec) 12:50:00 [GPUGRID] [rr_sim] 2233153.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982093858.82G/439.78G) 12:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852394.64 12:50:00 [---] pick_jobs_to_run() start 12:50:00 [---] pick_jobs_to_run() end 12:50:00 [---] [rr_sim] end 12:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:50:00 [---] [cpu_sched_debug] preliminary job list: 12:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:50:00 [---] [cpu_sched_debug] final job list: 12:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:50:00 [---] [cpu_sched_debug] enforce_run_list: end 12:50:00 [---] choose_project(): 46200.000000 12:50:00 [WUProp@Home] doing RPC to get work 12:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389399.09 cpu 2135732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135732.90 12:50:00 [---] Got 0 tasks 12:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:53:00 [---] [rr_sim] doing sim: CPU sched 12:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982066268.46G/439.78G) 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233090.99 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 223309.10 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223309.098624 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009781.89 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 200978.19 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424287.287386 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808803.70 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 180880.37 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605167.657272 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627923.33 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 162792.33 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767959.990169 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465131.00 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 146513.10 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914473.089776 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318617.90 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 131861.79 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046334.879423 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186756.11 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 118675.61 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1165010.490105 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068080.50 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 106808.05 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271818.539718 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961272.45 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 96127.24 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367945.784371 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865145.20 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 86514.52 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454460.304558 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778630.68 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 77863.07 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532323.372726 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700767.61 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 70076.76 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602400.134078 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630690.85 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 63069.09 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665469.219294 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567621.77 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 56762.18 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722231.395989 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510859.59 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 51085.96 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773317.355014 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459773.63 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 45977.36 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819294.718137 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413796.27 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 41379.63 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860674.344948 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372416.64 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 37241.66 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897916.009077 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335174.98 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 33517.50 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931433.506794 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301657.48 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 30165.75 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961599.254738 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271491.73 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 27149.17 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988748.427889 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244342.56 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 24434.26 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013182.683724 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219908.30 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 21990.83 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035173.513976 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197917.47 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 19791.75 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054965.261203 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178125.73 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 17812.57 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072777.833707 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160313.15 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 16031.32 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088809.148960 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144281.84 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 14428.18 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103237.332688 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129853.65 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 12985.37 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116222.698044 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116868.29 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 11686.83 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127909.526863 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105181.46 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 10518.15 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138427.672801 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94663.31 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 9466.33 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147894.004145 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85196.98 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 8519.70 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156413.702355 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76677.28 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 7667.73 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164081.430744 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69009.56 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 6900.96 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170982.386294 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62108.60 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 6210.86 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177193.246288 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55897.74 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 5589.77 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182783.020284 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50307.97 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 5030.80 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187813.816880 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45277.17 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 4527.72 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192341.533816 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40749.45 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 4074.95 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196416.479059 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36674.51 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3667.45 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200083.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33007.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203683.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29407.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207283.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25807.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210883.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22207.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214483.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18607.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218083.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15007.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221683.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11407.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225283.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7807.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228883.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4207.06 sec) 12:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232483.929777 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 607.06 sec) 12:53:00 [GPUGRID] [rr_sim] 2233090.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982066268.46G/439.78G) 12:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852511.90 12:53:00 [---] pick_jobs_to_run() start 12:53:00 [---] pick_jobs_to_run() end 12:53:00 [---] [rr_sim] end 12:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:53:00 [---] [cpu_sched_debug] preliminary job list: 12:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:53:00 [---] [cpu_sched_debug] final job list: 12:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:53:00 [---] [cpu_sched_debug] enforce_run_list: end 12:53:00 [---] choose_project(): 46380.000000 12:53:00 [WUProp@Home] doing RPC to get work 12:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389219.09 cpu 2135672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135672.90 12:53:00 [---] Got 0 tasks 12:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:54:00 [---] [rr_sim] doing sim: CPU sched 12:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982038678.11G/439.78G) 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2233028.25 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 223302.82 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223302.824936 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009725.42 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 200972.54 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424275.367378 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808752.88 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 180875.29 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605150.655577 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627877.59 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 162787.76 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767938.414955 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465089.83 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 146508.98 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914447.398395 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318580.85 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 131858.09 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046305.483492 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186722.77 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 118672.28 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164977.760079 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068050.49 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 106805.05 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271782.809007 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961245.44 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 96124.54 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367907.353042 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865120.90 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 86512.09 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454419.442674 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778608.81 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 77860.88 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532280.323342 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700747.93 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 70074.79 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602355.115944 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630673.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 63067.31 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665422.429286 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567605.82 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 56760.58 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722183.011293 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510845.24 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 51084.52 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773267.535100 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459760.71 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 45976.07 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819243.606526 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413784.64 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 41378.46 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860622.070809 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372406.18 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 37240.62 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897862.688664 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335165.56 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 33516.56 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931379.244734 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301649.00 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 30164.90 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961544.145197 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271484.10 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 27148.41 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988692.555613 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244335.69 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 24433.57 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013126.124988 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219902.12 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 21990.21 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035116.337425 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197911.91 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 19791.19 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054907.528618 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178120.72 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 17812.07 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072719.600692 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160308.65 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 16030.86 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088750.465559 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144277.78 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 14427.78 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103178.243939 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129850.01 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 12985.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116163.244481 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116865.00 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 11686.50 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127849.744969 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105178.50 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 10517.85 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138367.595408 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94660.65 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 9466.07 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147833.660803 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85194.59 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 8519.46 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156353.119659 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76675.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 7667.51 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164020.632629 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69007.62 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 6900.76 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170921.394302 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62106.86 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 6210.69 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177132.079808 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55896.17 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 5589.62 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182721.696763 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50306.55 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 5030.66 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187752.352023 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45275.90 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 4527.59 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192279.941757 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40748.31 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 4074.83 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196354.772517 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36673.48 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3667.35 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200022.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33006.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203622.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29406.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207222.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25806.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210822.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22206.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214422.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18606.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218022.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15006.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221622.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11406.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225222.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7806.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228822.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4206.13 sec) 12:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232422.120201 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 606.13 sec) 12:54:00 [GPUGRID] [rr_sim] 2233028.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982038678.11G/439.78G) 12:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852509.16 12:54:00 [---] pick_jobs_to_run() start 12:54:00 [---] pick_jobs_to_run() end 12:54:00 [---] [rr_sim] end 12:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:54:00 [---] [cpu_sched_debug] preliminary job list: 12:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:54:00 [---] [cpu_sched_debug] final job list: 12:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:54:00 [---] [cpu_sched_debug] enforce_run_list: end 12:54:00 [---] choose_project(): 46440.000000 12:54:00 [WUProp@Home] doing RPC to get work 12:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389159.09 cpu 2135612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135612.90 12:54:00 [---] Got 0 tasks 12:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:55:00 [---] [rr_sim] doing sim: CPU sched 12:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (982011087.75G/439.78G) 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232965.51 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 223296.55 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223296.551248 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009668.96 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 200966.90 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424263.447371 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808702.07 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 180870.21 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605133.653881 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627831.86 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 162783.19 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767916.839741 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465048.67 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 146504.87 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914421.707015 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318543.81 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 131854.38 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046276.087561 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186689.42 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 118668.94 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164945.030053 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1068020.48 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 106802.05 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271747.078295 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961218.43 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 96121.84 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367868.921714 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865096.59 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 86509.66 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454378.580790 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778586.93 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 77858.69 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532237.273959 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700728.24 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 70072.82 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602310.097811 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630655.41 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 63065.54 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665375.639277 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567589.87 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 56758.99 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722134.626597 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510830.89 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 51083.09 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773217.715185 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459747.80 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 45974.78 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819192.494915 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413773.02 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 41377.30 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860569.796671 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372395.72 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 37239.57 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897809.368252 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335156.14 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 33515.61 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931324.982674 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301640.53 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 30164.05 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961489.035655 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271476.48 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 27147.65 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988636.683337 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244328.83 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 24432.88 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013069.566251 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219895.95 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 21989.59 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035059.160874 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197906.35 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 19790.64 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054849.796034 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178115.72 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 17811.57 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072661.367678 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160304.14 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 16030.41 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088691.782158 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144273.73 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 14427.37 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103119.155190 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129846.36 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 12984.64 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116103.790919 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116861.72 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 11686.17 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127789.963075 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105175.55 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 10517.55 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138307.518015 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94657.99 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 9465.80 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147773.317461 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85192.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 8519.22 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156292.536963 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76672.98 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 7667.30 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163959.834515 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69005.68 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 6900.57 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170860.402311 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62105.11 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 6210.51 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177070.913328 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55894.60 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 5589.46 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182660.373243 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50305.14 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 5030.51 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187690.887166 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45274.63 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 4527.46 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192218.349697 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40747.16 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 4074.72 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196293.065975 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36672.45 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3667.24 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199960.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33005.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203560.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29405.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207160.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25805.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210760.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22205.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214360.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18605.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217960.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15005.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221560.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11405.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225160.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7805.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228760.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4205.20 sec) 12:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232360.310626 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 605.20 sec) 12:55:00 [GPUGRID] [rr_sim] 2232965.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (982011087.75G/439.78G) 12:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852506.43 12:55:00 [---] pick_jobs_to_run() start 12:55:00 [---] pick_jobs_to_run() end 12:55:00 [---] [rr_sim] end 12:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:55:00 [---] [cpu_sched_debug] preliminary job list: 12:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:55:00 [---] [cpu_sched_debug] final job list: 12:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:55:00 [---] [cpu_sched_debug] enforce_run_list: end 12:55:00 [---] choose_project(): 46500.000000 12:55:00 [WUProp@Home] doing RPC to get work 12:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389099.09 cpu 2135552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135552.90 12:55:00 [---] Got 0 tasks 12:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:56:00 [---] [rr_sim] doing sim: CPU sched 12:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981983497.39G/439.78G) 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232902.78 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 223290.28 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223290.277560 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009612.50 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 200961.25 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424251.527363 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808651.25 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 180865.12 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605116.652186 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627786.12 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 162778.61 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767895.264527 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1465007.51 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 146500.75 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914396.015634 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318506.76 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 131850.68 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046246.691630 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186656.08 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 118665.61 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164912.300027 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067990.48 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 106799.05 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271711.347584 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961191.43 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 96119.14 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367830.490385 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865072.29 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 86507.23 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454337.718906 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778565.06 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 77856.51 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532194.224575 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700708.55 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 70070.86 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602265.079677 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630637.70 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 63063.77 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665328.849269 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567573.93 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 56757.39 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722086.241902 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510816.53 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 51081.65 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773167.895271 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459734.88 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 45973.49 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819141.383303 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413761.39 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 41376.14 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860517.522533 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372385.25 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 37238.53 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897756.047839 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335146.73 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 33514.67 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931270.720615 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301632.05 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 30163.21 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961433.926113 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271468.85 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 27146.88 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988580.811061 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244321.96 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 24432.20 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013013.007514 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219889.77 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 21988.98 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035001.984323 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197900.79 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 19790.08 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054792.063450 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178110.71 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 17811.07 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072603.134664 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160299.64 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 16029.96 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088633.098758 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144269.68 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 14426.97 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103060.066441 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129842.71 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 12984.27 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116044.337357 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116858.44 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 11685.84 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127730.181181 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105172.59 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 10517.26 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138247.440622 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94655.33 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 9465.53 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147712.974120 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85189.80 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 8518.98 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156231.954267 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76670.82 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 7667.08 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163899.036400 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69003.74 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 6900.37 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170799.410320 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62103.37 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 6210.34 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177009.746847 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55893.03 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 5589.30 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182599.049722 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50303.73 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 5030.37 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187629.422309 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45273.35 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 4527.34 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192156.757638 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40746.02 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 4074.60 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196231.359434 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36671.42 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3667.14 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199898.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33004.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203498.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29404.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207098.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25804.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210698.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22204.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214298.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18604.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217898.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15004.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221498.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11404.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225098.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7804.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228698.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4204.27 sec) 12:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232298.501050 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 604.27 sec) 12:56:00 [GPUGRID] [rr_sim] 2232902.78: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981983497.39G/439.78G) 12:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852503.69 12:56:00 [---] pick_jobs_to_run() start 12:56:00 [---] pick_jobs_to_run() end 12:56:00 [---] [rr_sim] end 12:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:56:00 [---] [cpu_sched_debug] preliminary job list: 12:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:56:00 [---] [cpu_sched_debug] final job list: 12:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:56:00 [---] [cpu_sched_debug] enforce_run_list: end 12:56:00 [---] choose_project(): 46560.000000 12:56:00 [WUProp@Home] doing RPC to get work 12:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 389039.09 cpu 2135492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135492.90 12:56:00 [---] Got 0 tasks 12:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:57:00 [---] [rr_sim] doing sim: CPU sched 12:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981955907.04G/439.78G) 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232840.04 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 223284.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223284.003871 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009556.03 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 200955.60 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424239.607356 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808600.43 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 180860.04 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605099.650491 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627740.39 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 162774.04 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767873.689314 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464966.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 146496.63 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914370.324254 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318469.71 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 131846.97 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046217.295700 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186622.74 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 118662.27 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164879.570001 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067960.47 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 106796.05 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271675.616872 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961164.42 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 96116.44 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367792.059056 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865047.98 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 86504.80 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454296.857022 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778543.18 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 77854.32 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532151.175191 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700688.86 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 70068.89 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602220.061543 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630619.98 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 63062.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665282.059260 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567557.98 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 56755.80 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1722037.857206 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510802.18 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 51080.22 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773118.075356 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459721.96 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 45972.20 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819090.271692 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413749.77 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 41374.98 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860465.248394 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372374.79 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 37237.48 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897702.727426 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335137.31 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 33513.73 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931216.458555 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301623.58 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 30162.36 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961378.816571 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271461.22 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 27146.12 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988524.938785 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244315.10 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 24431.51 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012956.448778 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219883.59 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 21988.36 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034944.807771 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197895.23 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 19789.52 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054734.330866 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178105.71 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 17810.57 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072544.901650 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160295.14 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 16029.51 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088574.415357 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144265.62 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 14426.56 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103000.977692 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129839.06 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 12983.91 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115984.883795 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116855.15 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 11685.52 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127670.399286 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105169.64 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 10516.96 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138187.363229 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94652.68 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 9465.27 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147652.630778 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85187.41 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 8518.74 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156171.371571 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76668.67 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 7666.87 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163838.238285 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69001.80 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 6900.18 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170738.418328 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62101.62 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 6210.16 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176948.580367 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55891.46 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 5589.15 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182537.726201 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50302.31 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 5030.23 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187567.957453 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45272.08 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 4527.21 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192095.165579 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40744.87 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 4074.49 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196169.652892 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36670.39 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3667.04 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199836.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33003.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203436.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29403.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207036.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25803.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210636.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22203.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214236.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18603.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217836.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15003.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221436.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11403.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225036.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7803.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228636.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4203.35 sec) 12:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232236.691474 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 603.35 sec) 12:57:00 [GPUGRID] [rr_sim] 2232840.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981955907.04G/439.78G) 12:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852500.95 12:57:00 [---] pick_jobs_to_run() start 12:57:00 [---] pick_jobs_to_run() end 12:57:00 [---] [rr_sim] end 12:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:57:00 [---] [cpu_sched_debug] preliminary job list: 12:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:57:00 [---] [cpu_sched_debug] final job list: 12:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:57:00 [---] [cpu_sched_debug] enforce_run_list: end 12:57:00 [---] choose_project(): 46620.000000 12:57:00 [WUProp@Home] doing RPC to get work 12:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388979.09 cpu 2135432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135432.90 12:57:00 [---] Got 0 tasks 12:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:58:00 [---] [rr_sim] doing sim: CPU sched 12:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981928316.68G/439.78G) 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232777.30 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 223277.73 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223277.730183 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009499.57 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 200949.96 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424227.687348 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808549.61 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 180854.96 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605082.648796 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627694.65 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 162769.47 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767852.114100 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464925.19 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 146492.52 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914344.632873 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318432.67 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 131843.27 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046187.899769 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186589.40 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 118658.94 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164846.839975 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067930.46 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 106793.05 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271639.886161 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961137.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 96113.74 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367753.627728 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 865023.67 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 86502.37 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454255.995138 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778521.31 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 77852.13 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532108.125807 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700669.18 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 70066.92 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602175.043410 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630602.26 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 63060.23 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665235.269252 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567542.03 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 56754.20 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721989.472510 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510787.83 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 51078.78 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773068.255442 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459709.05 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 45970.90 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1819039.160081 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413738.14 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 41373.81 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860412.974256 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372364.33 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 37236.43 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897649.407013 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335127.89 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 33512.79 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931162.196495 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301615.11 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 30161.51 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961323.707029 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271453.59 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 27145.36 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988469.066509 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244308.24 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 24430.82 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012899.890041 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219877.41 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 21987.74 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034887.631220 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197889.67 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 19788.97 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054676.598281 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178100.70 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 17810.07 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072486.668636 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160290.63 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 16029.06 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088515.731956 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144261.57 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 14426.16 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102941.888943 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129835.41 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 12983.54 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115925.430232 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116851.87 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 11685.19 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127610.617392 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105166.68 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 10516.67 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138127.285836 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94650.02 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 9465.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147592.287436 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85185.01 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 8518.50 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156110.788875 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76666.51 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 7666.65 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163777.440171 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68999.86 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 6899.99 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170677.426337 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62099.88 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 6209.99 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176887.413886 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55889.89 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 5588.99 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182476.402681 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50300.90 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 5030.09 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187506.492596 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45270.81 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 4527.08 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192033.573519 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40743.73 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 4074.37 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196107.946351 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36669.36 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3666.94 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199774.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33002.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203374.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29402.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206974.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25802.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210574.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22202.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214174.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18602.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217774.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15002.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221374.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11402.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224974.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7802.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228574.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4202.42 sec) 12:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232174.881899 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 602.42 sec) 12:58:00 [GPUGRID] [rr_sim] 2232777.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981928316.68G/439.78G) 12:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852498.21 12:58:00 [---] pick_jobs_to_run() start 12:58:00 [---] pick_jobs_to_run() end 12:58:00 [---] [rr_sim] end 12:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:58:00 [---] [cpu_sched_debug] preliminary job list: 12:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:58:00 [---] [cpu_sched_debug] final job list: 12:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:58:00 [---] [cpu_sched_debug] enforce_run_list: end 12:58:00 [---] choose_project(): 46680.000000 12:58:00 [WUProp@Home] doing RPC to get work 12:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388919.09 cpu 2135372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135372.90 12:58:00 [---] Got 0 tasks 12:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 12:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 12:59:00 [---] [rr_sim] doing sim: CPU sched 12:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981900726.33G/439.78G) 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232714.56 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 223271.46 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223271.456495 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009443.11 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 200944.31 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424215.767340 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808498.80 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 180849.88 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605065.647101 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627648.92 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 162764.89 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767830.538886 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464884.03 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 146488.40 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914318.941492 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318395.62 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 131839.56 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046158.503838 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186556.06 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 118655.61 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164814.109949 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067900.46 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 106790.05 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271604.155449 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961110.41 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 96111.04 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367715.196399 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864999.37 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 86499.94 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454215.133254 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778499.43 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 77849.94 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532065.076424 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700649.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 70064.95 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602130.025276 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630584.54 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 63058.45 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665188.479243 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567526.09 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 56752.61 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721941.087814 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510773.48 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 51077.35 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1773018.435528 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459696.13 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 45969.61 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818988.048470 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413726.52 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 41372.65 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860360.700118 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372353.86 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 37235.39 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897596.086601 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335118.48 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 33511.85 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931107.934436 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301606.63 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 30160.66 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961268.597487 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271445.97 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 27144.60 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988413.194233 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244301.37 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 24430.14 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012843.331305 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219871.23 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 21987.12 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034830.454669 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197884.11 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 19788.41 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054618.865697 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178095.70 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 17809.57 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072428.435622 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160286.13 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 16028.61 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088457.048555 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144257.52 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 14425.75 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102882.800194 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129831.76 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 12983.18 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115865.976670 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116848.59 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 11684.86 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127550.835498 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105163.73 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 10516.37 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138067.208443 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94647.36 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 9464.74 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147531.944094 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85182.62 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 8518.26 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156050.206179 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76664.36 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 7666.44 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163716.642056 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68997.92 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 6899.79 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170616.434345 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62098.13 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 6209.81 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176826.247406 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55888.32 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 5588.83 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182415.079160 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50299.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 5029.95 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187445.027739 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45269.54 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 4526.95 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191971.981460 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40742.58 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 4074.26 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196046.239809 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36668.33 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3666.83 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199713.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33001.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203313.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29401.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206913.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25801.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210513.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22201.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214113.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18601.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217713.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15001.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221313.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11401.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224913.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7801.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228513.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4201.49 sec) 12:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232113.072323 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 601.49 sec) 12:59:00 [GPUGRID] [rr_sim] 2232714.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981900726.33G/439.78G) 12:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852495.48 12:59:00 [---] pick_jobs_to_run() start 12:59:00 [---] pick_jobs_to_run() end 12:59:00 [---] [rr_sim] end 12:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 12:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 12:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 12:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 12:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 12:59:00 [---] [cpu_sched_debug] preliminary job list: 12:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:59:00 [---] [cpu_sched_debug] final job list: 12:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 12:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 12:59:00 [---] [cpu_sched_debug] enforce_run_list: end 12:59:00 [---] choose_project(): 46740.000000 12:59:00 [WUProp@Home] doing RPC to get work 12:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388859.09 cpu 2135312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135312.90 12:59:00 [---] Got 0 tasks 13:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:00:00 [---] [rr_sim] doing sim: CPU sched 13:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981873135.97G/439.78G) 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232651.83 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 223265.18 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223265.182807 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009386.65 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 200938.66 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424203.847333 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808447.98 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 180844.80 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605048.645406 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627603.18 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 162760.32 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767808.963672 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464842.86 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 146484.29 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914293.250112 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318358.58 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 131835.86 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046129.107907 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186522.72 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 118652.27 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164781.379923 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067870.45 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 106787.04 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271568.424738 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961083.40 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 96108.34 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367676.765071 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864975.06 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 86497.51 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454174.271370 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778477.56 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 77847.76 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1532022.027040 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700629.80 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 70062.98 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602085.007143 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630566.82 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 63056.68 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665141.689235 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567510.14 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 56751.01 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721892.703118 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510759.12 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 51075.91 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772968.615613 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459683.21 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 45968.32 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818936.936858 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413714.89 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 41371.49 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860308.425979 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372343.40 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 37234.34 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897542.766188 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335109.06 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 33510.91 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931053.672376 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301598.16 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 30159.82 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961213.487945 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271438.34 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 27143.83 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988357.321957 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244294.51 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 24429.45 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012786.772568 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219865.06 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 21986.51 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034773.278118 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197878.55 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 19787.85 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054561.133113 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178090.69 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 17809.07 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072370.202608 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160281.63 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 16028.16 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088398.365154 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144253.46 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 14425.35 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102823.711446 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129828.12 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 12982.81 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115806.523108 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116845.30 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 11684.53 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127491.053604 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105160.77 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 10516.08 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138007.131050 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94644.70 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 9464.47 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147471.600752 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85180.23 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 8518.02 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155989.623483 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76662.20 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 7666.22 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163655.843942 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68995.98 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 6899.60 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170555.442354 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62096.39 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 6209.64 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176765.080925 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55886.75 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 5588.67 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182353.755640 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50298.07 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 5029.81 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187383.562882 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45268.27 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 4526.83 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191910.389401 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40741.44 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 4074.14 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195984.533267 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36667.29 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3666.73 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199651.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33000.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203251.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29400.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206851.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25800.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210451.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22200.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214051.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18600.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217651.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15000.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221251.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11400.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224851.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7800.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228451.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4200.57 sec) 13:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2232051.262747 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 600.57 sec) 13:00:00 [GPUGRID] [rr_sim] 2232651.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981873135.97G/439.78G) 13:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852492.74 13:00:00 [---] pick_jobs_to_run() start 13:00:00 [---] pick_jobs_to_run() end 13:00:00 [---] [rr_sim] end 13:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:00:00 [---] [cpu_sched_debug] preliminary job list: 13:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:00:00 [---] [cpu_sched_debug] final job list: 13:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:00:00 [---] [cpu_sched_debug] enforce_run_list: end 13:00:00 [---] choose_project(): 46800.000000 13:00:00 [WUProp@Home] doing RPC to get work 13:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388799.09 cpu 2135252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135252.90 13:00:00 [---] Got 0 tasks 13:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:01:00 [---] [rr_sim] doing sim: CPU sched 13:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981845545.61G/439.78G) 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232589.09 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 223258.91 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223258.909118 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009330.18 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 200933.02 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424191.927325 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808397.16 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 180839.72 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605031.643711 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627557.45 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 162755.74 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767787.388458 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464801.70 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 146480.17 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914267.558731 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318321.53 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 131832.15 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046099.711976 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186489.38 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 118648.94 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164748.649897 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067840.44 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 106784.04 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271532.694026 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961056.40 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 96105.64 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367638.333742 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864950.76 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 86495.08 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454133.409486 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778455.68 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 77845.57 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531978.977656 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700610.11 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 70061.01 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1602039.989009 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630549.10 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 63054.91 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665094.899227 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567494.19 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 56749.42 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721844.318422 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510744.77 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 51074.48 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772918.795699 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459670.30 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 45967.03 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818885.825247 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413703.27 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 41370.33 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860256.151841 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372332.94 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 37233.29 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897489.445775 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335099.65 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 33509.96 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930999.410316 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301589.68 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 30158.97 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961158.378403 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271430.71 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 27143.07 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988301.449681 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244287.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 24428.76 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012730.213832 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219858.88 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 21985.89 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034716.101567 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197872.99 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 19787.30 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054503.400529 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178085.69 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 17808.57 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072311.969594 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160277.12 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 16027.71 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088339.681753 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144249.41 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 14424.94 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102764.622697 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129824.47 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 12982.45 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115747.069545 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116842.02 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 11684.20 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127431.271709 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105157.82 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 10515.78 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137947.053657 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94642.04 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 9464.20 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147411.257410 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85177.83 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 8517.78 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155929.040787 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76660.05 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 7666.01 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163595.045827 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68994.05 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 6899.40 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170494.450363 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62094.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 6209.46 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176703.914445 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55885.18 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 5588.52 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182292.432119 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50296.66 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 5029.67 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187322.098025 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45266.99 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 4526.70 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191848.797341 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40740.29 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 4074.03 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195922.826726 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36666.26 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3666.63 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199589.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32999.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203189.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29399.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206789.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25799.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210389.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22199.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213989.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18599.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217589.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14999.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221189.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11399.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224789.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7799.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228389.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4199.64 sec) 13:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231989.453172 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 599.64 sec) 13:01:00 [GPUGRID] [rr_sim] 2232589.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981845545.61G/439.78G) 13:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852490.00 13:01:00 [---] pick_jobs_to_run() start 13:01:00 [---] pick_jobs_to_run() end 13:01:00 [---] [rr_sim] end 13:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:01:00 [---] [cpu_sched_debug] preliminary job list: 13:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:01:00 [---] [cpu_sched_debug] final job list: 13:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:01:00 [---] [cpu_sched_debug] enforce_run_list: end 13:01:00 [---] choose_project(): 46860.000000 13:01:00 [WUProp@Home] doing RPC to get work 13:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388739.09 cpu 2135192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135192.90 13:01:00 [---] Got 0 tasks 13:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:02:00 [---] [rr_sim] doing sim: CPU sched 13:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981817955.26G/439.78G) 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232526.35 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 223252.64 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223252.635430 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009273.72 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 200927.37 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424180.007317 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808346.35 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 180834.63 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 605014.642016 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627511.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 162751.17 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767765.813245 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464760.54 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 146476.05 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914241.867350 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318284.49 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 131828.45 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046070.316046 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186456.04 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 118645.60 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164715.919871 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067810.43 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 106781.04 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271496.963315 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961029.39 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 96102.94 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367599.902413 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864926.45 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 86492.65 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454092.547602 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778433.81 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 77843.38 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531935.928272 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700590.43 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 70059.04 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601994.970875 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630531.38 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 63053.14 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665048.109218 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567478.25 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 56747.82 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721795.933727 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510730.42 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 51073.04 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772868.975784 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459657.38 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 45965.74 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818834.713636 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413691.64 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 41369.16 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860203.877703 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372322.48 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 37232.25 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897436.125363 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335090.23 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 33509.02 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930945.148257 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301581.21 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 30158.12 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961103.268861 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271423.09 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 27142.31 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988245.577405 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244280.78 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 24428.08 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012673.655095 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219852.70 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 21985.27 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034658.925016 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197867.43 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 19786.74 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054445.667945 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178080.69 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 17808.07 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072253.736580 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160272.62 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 16027.26 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088280.998353 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144245.36 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 14424.54 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102705.533948 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129820.82 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 12982.08 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115687.615983 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116838.74 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 11683.87 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127371.489815 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105154.86 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 10515.49 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137886.976264 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94639.38 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 9463.94 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147350.914068 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85175.44 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 8517.54 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155868.458091 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76657.90 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 7665.79 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163534.247712 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68992.11 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 6899.21 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170433.458371 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62092.90 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 6209.29 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176642.747964 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55883.61 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 5588.36 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182231.108598 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50295.25 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 5029.52 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187260.633169 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45265.72 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 4526.57 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191787.205282 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40739.15 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 4073.91 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195861.120184 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36665.23 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3666.52 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199527.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32998.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203127.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29398.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206727.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25798.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210327.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22198.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213927.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18598.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217527.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14998.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221127.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11398.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224727.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7798.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228327.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4198.71 sec) 13:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231927.643596 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 598.71 sec) 13:02:00 [GPUGRID] [rr_sim] 2232526.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981817955.26G/439.78G) 13:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852487.27 13:02:00 [---] pick_jobs_to_run() start 13:02:00 [---] pick_jobs_to_run() end 13:02:00 [---] [rr_sim] end 13:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:02:00 [---] [cpu_sched_debug] preliminary job list: 13:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:02:00 [---] [cpu_sched_debug] final job list: 13:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:02:00 [---] [cpu_sched_debug] enforce_run_list: end 13:02:00 [---] choose_project(): 46920.000000 13:02:00 [WUProp@Home] doing RPC to get work 13:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388679.09 cpu 2135132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135132.90 13:02:00 [---] Got 0 tasks 13:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:03:00 [---] [rr_sim] doing sim: CPU sched 13:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232463.62 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 223246.36 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223246.361742 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009217.26 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 200921.73 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424168.087310 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808295.53 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 180829.55 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604997.640321 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627465.98 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 162746.60 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767744.238031 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464719.38 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 146471.94 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914216.175970 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318247.44 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 131824.74 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046040.920115 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186422.70 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 118642.27 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164683.189845 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067780.43 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 106778.04 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271461.232603 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961002.38 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 96100.24 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367561.471085 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864902.15 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 86490.21 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454051.685718 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778411.93 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 77841.19 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531892.878889 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700570.74 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 70057.07 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601949.952742 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630513.66 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 63051.37 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665001.319210 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567462.30 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 56746.23 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721747.549031 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510716.07 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 51071.61 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772819.155870 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459644.46 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 45964.45 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818783.602025 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413680.02 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 41368.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860151.603564 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372312.01 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 37231.20 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897382.804950 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335080.81 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 33508.08 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930890.886197 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301572.73 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 30157.27 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961048.159319 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271415.46 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 27141.55 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988189.705129 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244273.91 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 24427.39 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012617.096359 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219846.52 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 21984.65 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034601.748465 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197861.87 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 19786.19 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054387.935360 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178075.68 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 17807.57 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072195.503566 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160268.11 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 16026.81 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088222.314952 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144241.30 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 14424.13 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102646.445199 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129817.17 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 12981.72 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115628.162421 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116835.45 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 11683.55 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127311.707921 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105151.91 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 10515.19 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137826.898871 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94636.72 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 9463.67 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147290.570726 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85173.05 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 8517.30 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155807.875395 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76655.74 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 7665.57 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163473.449598 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68990.17 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 6899.02 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170372.466380 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62091.15 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 6209.12 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176581.581484 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55882.04 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 5588.20 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182169.785078 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50293.83 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 5029.38 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187199.168312 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45264.45 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 4526.44 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191725.613223 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40738.00 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 4073.80 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195799.413643 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36664.20 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3666.42 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199465.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32997.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203065.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29397.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206665.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25797.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210265.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22197.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213865.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18597.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217465.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14997.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221065.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11397.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224665.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7797.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228265.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4197.78 sec) 13:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231865.834020 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 597.78 sec) 13:03:00 [GPUGRID] [rr_sim] 2232463.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852484.53 13:03:00 [---] pick_jobs_to_run() start 13:03:00 [---] pick_jobs_to_run() end 13:03:00 [---] [rr_sim] end 13:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:03:00 [---] [cpu_sched_debug] preliminary job list: 13:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:03:00 [---] [cpu_sched_debug] final job list: 13:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:03:00 [---] [cpu_sched_debug] enforce_run_list: end 13:03:00 [---] choose_project(): 46980.000000 13:03:00 [WUProp@Home] doing RPC to get work 13:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388619.09 cpu 2135072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135072.90 13:03:00 [---] Got 0 tasks 13:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:05:00 [---] [rr_sim] doing sim: CPU sched 13:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232463.62 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 223246.36 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223246.361742 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009217.26 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 200921.73 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424168.087310 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808295.53 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 180829.55 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604997.640321 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627465.98 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 162746.60 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767744.238031 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464719.38 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 146471.94 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914216.175970 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318247.44 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 131824.74 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046040.920115 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186422.70 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 118642.27 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164683.189845 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067780.43 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 106778.04 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271461.232603 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961002.38 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 96100.24 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367561.471085 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864902.15 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 86490.21 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454051.685718 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778411.93 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 77841.19 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531892.878889 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700570.74 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 70057.07 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601949.952742 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630513.66 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 63051.37 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665001.319210 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567462.30 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 56746.23 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721747.549031 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510716.07 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 51071.61 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772819.155870 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459644.46 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 45964.45 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818783.602025 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413680.02 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 41368.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860151.603564 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372312.01 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 37231.20 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897382.804950 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335080.81 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 33508.08 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930890.886197 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301572.73 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 30157.27 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961048.159319 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271415.46 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 27141.55 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988189.705129 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244273.91 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 24427.39 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012617.096359 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219846.52 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 21984.65 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034601.748465 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197861.87 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 19786.19 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054387.935360 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178075.68 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 17807.57 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072195.503566 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160268.11 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 16026.81 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088222.314952 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144241.30 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 14424.13 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102646.445199 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129817.17 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 12981.72 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115628.162421 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116835.45 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 11683.55 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127311.707921 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105151.91 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 10515.19 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137826.898871 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94636.72 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 9463.67 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147290.570726 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85173.05 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 8517.30 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155807.875395 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76655.74 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 7665.57 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163473.449598 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68990.17 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 6899.02 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170372.466380 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62091.15 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 6209.12 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176581.581484 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55882.04 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 5588.20 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182169.785078 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50293.83 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 5029.38 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187199.168312 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45264.45 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 4526.44 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191725.613223 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40738.00 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 4073.80 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195799.413643 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36664.20 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3666.42 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199465.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32997.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203065.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29397.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206665.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25797.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210265.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22197.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213865.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18597.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217465.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14997.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221065.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11397.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224665.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7797.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228265.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4197.78 sec) 13:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231865.834020 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 597.78 sec) 13:05:00 [GPUGRID] [rr_sim] 2232463.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852604.53 13:05:00 [---] pick_jobs_to_run() start 13:05:00 [---] pick_jobs_to_run() end 13:05:00 [---] [rr_sim] end 13:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:05:00 [---] [cpu_sched_debug] preliminary job list: 13:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:05:00 [---] [cpu_sched_debug] final job list: 13:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:05:00 [---] [cpu_sched_debug] enforce_run_list: end 13:05:00 [---] choose_project(): 47100.000000 13:05:00 [WUProp@Home] doing RPC to get work 13:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388499.09 cpu 2135072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135072.90 13:05:00 [---] Got 0 tasks 13:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:06:00 [---] [rr_sim] doing sim: CPU sched 13:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232463.62 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 223246.36 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223246.361742 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009217.26 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 200921.73 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424168.087310 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808295.53 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 180829.55 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604997.640321 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627465.98 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 162746.60 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767744.238031 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464719.38 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 146471.94 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914216.175970 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318247.44 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 131824.74 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046040.920115 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186422.70 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 118642.27 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164683.189845 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067780.43 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 106778.04 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271461.232603 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961002.38 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 96100.24 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367561.471085 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864902.15 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 86490.21 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454051.685718 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778411.93 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 77841.19 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531892.878889 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700570.74 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 70057.07 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601949.952742 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630513.66 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 63051.37 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665001.319210 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567462.30 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 56746.23 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721747.549031 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510716.07 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 51071.61 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772819.155870 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459644.46 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 45964.45 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818783.602025 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413680.02 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 41368.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860151.603564 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372312.01 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 37231.20 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897382.804950 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335080.81 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 33508.08 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930890.886197 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301572.73 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 30157.27 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961048.159319 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271415.46 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 27141.55 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988189.705129 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244273.91 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 24427.39 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012617.096359 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219846.52 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 21984.65 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034601.748465 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197861.87 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 19786.19 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054387.935360 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178075.68 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 17807.57 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072195.503566 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160268.11 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 16026.81 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088222.314952 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144241.30 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 14424.13 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102646.445199 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129817.17 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 12981.72 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115628.162421 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116835.45 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 11683.55 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127311.707921 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105151.91 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 10515.19 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137826.898871 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94636.72 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 9463.67 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147290.570726 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85173.05 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 8517.30 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155807.875395 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76655.74 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 7665.57 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163473.449598 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68990.17 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 6899.02 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170372.466380 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62091.15 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 6209.12 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176581.581484 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55882.04 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 5588.20 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182169.785078 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50293.83 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 5029.38 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187199.168312 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45264.45 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 4526.44 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191725.613223 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40738.00 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 4073.80 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195799.413643 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36664.20 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3666.42 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199465.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32997.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203065.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29397.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206665.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25797.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210265.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22197.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213865.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18597.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217465.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14997.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221065.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11397.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224665.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7797.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228265.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4197.78 sec) 13:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231865.834020 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 597.78 sec) 13:06:00 [GPUGRID] [rr_sim] 2232463.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852664.53 13:06:00 [---] pick_jobs_to_run() start 13:06:00 [---] pick_jobs_to_run() end 13:06:00 [---] [rr_sim] end 13:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:06:00 [---] [cpu_sched_debug] preliminary job list: 13:06:00 [---] [cpu_sched_debug] final job list: 13:06:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 13:06:00 [---] [work_fetch] Request work fetch: CPUs idle 13:06:00 [---] [cpu_sched_debug] enforce_run_list: end 13:06:00 [---] choose_project(): 47160.000000 13:06:00 [WUProp@Home] doing RPC to get work 13:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388439.09 cpu 2135072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135072.90 13:06:00 [---] Got 0 tasks 13:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:07:00 [---] [rr_sim] doing sim: CPU sched 13:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232463.62 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 223246.36 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223246.361742 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009217.26 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 200921.73 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424168.087310 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808295.53 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 180829.55 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604997.640321 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627465.98 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 162746.60 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767744.238031 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464719.38 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 146471.94 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914216.175970 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318247.44 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 131824.74 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046040.920115 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186422.70 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 118642.27 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164683.189845 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067780.43 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 106778.04 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271461.232603 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961002.38 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 96100.24 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367561.471085 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864902.15 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 86490.21 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454051.685718 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778411.93 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 77841.19 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531892.878889 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700570.74 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 70057.07 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601949.952742 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630513.66 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 63051.37 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665001.319210 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567462.30 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 56746.23 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721747.549031 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510716.07 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 51071.61 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772819.155870 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459644.46 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 45964.45 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818783.602025 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413680.02 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 41368.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860151.603564 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372312.01 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 37231.20 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897382.804950 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335080.81 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 33508.08 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930890.886197 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301572.73 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 30157.27 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961048.159319 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271415.46 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 27141.55 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988189.705129 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244273.91 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 24427.39 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012617.096359 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219846.52 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 21984.65 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034601.748465 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197861.87 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 19786.19 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054387.935360 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178075.68 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 17807.57 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072195.503566 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160268.11 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 16026.81 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088222.314952 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144241.30 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 14424.13 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102646.445199 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129817.17 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 12981.72 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115628.162421 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116835.45 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 11683.55 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127311.707921 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105151.91 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 10515.19 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137826.898871 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94636.72 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 9463.67 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147290.570726 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85173.05 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 8517.30 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155807.875395 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76655.74 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 7665.57 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163473.449598 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68990.17 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 6899.02 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170372.466380 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62091.15 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 6209.12 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176581.581484 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55882.04 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 5588.20 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182169.785078 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50293.83 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 5029.38 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187199.168312 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45264.45 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 4526.44 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191725.613223 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40738.00 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 4073.80 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195799.413643 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36664.20 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3666.42 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199465.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32997.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203065.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29397.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206665.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25797.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210265.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22197.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213865.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18597.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217465.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14997.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221065.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11397.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224665.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7797.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228265.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4197.78 sec) 13:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231865.834020 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 597.78 sec) 13:07:00 [GPUGRID] [rr_sim] 2232463.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852724.53 13:07:00 [---] pick_jobs_to_run() start 13:07:00 [---] pick_jobs_to_run() end 13:07:00 [---] [rr_sim] end 13:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:07:00 [---] [cpu_sched_debug] preliminary job list: 13:07:00 [---] [cpu_sched_debug] final job list: 13:07:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 13:07:00 [---] [work_fetch] Request work fetch: CPUs idle 13:07:00 [---] [cpu_sched_debug] enforce_run_list: end 13:07:00 [---] choose_project(): 47220.000000 13:07:00 [WUProp@Home] doing RPC to get work 13:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388379.09 cpu 2135072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135072.90 13:07:00 [---] Got 0 tasks 13:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:08:00 [---] [rr_sim] doing sim: CPU sched 13:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232463.62 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 223246.36 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223246.361742 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009217.26 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 200921.73 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424168.087310 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808295.53 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 180829.55 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604997.640321 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627465.98 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 162746.60 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767744.238031 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464719.38 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 146471.94 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914216.175970 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318247.44 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 131824.74 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046040.920115 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186422.70 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 118642.27 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164683.189845 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067780.43 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 106778.04 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271461.232603 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 961002.38 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 96100.24 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367561.471085 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864902.15 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 86490.21 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454051.685718 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778411.93 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 77841.19 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531892.878889 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700570.74 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 70057.07 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601949.952742 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630513.66 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 63051.37 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1665001.319210 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567462.30 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 56746.23 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721747.549031 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510716.07 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 51071.61 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772819.155870 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459644.46 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 45964.45 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818783.602025 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413680.02 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 41368.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860151.603564 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372312.01 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 37231.20 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897382.804950 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335080.81 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 33508.08 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930890.886197 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301572.73 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 30157.27 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961048.159319 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271415.46 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 27141.55 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988189.705129 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244273.91 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 24427.39 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012617.096359 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219846.52 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 21984.65 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034601.748465 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197861.87 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 19786.19 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054387.935360 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178075.68 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 17807.57 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072195.503566 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160268.11 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 16026.81 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088222.314952 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144241.30 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 14424.13 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102646.445199 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129817.17 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 12981.72 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115628.162421 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116835.45 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 11683.55 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127311.707921 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105151.91 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 10515.19 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137826.898871 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94636.72 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 9463.67 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147290.570726 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85173.05 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 8517.30 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155807.875395 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76655.74 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 7665.57 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163473.449598 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68990.17 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 6899.02 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170372.466380 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62091.15 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 6209.12 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176581.581484 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55882.04 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 5588.20 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182169.785078 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50293.83 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 5029.38 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187199.168312 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45264.45 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 4526.44 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191725.613223 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40738.00 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 4073.80 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195799.413643 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36664.20 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3666.42 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199465.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32997.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203065.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29397.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206665.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25797.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210265.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22197.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213865.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18597.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217465.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14997.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221065.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11397.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224665.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7797.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228265.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4197.78 sec) 13:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231865.834020 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 597.78 sec) 13:08:00 [GPUGRID] [rr_sim] 2232463.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981790364.90G/439.78G) 13:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852784.53 13:08:00 [---] pick_jobs_to_run() start 13:08:00 [---] pick_jobs_to_run() end 13:08:00 [---] [rr_sim] end 13:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:08:00 [---] [cpu_sched_debug] preliminary job list: 13:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:08:00 [---] [cpu_sched_debug] final job list: 13:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 13:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:08:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 981790364.90G 13:08:00 [---] [cpu_sched_debug] enforce_run_list: end 13:08:00 [---] choose_project(): 47280.000000 13:08:00 [WUProp@Home] doing RPC to get work 13:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388319.09 cpu 2135072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135072.90 13:08:00 [---] Got 0 tasks 13:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:09:00 [---] [rr_sim] doing sim: CPU sched 13:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981762774.54G/439.78G) 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232400.88 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 223240.09 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223240.088054 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009160.79 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 200916.08 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424156.167302 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808244.71 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 180824.47 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604980.638626 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627420.24 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 162742.02 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767722.662817 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464678.22 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 146467.82 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914190.484589 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318210.40 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 131821.04 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046011.524184 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186389.36 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 118638.94 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164650.459820 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067750.42 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 106775.04 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271425.501891 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960975.38 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 96097.54 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367523.039756 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864877.84 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 86487.78 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454010.823834 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778390.06 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 77839.01 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531849.829505 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700551.05 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 70055.11 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601904.934608 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630495.95 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 63049.59 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664954.529201 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567446.35 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 56744.64 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721699.164335 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510701.72 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 51070.17 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772769.335955 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459631.54 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 45963.15 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818732.490413 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413668.39 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 41366.84 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860099.329426 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372301.55 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 37230.16 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897329.484537 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335071.40 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 33507.14 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930836.624137 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301564.26 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 30156.43 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960993.049777 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271407.83 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 27140.78 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988133.832853 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244267.05 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 24426.70 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012560.537622 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219840.34 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 21984.03 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034544.571914 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197856.31 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 19785.63 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054330.202776 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178070.68 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 17807.07 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072137.270552 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160263.61 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 16026.36 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088163.631551 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144237.25 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 14423.72 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102587.356450 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129813.52 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 12981.35 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115568.708859 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116832.17 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 11683.22 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127251.926026 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105148.95 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 10514.90 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137766.821478 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94634.06 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 9463.41 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147230.227384 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85170.65 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 8517.07 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155747.292699 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76653.59 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 7665.36 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163412.651483 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68988.23 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 6898.82 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170311.474389 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62089.41 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 6208.94 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176520.415004 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55880.47 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 5588.05 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182108.461557 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50292.42 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 5029.24 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187137.703455 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45263.18 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 4526.32 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191664.021163 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40736.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 4073.69 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195737.707101 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36663.17 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3666.32 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199404.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32996.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203004.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29396.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206604.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25796.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210204.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22196.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213804.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18596.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217404.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14996.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221004.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11396.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224604.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7796.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228204.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4196.86 sec) 13:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231804.024445 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 596.86 sec) 13:09:00 [GPUGRID] [rr_sim] 2232400.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981762774.54G/439.78G) 13:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852781.79 13:09:00 [---] pick_jobs_to_run() start 13:09:00 [---] pick_jobs_to_run() end 13:09:00 [---] [rr_sim] end 13:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:09:00 [---] [cpu_sched_debug] preliminary job list: 13:09:00 [---] [cpu_sched_debug] final job list: 13:09:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 13:09:00 [---] [work_fetch] Request work fetch: CPUs idle 13:09:00 [---] [cpu_sched_debug] enforce_run_list: end 13:09:00 [---] choose_project(): 47340.000000 13:09:00 [WUProp@Home] doing RPC to get work 13:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388259.09 cpu 2135012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135012.90 13:09:00 [---] Got 0 tasks 13:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:10:00 [---] [rr_sim] doing sim: CPU sched 13:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981762774.54G/439.78G) 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232400.88 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 223240.09 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223240.088054 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009160.79 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 200916.08 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424156.167302 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808244.71 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 180824.47 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604980.638626 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627420.24 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 162742.02 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767722.662817 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464678.22 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 146467.82 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914190.484589 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318210.40 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 131821.04 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1046011.524184 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186389.36 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 118638.94 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164650.459820 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067750.42 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 106775.04 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271425.501891 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960975.38 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 96097.54 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367523.039756 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864877.84 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 86487.78 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1454010.823834 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778390.06 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 77839.01 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531849.829505 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700551.05 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 70055.11 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601904.934608 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630495.95 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 63049.59 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664954.529201 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567446.35 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 56744.64 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721699.164335 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510701.72 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 51070.17 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772769.335955 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459631.54 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 45963.15 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818732.490413 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413668.39 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 41366.84 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860099.329426 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372301.55 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 37230.16 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897329.484537 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335071.40 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 33507.14 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930836.624137 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301564.26 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 30156.43 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960993.049777 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271407.83 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 27140.78 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988133.832853 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244267.05 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 24426.70 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012560.537622 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219840.34 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 21984.03 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034544.571914 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197856.31 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 19785.63 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054330.202776 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178070.68 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 17807.07 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072137.270552 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160263.61 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 16026.36 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088163.631551 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144237.25 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 14423.72 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102587.356450 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129813.52 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 12981.35 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115568.708859 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116832.17 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 11683.22 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127251.926026 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105148.95 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 10514.90 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137766.821478 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94634.06 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 9463.41 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147230.227384 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85170.65 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 8517.07 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155747.292699 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76653.59 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 7665.36 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163412.651483 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68988.23 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 6898.82 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170311.474389 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62089.41 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 6208.94 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176520.415004 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55880.47 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 5588.05 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182108.461557 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50292.42 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 5029.24 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187137.703455 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45263.18 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 4526.32 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191664.021163 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40736.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 4073.69 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195737.707101 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36663.17 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3666.32 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199404.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32996.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203004.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29396.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206604.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25796.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210204.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22196.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213804.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18596.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217404.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14996.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221004.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11396.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224604.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7796.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228204.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4196.86 sec) 13:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231804.024445 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 596.86 sec) 13:10:00 [GPUGRID] [rr_sim] 2232400.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981762774.54G/439.78G) 13:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852841.79 13:10:00 [---] pick_jobs_to_run() start 13:10:00 [---] pick_jobs_to_run() end 13:10:00 [---] [rr_sim] end 13:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:10:00 [---] [cpu_sched_debug] preliminary job list: 13:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 13:10:00 [---] [cpu_sched_debug] final job list: 13:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 13:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:10:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 981762774.54G 13:10:00 [---] [cpu_sched_debug] enforce_run_list: end 13:10:00 [---] choose_project(): 47400.000000 13:10:00 [WUProp@Home] doing RPC to get work 13:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388199.09 cpu 2135012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2135012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2135012.90 13:10:00 [---] Got 0 tasks 13:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:11:00 [---] [rr_sim] doing sim: CPU sched 13:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981735184.19G/439.78G) 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232338.14 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 223233.81 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223233.814366 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009104.33 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 200910.43 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424144.247295 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808193.90 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 180819.39 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604963.636931 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627374.51 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 162737.45 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767701.087603 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464637.06 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 146463.71 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914164.793209 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318173.35 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 131817.34 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045982.128253 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186356.02 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 118635.60 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164617.729794 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067720.41 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 106772.04 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271389.771180 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960948.37 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 96094.84 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367484.608428 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864853.54 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 86485.35 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453969.961950 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778368.18 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 77836.82 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531806.780121 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700531.36 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 70053.14 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601859.916474 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630478.23 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 63047.82 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664907.739193 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567430.40 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 56743.04 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721650.779639 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510687.36 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 51068.74 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772719.516041 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459618.63 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 45961.86 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818681.378802 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413656.76 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 41365.68 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1860047.055288 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372291.09 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 37229.11 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897276.164124 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335061.98 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 33506.20 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930782.362078 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301555.78 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 30155.58 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960937.940235 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271400.20 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 27140.02 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988077.960578 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244260.18 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 24426.02 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012503.978885 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219834.16 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 21983.42 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034487.395362 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197850.75 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 19785.07 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054272.470192 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178065.67 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 17806.57 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072079.037538 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160259.11 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 16025.91 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088104.948150 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144233.20 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 14423.32 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102528.267701 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129809.88 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 12980.99 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115509.255296 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116828.89 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 11682.89 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127192.144132 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105146.00 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 10514.60 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137706.744085 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94631.40 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 9463.14 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147169.884042 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85168.26 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 8516.83 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155686.710003 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76651.43 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 7665.14 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163351.853368 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68986.29 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 6898.63 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170250.482397 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62087.66 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 6208.77 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176459.248523 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55878.90 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 5587.89 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182047.138036 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50291.01 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 5029.10 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187076.238598 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45261.91 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 4526.19 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191602.429104 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40735.71 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 4073.57 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195676.000559 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36662.14 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3666.21 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199342.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32995.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202942.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29395.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206542.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25795.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210142.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22195.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213742.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18595.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217342.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14995.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220942.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11395.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224542.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7795.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228142.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4195.93 sec) 13:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231742.214869 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 595.93 sec) 13:11:00 [GPUGRID] [rr_sim] 2232338.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981735184.19G/439.78G) 13:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852839.06 13:11:00 [---] pick_jobs_to_run() start 13:11:00 [---] pick_jobs_to_run() end 13:11:00 [---] [rr_sim] end 13:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:11:00 [---] [cpu_sched_debug] preliminary job list: 13:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 13:11:00 [---] [cpu_sched_debug] final job list: 13:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:11:00 [---] [cpu_sched_debug] enforce_run_list: end 13:11:00 [---] choose_project(): 47460.000000 13:11:00 [WUProp@Home] doing RPC to get work 13:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388139.09 cpu 2134952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134952.90 13:11:00 [---] Got 0 tasks 13:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:12:00 [---] [rr_sim] doing sim: CPU sched 13:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981707593.83G/439.78G) 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232275.41 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 223227.54 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223227.540677 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2009047.87 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 200904.79 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424132.327287 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808143.08 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 180814.31 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604946.635236 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627328.77 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 162732.88 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767679.512390 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464595.89 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 146459.59 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914139.101828 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318136.30 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 131813.63 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045952.732323 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186322.67 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 118632.27 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164584.999768 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067690.41 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 106769.04 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271354.040468 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960921.37 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 96092.14 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367446.177099 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864829.23 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 86482.92 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453929.100066 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778346.31 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 77834.63 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531763.730737 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700511.68 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 70051.17 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601814.898341 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630460.51 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 63046.05 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664860.949184 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567414.46 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 56741.45 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721602.394943 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510673.01 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 51067.30 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772669.696126 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459605.71 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 45960.57 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818630.267191 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413645.14 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 41364.51 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859994.781149 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372280.63 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 37228.06 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897222.843712 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335052.56 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 33505.26 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930728.100018 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301547.31 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 30154.73 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960882.830694 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271392.58 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 27139.26 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988022.088302 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244253.32 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 24425.33 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012447.420149 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219827.99 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 21982.80 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034430.218811 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197845.19 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 19784.52 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054214.737608 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178060.67 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 17806.07 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072020.804524 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160254.60 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 16025.46 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088046.264749 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144229.14 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 14422.91 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102469.178952 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129806.23 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 12980.62 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115449.801734 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116825.61 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 11682.56 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127132.362238 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105143.04 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 10514.30 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137646.666692 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94628.74 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 9462.87 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147109.540700 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85165.87 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 8516.59 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155626.127307 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76649.28 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 7664.93 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163291.055254 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68984.35 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 6898.44 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170189.490406 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62085.92 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 6208.59 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176398.082043 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55877.32 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 5587.73 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181985.814516 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50289.59 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 5028.96 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187014.773742 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45260.63 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 4526.06 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191540.837045 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40734.57 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 4073.46 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195614.294018 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36661.11 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3666.11 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199280.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32995.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202880.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29395.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206480.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25795.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210080.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22195.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213680.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18595.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217280.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14995.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220880.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11395.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224480.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7795.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228080.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4195.00 sec) 13:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231680.405293 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 595.00 sec) 13:12:00 [GPUGRID] [rr_sim] 2232275.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981707593.83G/439.78G) 13:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852836.32 13:12:00 [---] pick_jobs_to_run() start 13:12:00 [---] pick_jobs_to_run() end 13:12:00 [---] [rr_sim] end 13:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:12:00 [---] [cpu_sched_debug] preliminary job list: 13:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:12:00 [---] [cpu_sched_debug] final job list: 13:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:12:00 [---] [cpu_sched_debug] enforce_run_list: end 13:12:00 [---] choose_project(): 47520.000000 13:12:00 [WUProp@Home] doing RPC to get work 13:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388079.09 cpu 2134892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134892.90 13:12:00 [---] Got 0 tasks 13:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:13:00 [---] [rr_sim] doing sim: CPU sched 13:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981680003.48G/439.78G) 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232212.67 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 223221.27 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223221.266989 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008991.40 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 200899.14 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424120.407279 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808092.26 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 180809.23 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604929.633541 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627283.04 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 162728.30 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767657.937176 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464554.73 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 146455.47 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914113.410447 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318099.26 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 131809.93 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045923.336392 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186289.33 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 118628.93 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164552.269742 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067660.40 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 106766.04 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271318.309757 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960894.36 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 96089.44 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367407.745770 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864804.92 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 86480.49 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453888.238182 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778324.43 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 77832.44 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531720.681353 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700491.99 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 70049.20 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601769.880207 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630442.79 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 63044.28 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664814.159176 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567398.51 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 56739.85 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721554.010247 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510658.66 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 51065.87 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772619.876212 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459592.79 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 45959.28 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818579.155580 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413633.51 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 41363.35 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859942.507011 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372270.16 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 37227.02 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897169.523299 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335043.15 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 33504.31 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930673.837958 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301538.83 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 30153.88 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960827.721152 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271384.95 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 27138.49 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987966.216026 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244246.45 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 24424.65 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012390.861412 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219821.81 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 21982.18 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034373.042260 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197839.63 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 19783.96 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054157.005023 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178055.66 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 17805.57 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071962.571510 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160250.10 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 16025.01 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087987.581348 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144225.09 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 14422.51 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102410.090203 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129802.58 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 12980.26 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115390.348172 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116822.32 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 11682.23 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127072.580344 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105140.09 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 10514.01 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137586.589298 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94626.08 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 9462.61 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147049.197358 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85163.47 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 8516.35 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155565.544611 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76647.13 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 7664.71 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163230.257139 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68982.41 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 6898.24 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170128.498415 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62084.17 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 6208.42 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176336.915562 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55875.75 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 5587.58 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181924.490995 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50288.18 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 5028.82 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186953.308885 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45259.36 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 4525.94 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191479.244986 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40733.42 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 4073.34 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195552.587476 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36660.08 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3666.01 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199218.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32994.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202818.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29394.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206418.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25794.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210018.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22194.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213618.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18594.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217218.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14994.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220818.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11394.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224418.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7794.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228018.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4194.07 sec) 13:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231618.595718 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 594.07 sec) 13:13:00 [GPUGRID] [rr_sim] 2232212.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981680003.48G/439.78G) 13:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852833.58 13:13:00 [---] pick_jobs_to_run() start 13:13:00 [---] pick_jobs_to_run() end 13:13:00 [---] [rr_sim] end 13:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:13:00 [---] [cpu_sched_debug] preliminary job list: 13:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:13:00 [---] [cpu_sched_debug] final job list: 13:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:13:00 [---] [cpu_sched_debug] enforce_run_list: end 13:13:00 [---] choose_project(): 47580.000000 13:13:00 [WUProp@Home] doing RPC to get work 13:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 388019.09 cpu 2134832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134832.90 13:13:00 [---] Got 0 tasks 13:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:14:00 [---] [rr_sim] doing sim: CPU sched 13:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981652413.12G/439.78G) 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232149.93 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 223214.99 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223214.993301 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008934.94 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 200893.49 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424108.487272 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1808041.45 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 180804.14 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604912.631846 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627237.30 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 162723.73 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767636.361962 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464513.57 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 146451.36 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914087.719067 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318062.21 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 131806.22 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045893.940461 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186255.99 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 118625.60 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164519.539716 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067630.39 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 106763.04 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271282.579045 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960867.35 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 96086.74 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367369.314442 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864780.62 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 86478.06 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453847.376299 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778302.56 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 77830.26 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531677.631970 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700472.30 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 70047.23 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601724.862074 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630425.07 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 63042.51 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664767.369167 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567382.56 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 56738.26 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721505.625551 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510644.31 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 51064.43 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772570.056297 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459579.88 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 45957.99 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818528.043969 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413621.89 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 41362.19 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859890.232873 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372259.70 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 37225.97 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897116.202886 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335033.73 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 33503.37 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930619.575899 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301530.36 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 30153.04 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960772.611610 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271377.32 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 27137.73 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987910.343750 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244239.59 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 24423.96 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012334.302676 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219815.63 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 21981.56 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034315.865709 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197834.07 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 19783.41 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054099.272439 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178050.66 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 17805.07 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071904.338496 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160245.59 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 16024.56 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087928.897948 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144221.04 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 14422.10 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102351.001454 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129798.93 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 12979.89 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115330.894609 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116819.04 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 11681.90 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127012.798449 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105137.13 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 10513.71 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137526.511905 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94623.42 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 9462.34 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146988.854016 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85161.08 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 8516.11 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155504.961915 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76644.97 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 7664.50 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163169.459025 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68980.47 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 6898.05 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170067.506423 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62082.43 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 6208.24 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176275.749082 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55874.18 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 5587.42 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181863.167475 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50286.77 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 5028.68 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186891.844028 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45258.09 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 4525.81 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191417.652926 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40732.28 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 4073.23 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195490.880935 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36659.05 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3665.91 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199156.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32993.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202756.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29393.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206356.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25793.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209956.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22193.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213556.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18593.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217156.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14993.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220756.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11393.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224356.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7793.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227956.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4193.15 sec) 13:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231556.786142 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 593.15 sec) 13:14:00 [GPUGRID] [rr_sim] 2232149.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981652413.12G/439.78G) 13:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852830.85 13:14:00 [---] pick_jobs_to_run() start 13:14:00 [---] pick_jobs_to_run() end 13:14:00 [---] [rr_sim] end 13:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:14:00 [---] [cpu_sched_debug] preliminary job list: 13:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:14:00 [---] [cpu_sched_debug] final job list: 13:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:14:00 [---] [cpu_sched_debug] enforce_run_list: end 13:14:00 [---] choose_project(): 47640.000000 13:14:00 [WUProp@Home] doing RPC to get work 13:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387959.09 cpu 2134772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134772.90 13:14:00 [---] Got 0 tasks 13:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:15:00 [---] [rr_sim] doing sim: CPU sched 13:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981624822.76G/439.78G) 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232087.20 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 223208.72 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223208.719613 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008878.48 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 200887.85 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424096.567264 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807990.63 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 180799.06 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604895.630151 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627191.57 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 162719.16 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767614.786748 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464472.41 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 146447.24 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914062.027686 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1318025.17 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 131802.52 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045864.544530 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186222.65 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 118622.27 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164486.809690 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067600.39 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 106760.04 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271246.848334 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960840.35 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 96084.03 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367330.883113 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864756.31 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 86475.63 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453806.514415 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778280.68 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 77828.07 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531634.582586 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700452.61 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 70045.26 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601679.843940 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630407.35 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 63040.74 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664720.579159 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567366.62 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 56736.66 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721457.240856 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510629.96 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 51063.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772520.236383 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459566.96 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 45956.70 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818476.932357 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413610.26 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 41361.03 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859837.958734 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372249.24 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 37224.92 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897062.882474 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335024.31 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 33502.43 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930565.313839 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301521.88 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 30152.19 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960717.502068 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271369.69 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 27136.97 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987854.471474 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244232.72 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 24423.27 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012277.743939 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219809.45 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 21980.95 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034258.689158 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197828.51 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 19782.85 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054041.539855 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178045.66 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 17804.57 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071846.105482 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160241.09 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 16024.11 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087870.214547 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144216.98 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 14421.70 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102291.912705 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129795.28 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 12979.53 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115271.441047 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116815.76 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 11681.58 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126953.016555 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105134.18 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 10513.42 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137466.434512 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94620.76 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 9462.08 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146928.510674 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85158.69 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 8515.87 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155444.379219 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76642.82 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 7664.28 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163108.660910 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68978.54 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 6897.85 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170006.514432 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62080.68 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 6208.07 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176214.582601 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55872.61 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 5587.26 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181801.843954 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50285.35 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 5028.54 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186830.379171 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45256.82 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 4525.68 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191356.060867 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40731.14 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 4073.11 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195429.174393 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36658.02 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3665.80 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199094.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32992.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202694.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29392.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206294.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25792.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209894.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22192.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213494.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18592.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217094.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14992.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220694.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11392.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224294.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7792.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227894.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4192.22 sec) 13:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231494.976566 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 592.22 sec) 13:15:00 [GPUGRID] [rr_sim] 2232087.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981624822.76G/439.78G) 13:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852828.11 13:15:00 [---] pick_jobs_to_run() start 13:15:00 [---] pick_jobs_to_run() end 13:15:00 [---] [rr_sim] end 13:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:15:00 [---] [cpu_sched_debug] preliminary job list: 13:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:15:00 [---] [cpu_sched_debug] final job list: 13:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:15:00 [---] [cpu_sched_debug] enforce_run_list: end 13:15:00 [---] choose_project(): 47700.000000 13:15:00 [WUProp@Home] doing RPC to get work 13:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387899.09 cpu 2134712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134712.90 13:15:00 [---] Got 0 tasks 13:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:16:00 [---] [rr_sim] doing sim: CPU sched 13:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981597232.41G/439.78G) 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2232024.46 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 223202.45 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223202.445925 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008822.01 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 200882.20 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424084.647257 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807939.81 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 180793.98 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604878.628455 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627145.83 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 162714.58 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767593.211534 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464431.25 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 146443.12 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914036.336306 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317988.12 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 131798.81 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045835.148600 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186189.31 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 118618.93 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164454.079664 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067570.38 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 106757.04 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271211.117622 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960813.34 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 96081.33 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367292.451785 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864732.01 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 86473.20 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453765.652531 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778258.81 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 77825.88 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531591.533202 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700432.93 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 70043.29 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601634.825806 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630389.63 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 63038.96 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664673.789150 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567350.67 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 56735.07 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721408.856160 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510615.60 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 51061.56 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772470.416468 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459554.04 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 45955.40 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818425.820746 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413598.64 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 41359.86 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859785.684596 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372238.77 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 37223.88 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897009.562061 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335014.90 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 33501.49 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930511.051779 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301513.41 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 30151.34 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960662.392526 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271362.07 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 27136.21 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987798.599198 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244225.86 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 24422.59 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012221.185203 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219803.27 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 21980.33 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034201.512607 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197822.95 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 19782.29 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053983.807271 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178040.65 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 17804.07 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071787.872468 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160236.59 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 16023.66 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087811.531146 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144212.93 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 14421.29 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102232.823956 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129791.64 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 12979.16 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115211.987485 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116812.47 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 11681.25 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126893.234661 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105131.22 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 10513.12 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137406.357119 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94618.10 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 9461.81 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146868.167332 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85156.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 8515.63 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155383.796523 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76640.66 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 7664.07 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163047.862795 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68976.60 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 6897.66 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169945.522440 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62078.94 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 6207.89 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176153.416121 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55871.04 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 5587.10 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181740.520433 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50283.94 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 5028.39 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186768.914315 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45255.54 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 4525.55 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191294.468808 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40729.99 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 4073.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195367.467851 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36656.99 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3665.70 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199033.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32991.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202633.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29391.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206233.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25791.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209833.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22191.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213433.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18591.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217033.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14991.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220633.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11391.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224233.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7791.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227833.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4191.29 sec) 13:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231433.166991 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 591.29 sec) 13:16:00 [GPUGRID] [rr_sim] 2232024.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981597232.41G/439.78G) 13:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852825.37 13:16:00 [---] pick_jobs_to_run() start 13:16:00 [---] pick_jobs_to_run() end 13:16:00 [---] [rr_sim] end 13:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:16:00 [---] [cpu_sched_debug] preliminary job list: 13:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:16:00 [---] [cpu_sched_debug] final job list: 13:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:16:00 [---] [cpu_sched_debug] enforce_run_list: end 13:16:00 [---] choose_project(): 47760.000000 13:16:00 [WUProp@Home] doing RPC to get work 13:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387839.09 cpu 2134652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134652.90 13:16:00 [---] Got 0 tasks 13:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:17:00 [---] [rr_sim] doing sim: CPU sched 13:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981569642.05G/439.78G) 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231961.72 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 223196.17 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223196.172236 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008765.55 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 200876.56 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424072.727249 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807889.00 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 180788.90 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604861.626760 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627100.10 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 162710.01 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767571.636321 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464390.09 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 146439.01 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 914010.644925 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317951.08 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 131795.11 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045805.752669 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186155.97 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 118615.60 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164421.349638 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067540.37 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 106754.04 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271175.386911 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960786.34 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 96078.63 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367254.020456 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864707.70 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 86470.77 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453724.790647 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778236.93 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 77823.69 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531548.483818 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700413.24 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 70041.32 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601589.807673 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630371.91 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 63037.19 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664626.999142 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567334.72 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 56733.47 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721360.471464 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510601.25 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 51060.13 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772420.596554 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459541.13 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 45954.11 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818374.709135 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413587.01 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 41358.70 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859733.410458 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372228.31 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 37222.83 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896956.241648 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335005.48 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 33500.55 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930456.789720 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301504.93 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 30150.49 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960607.282984 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271354.44 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 27135.44 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987742.726922 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244219.00 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 24421.90 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012164.626466 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219797.10 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 21979.71 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034144.336056 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197817.39 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 19781.74 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053926.074687 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178035.65 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 17803.56 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071729.639454 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160232.08 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 16023.21 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087752.847745 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144208.87 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 14420.89 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102173.735207 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129787.99 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 12978.80 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115152.533923 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116809.19 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 11680.92 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126833.452767 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105128.27 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 10512.83 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137346.279726 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94615.44 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 9461.54 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146807.823990 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85153.90 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 8515.39 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155323.213827 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76638.51 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 7663.85 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162987.064681 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68974.66 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 6897.47 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169884.530449 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62077.19 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 6207.72 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176092.249640 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55869.47 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 5586.95 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181679.196913 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50282.53 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 5028.25 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186707.449458 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45254.27 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 4525.43 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191232.876748 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40728.85 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 4072.88 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195305.761310 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36655.96 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3665.60 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198971.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32990.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202571.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29390.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206171.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25790.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209771.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22190.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213371.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18590.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216971.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14990.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220571.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11390.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224171.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7790.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227771.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4190.36 sec) 13:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231371.357415 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 590.36 sec) 13:17:00 [GPUGRID] [rr_sim] 2231961.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981569642.05G/439.78G) 13:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852822.64 13:17:00 [---] pick_jobs_to_run() start 13:17:00 [---] pick_jobs_to_run() end 13:17:00 [---] [rr_sim] end 13:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:17:00 [---] [cpu_sched_debug] preliminary job list: 13:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:17:00 [---] [cpu_sched_debug] final job list: 13:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:17:00 [---] [cpu_sched_debug] enforce_run_list: end 13:17:00 [---] choose_project(): 47820.000000 13:17:00 [WUProp@Home] doing RPC to get work 13:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387779.09 cpu 2134592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134592.90 13:17:00 [---] Got 0 tasks 13:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:18:00 [---] [rr_sim] doing sim: CPU sched 13:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981542051.69G/439.78G) 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231898.99 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 223189.90 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223189.898548 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008709.09 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 200870.91 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424060.807241 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807838.18 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 180783.82 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604844.625065 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627054.36 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 162705.44 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767550.061107 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464348.92 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 146434.89 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913984.953544 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317914.03 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 131791.40 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045776.356738 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186122.63 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 118612.26 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164388.619612 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067510.37 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 106751.04 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271139.656199 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960759.33 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 96075.93 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367215.589127 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864683.40 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 86468.34 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453683.928763 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778215.06 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 77821.51 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531505.434435 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700393.55 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 70039.36 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601544.789539 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630354.20 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 63035.42 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664580.209133 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567318.78 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 56731.88 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721312.086768 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510586.90 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 51058.69 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772370.776639 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459528.21 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 45952.82 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818323.597524 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413575.39 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 41357.54 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859681.136319 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372217.85 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 37221.78 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896902.921235 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334996.06 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 33499.61 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930402.527660 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301496.46 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 30149.65 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960552.173442 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271346.81 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 27134.68 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987686.854646 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244212.13 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 24421.21 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012108.067730 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219790.92 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 21979.09 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034087.159505 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197811.83 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 19781.18 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053868.342102 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178030.64 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 17803.06 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071671.406440 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160227.58 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 16022.76 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087694.164344 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144204.82 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 14420.48 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102114.646458 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129784.34 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 12978.43 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115093.080360 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116805.91 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 11680.59 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126773.670872 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105125.31 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 10512.53 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137286.202333 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94612.78 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 9461.28 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146747.480648 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85151.50 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 8515.15 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155262.631131 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76636.35 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 7663.64 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162926.266566 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68972.72 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 6897.27 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169823.538458 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62075.45 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 6207.54 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176031.083160 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55867.90 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 5586.79 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181617.873392 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50281.11 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 5028.11 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186645.984601 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45253.00 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 4525.30 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191171.284689 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40727.70 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 4072.77 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195244.054768 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36654.93 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3665.49 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198909.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32989.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202509.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29389.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206109.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25789.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209709.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22189.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213309.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18589.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216909.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14989.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220509.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11389.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224109.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7789.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227709.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4189.44 sec) 13:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231309.547839 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 589.44 sec) 13:18:00 [GPUGRID] [rr_sim] 2231898.99: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981542051.69G/439.78G) 13:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852819.90 13:18:00 [---] pick_jobs_to_run() start 13:18:00 [---] pick_jobs_to_run() end 13:18:00 [---] [rr_sim] end 13:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:18:00 [---] [cpu_sched_debug] preliminary job list: 13:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:18:00 [---] [cpu_sched_debug] final job list: 13:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:18:00 [---] [cpu_sched_debug] enforce_run_list: end 13:18:00 [---] choose_project(): 47880.000000 13:18:00 [WUProp@Home] doing RPC to get work 13:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387719.09 cpu 2134532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134532.90 13:18:00 [---] Got 0 tasks 13:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:19:00 [---] [rr_sim] doing sim: CPU sched 13:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981514461.34G/439.78G) 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231836.25 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 223183.62 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223183.624860 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008652.62 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 200865.26 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424048.887234 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807787.36 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 180778.74 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604827.623370 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1627008.63 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 162700.86 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767528.485893 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464307.76 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 146430.78 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913959.262164 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317876.99 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 131787.70 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045746.960807 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186089.29 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 118608.93 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164355.889586 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067480.36 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 106748.04 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271103.925488 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960732.32 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 96073.23 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367177.157799 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864659.09 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 86465.91 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453643.066879 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778193.18 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 77819.32 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531462.385051 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700373.86 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 70037.39 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601499.771406 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630336.48 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 63033.65 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664533.419125 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567302.83 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 56730.28 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721263.702072 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510572.55 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 51057.25 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772320.956725 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459515.29 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 45951.53 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818272.485912 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413563.76 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 41356.38 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859628.862181 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372207.39 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 37220.74 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896849.600823 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334986.65 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 33498.66 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930348.265600 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301487.98 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 30148.80 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960497.063900 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271339.18 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 27133.92 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987630.982370 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244205.27 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 24420.53 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012051.508993 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219784.74 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 21978.47 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034029.982954 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197806.27 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 19780.63 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053810.609518 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178025.64 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 17802.56 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071613.173426 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160223.08 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 16022.31 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087635.480943 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144200.77 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 14420.08 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102055.557709 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129780.69 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 12978.07 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115033.626798 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116802.62 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 11680.26 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126713.888978 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105122.36 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 10512.24 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137226.124940 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94610.12 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 9461.01 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146687.137306 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85149.11 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 8514.91 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155202.048435 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76634.20 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 7663.42 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162865.468452 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68970.78 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 6897.08 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169762.546466 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62073.70 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 6207.37 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175969.916680 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55866.33 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 5586.63 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181556.549872 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50279.70 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 5027.97 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186584.519744 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45251.73 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 4525.17 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191109.692630 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40726.56 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 4072.66 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195182.348227 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36653.90 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3665.39 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198847.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32988.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202447.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29388.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206047.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25788.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209647.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22188.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213247.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18588.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216847.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14988.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220447.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11388.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224047.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7788.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227647.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4188.51 sec) 13:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231247.738264 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 588.51 sec) 13:19:00 [GPUGRID] [rr_sim] 2231836.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981514461.34G/439.78G) 13:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852817.16 13:19:00 [---] pick_jobs_to_run() start 13:19:00 [---] pick_jobs_to_run() end 13:19:00 [---] [rr_sim] end 13:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:19:00 [---] [cpu_sched_debug] preliminary job list: 13:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:19:00 [---] [cpu_sched_debug] final job list: 13:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:19:00 [---] [cpu_sched_debug] enforce_run_list: end 13:19:00 [---] choose_project(): 47940.000000 13:19:00 [WUProp@Home] doing RPC to get work 13:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387659.09 cpu 2134472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134472.90 13:19:00 [---] Got 0 tasks 13:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:20:00 [---] [rr_sim] doing sim: CPU sched 13:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981486870.98G/439.78G) 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231773.51 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 223177.35 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223177.351172 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008596.16 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 200859.62 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424036.967226 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807736.54 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 180773.65 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604810.621675 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626962.89 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 162696.29 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767506.910679 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464266.60 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 146426.66 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913933.570783 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317839.94 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 131783.99 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045717.564876 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186055.95 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 118605.59 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164323.159560 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067450.35 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 106745.04 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271068.194776 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960705.32 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 96070.53 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367138.726470 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864634.79 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 86463.48 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453602.204995 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778171.31 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 77817.13 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531419.335667 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700354.18 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 70035.42 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601454.753272 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630318.76 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 63031.88 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664486.629116 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567286.88 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 56728.69 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721215.317376 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510558.19 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 51055.82 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772271.136810 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459502.37 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 45950.24 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818221.374301 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413552.14 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 41355.21 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859576.588043 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372196.92 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 37219.69 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896796.280410 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334977.23 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 33497.72 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930294.003541 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301479.51 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 30147.95 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960441.954358 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271331.56 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 27133.16 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987575.110094 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244198.40 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 24419.84 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011994.950256 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219778.56 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 21977.86 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033972.806402 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197800.71 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 19780.07 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053752.876934 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178020.63 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 17802.06 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071554.940412 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160218.57 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 16021.86 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087576.797543 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144196.71 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 14419.67 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101996.468960 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129777.04 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 12977.70 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114974.173236 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116799.34 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 11679.93 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126654.107084 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105119.40 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 10511.94 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137166.047547 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94607.46 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 9460.75 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146626.793964 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85146.72 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 8514.67 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155141.465739 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76632.05 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 7663.20 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162804.670337 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68968.84 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 6896.88 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169701.554475 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62071.96 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 6207.20 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175908.750199 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55864.76 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 5586.48 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181495.226351 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50278.29 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 5027.83 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186523.054887 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45250.46 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 4525.05 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191048.100570 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40725.41 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 4072.54 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195120.641685 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36652.87 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3665.29 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198785.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32987.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202385.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29387.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205985.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25787.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209585.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22187.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213185.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18587.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216785.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14987.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220385.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11387.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223985.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7787.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227585.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4187.58 sec) 13:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231185.928688 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 587.58 sec) 13:20:00 [GPUGRID] [rr_sim] 2231773.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981486870.98G/439.78G) 13:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852814.42 13:20:00 [---] pick_jobs_to_run() start 13:20:00 [---] pick_jobs_to_run() end 13:20:00 [---] [rr_sim] end 13:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:20:00 [---] [cpu_sched_debug] preliminary job list: 13:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:20:00 [---] [cpu_sched_debug] final job list: 13:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:20:00 [---] [cpu_sched_debug] enforce_run_list: end 13:20:00 [---] choose_project(): 48000.000000 13:20:00 [WUProp@Home] doing RPC to get work 13:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387599.09 cpu 2134412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134412.90 13:20:00 [---] Got 0 tasks 13:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:21:00 [---] [rr_sim] doing sim: CPU sched 13:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981459280.63G/439.78G) 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231710.77 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 223171.08 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223171.077483 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008539.70 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 200853.97 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424025.047219 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807685.73 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 180768.57 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604793.619980 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626917.15 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 162691.72 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767485.335466 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464225.44 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 146422.54 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913907.879402 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317802.90 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 131780.29 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045688.168946 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1186022.61 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 118602.26 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164290.429535 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067420.35 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 106742.03 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1271032.464065 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960678.31 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 96067.83 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367100.295142 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864610.48 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 86461.05 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453561.343111 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778149.43 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 77814.94 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531376.286283 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700334.49 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 70033.45 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601409.735138 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630301.04 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 63030.10 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664439.839108 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567270.94 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 56727.09 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721166.932681 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510543.84 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 51054.38 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772221.316896 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459489.46 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 45948.95 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818170.262690 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413540.51 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 41354.05 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859524.313904 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372186.46 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 37218.65 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896742.959997 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334967.81 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 33496.78 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930239.741481 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301471.03 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 30147.10 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960386.844816 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271323.93 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 27132.39 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987519.237818 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244191.54 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 24419.15 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011938.391520 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219772.38 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 21977.24 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033915.629851 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197795.14 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 19779.51 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053695.144350 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178015.63 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 17801.56 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071496.707398 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160214.07 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 16021.41 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087518.114142 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144192.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 14419.27 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101937.380211 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129773.39 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 12977.34 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114914.719673 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116796.06 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 11679.61 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126594.325189 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105116.45 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 10511.64 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137105.970154 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94604.80 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 9460.48 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146566.450622 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85144.32 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 8514.43 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155080.883043 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76629.89 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 7662.99 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162743.872222 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68966.90 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 6896.69 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169640.562484 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62070.21 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 6207.02 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175847.583719 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55863.19 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 5586.32 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181433.902830 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50276.87 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 5027.69 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186461.590031 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45249.18 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 4524.92 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190986.508511 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40724.27 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 4072.43 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195058.935143 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36651.84 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3665.18 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198724.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32986.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202324.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29386.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205924.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25786.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209524.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22186.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213124.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18586.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216724.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14986.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220324.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11386.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223924.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7786.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227524.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4186.66 sec) 13:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231124.119113 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 586.66 sec) 13:21:00 [GPUGRID] [rr_sim] 2231710.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981459280.63G/439.78G) 13:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852811.69 13:21:00 [---] pick_jobs_to_run() start 13:21:00 [---] pick_jobs_to_run() end 13:21:00 [---] [rr_sim] end 13:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:21:00 [---] [cpu_sched_debug] preliminary job list: 13:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:21:00 [---] [cpu_sched_debug] final job list: 13:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:21:00 [---] [cpu_sched_debug] enforce_run_list: end 13:21:00 [---] choose_project(): 48060.000000 13:21:00 [WUProp@Home] doing RPC to get work 13:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387539.09 cpu 2134352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134352.90 13:21:00 [---] Got 0 tasks 13:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:22:00 [---] [rr_sim] doing sim: CPU sched 13:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981431690.27G/439.78G) 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231648.04 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 223164.80 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223164.803795 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008483.23 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 200848.32 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424013.127211 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807634.91 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 180763.49 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604776.618285 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626871.42 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 162687.14 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767463.760252 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464184.28 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 146418.43 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913882.188022 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317765.85 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 131776.58 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045658.773015 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185989.26 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 118598.93 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164257.699509 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067390.34 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 106739.03 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270996.733353 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960651.30 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 96065.13 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367061.863813 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864586.17 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 86458.62 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453520.481227 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778127.56 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 77812.76 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531333.236899 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700314.80 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 70031.48 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601364.717005 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630283.32 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 63028.33 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664393.049099 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567254.99 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 56725.50 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721118.547985 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510529.49 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 51052.95 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772171.496982 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459476.54 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 45947.65 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818119.151079 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413528.89 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 41352.89 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859472.039766 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372176.00 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 37217.60 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896689.639585 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334958.40 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 33495.84 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930185.479421 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301462.56 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 30146.26 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960331.735274 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271316.30 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 27131.63 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987463.365542 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244184.67 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 24418.47 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011881.832783 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219766.21 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 21976.62 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033858.453300 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197789.58 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 19778.96 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053637.411765 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178010.63 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 17801.06 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071438.474384 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160209.56 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 16020.96 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087459.430741 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144188.61 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 14418.86 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101878.291462 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129769.75 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 12976.97 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114855.266111 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116792.77 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 11679.28 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126534.543295 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105113.49 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 10511.35 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137045.892761 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94602.15 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 9460.21 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146506.107280 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85141.93 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 8514.19 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155020.300347 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76627.74 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 7662.77 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162683.074108 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68964.96 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 6896.50 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169579.570492 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62068.47 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 6206.85 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175786.417238 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55861.62 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 5586.16 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181372.579310 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50275.46 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 5027.55 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186400.125174 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45247.91 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 4524.79 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190924.916452 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40723.12 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 4072.31 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194997.228602 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36650.81 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3665.08 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198662.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32985.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202262.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29385.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205862.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25785.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209462.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22185.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213062.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18585.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216662.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14985.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220262.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11385.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223862.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7785.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227462.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4185.73 sec) 13:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231062.309537 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 585.73 sec) 13:22:00 [GPUGRID] [rr_sim] 2231648.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981431690.27G/439.78G) 13:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852808.95 13:22:00 [---] pick_jobs_to_run() start 13:22:00 [---] pick_jobs_to_run() end 13:22:00 [---] [rr_sim] end 13:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:22:00 [---] [cpu_sched_debug] preliminary job list: 13:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:22:00 [---] [cpu_sched_debug] final job list: 13:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:22:00 [---] [cpu_sched_debug] enforce_run_list: end 13:22:00 [---] choose_project(): 48120.000000 13:22:00 [WUProp@Home] doing RPC to get work 13:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387479.09 cpu 2134292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134292.90 13:22:00 [---] Got 0 tasks 13:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:23:00 [---] [rr_sim] doing sim: CPU sched 13:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981404099.91G/439.78G) 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231585.30 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 223158.53 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223158.530107 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008426.77 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 200842.68 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 424001.207203 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807584.09 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 180758.41 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604759.616590 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626825.68 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 162682.57 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767442.185038 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464143.12 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 146414.31 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913856.496641 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317728.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 131772.88 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045629.377084 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185955.92 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 118595.59 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164224.969483 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067360.33 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 106736.03 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270961.002641 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960624.30 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 96062.43 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1367023.432484 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864561.87 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 86456.19 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453479.619343 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778105.68 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 77810.57 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531290.187516 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700295.11 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 70029.51 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601319.698871 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630265.60 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 63026.56 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664346.259091 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567239.04 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 56723.90 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721070.163289 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510515.14 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 51051.51 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772121.677067 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459463.62 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 45946.36 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818068.039467 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413517.26 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 41351.73 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859419.765628 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372165.54 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 37216.55 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896636.319172 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334948.98 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 33494.90 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930131.217362 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301454.08 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 30145.41 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960276.625733 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271308.68 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 27130.87 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987407.493266 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244177.81 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 24417.78 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011825.274047 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219760.03 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 21976.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033801.276749 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197784.02 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 19778.40 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053579.679181 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178005.62 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 17800.56 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071380.241370 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160205.06 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 16020.51 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087400.747340 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144184.55 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 14418.46 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101819.202713 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129766.10 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 12976.61 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114795.812549 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116789.49 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 11678.95 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126474.761401 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105110.54 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 10511.05 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136985.815368 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94599.49 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 9459.95 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146445.763938 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85139.54 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 8513.95 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154959.717651 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76625.58 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 7662.56 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162622.275993 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68963.03 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 6896.30 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169518.578501 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62066.72 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 6206.67 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175725.250758 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55860.05 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 5586.01 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181311.255789 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50274.05 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 5027.40 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186338.660317 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45246.64 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 4524.66 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190863.324392 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40721.98 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 4072.20 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194935.522060 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36649.78 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3664.98 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198600.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32984.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202200.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29384.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205800.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25784.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209400.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22184.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213000.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18584.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216600.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14984.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220200.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11384.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223800.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7784.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227400.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4184.80 sec) 13:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2231000.499961 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 584.80 sec) 13:23:00 [GPUGRID] [rr_sim] 2231585.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981404099.91G/439.78G) 13:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852806.21 13:23:00 [---] pick_jobs_to_run() start 13:23:00 [---] pick_jobs_to_run() end 13:23:00 [---] [rr_sim] end 13:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:23:00 [---] [cpu_sched_debug] preliminary job list: 13:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:23:00 [---] [cpu_sched_debug] final job list: 13:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:23:00 [---] [cpu_sched_debug] enforce_run_list: end 13:23:00 [---] choose_project(): 48180.000000 13:23:00 [WUProp@Home] doing RPC to get work 13:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387419.09 cpu 2134232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134232.90 13:23:00 [---] Got 0 tasks 13:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:24:00 [---] [rr_sim] doing sim: CPU sched 13:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981376509.56G/439.78G) 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231522.56 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 223152.26 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223152.256419 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008370.31 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 200837.03 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423989.287196 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807533.28 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 180753.33 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604742.614895 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626779.95 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 162677.99 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767420.609824 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464101.95 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 146410.20 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913830.805261 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317691.76 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 131769.18 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045599.981153 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185922.58 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 118592.26 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164192.239457 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067330.32 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 106733.03 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270925.271930 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960597.29 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 96059.73 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366985.001156 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864537.56 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 86453.76 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453438.757459 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778083.81 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 77808.38 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531247.138132 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700275.43 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 70027.54 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601274.680737 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630247.88 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 63024.79 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664299.469083 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567223.10 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 56722.31 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1721021.778593 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510500.79 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 51050.08 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772071.857153 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459450.71 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 45945.07 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1818016.927856 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413505.64 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 41350.56 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859367.491489 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372155.07 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 37215.51 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896582.998759 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334939.57 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 33493.96 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930076.955302 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301445.61 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 30144.56 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960221.516191 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271301.05 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 27130.10 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987351.620990 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244170.94 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 24417.09 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011768.715310 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219753.85 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 21975.38 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033744.100198 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197778.46 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 19777.85 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053521.946597 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178000.62 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 17800.06 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071322.008356 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160200.56 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 16020.06 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087342.063939 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144180.50 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 14418.05 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101760.113964 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129762.45 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 12976.25 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114736.358986 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116786.21 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 11678.62 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126414.979507 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105107.58 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 10510.76 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136925.737975 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94596.83 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 9459.68 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146385.420596 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85137.14 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 8513.71 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154899.134955 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76623.43 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 7662.34 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162561.477879 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68961.09 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 6896.11 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169457.586510 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62064.98 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 6206.50 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175664.084277 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55858.48 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 5585.85 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181249.932268 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50272.63 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 5027.26 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186277.195460 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45245.37 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 4524.54 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190801.732333 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40720.83 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 4072.08 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194873.815519 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36648.75 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3664.87 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198538.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32983.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202138.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29383.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205738.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25783.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209338.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22183.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212938.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18583.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216538.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14983.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220138.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11383.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223738.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7783.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227338.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4183.87 sec) 13:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230938.690386 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 583.87 sec) 13:24:00 [GPUGRID] [rr_sim] 2231522.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981376509.56G/439.78G) 13:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852803.48 13:24:00 [---] pick_jobs_to_run() start 13:24:00 [---] pick_jobs_to_run() end 13:24:00 [---] [rr_sim] end 13:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:24:00 [---] [cpu_sched_debug] preliminary job list: 13:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:24:00 [---] [cpu_sched_debug] final job list: 13:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:24:00 [---] [cpu_sched_debug] enforce_run_list: end 13:24:00 [---] choose_project(): 48240.000000 13:24:00 [WUProp@Home] doing RPC to get work 13:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387359.09 cpu 2134172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134172.90 13:24:00 [---] Got 0 tasks 13:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:25:00 [---] [rr_sim] doing sim: CPU sched 13:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981348919.20G/439.78G) 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231459.83 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 223145.98 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223145.982731 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008313.84 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 200831.38 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423977.367188 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807482.46 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 180748.25 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604725.613200 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626734.21 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 162673.42 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767399.034610 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464060.79 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 146406.08 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913805.113880 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317654.71 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 131765.47 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045570.585223 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185889.24 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 118588.92 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164159.509431 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067300.32 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 106730.03 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270889.541218 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960570.29 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 96057.03 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366946.569827 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864513.26 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 86451.33 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453397.895575 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778061.93 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 77806.19 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531204.088748 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700255.74 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 70025.57 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601229.662604 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630230.16 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 63023.02 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664252.679074 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567207.15 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 56720.71 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720973.393897 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510486.43 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 51048.64 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1772022.037238 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459437.79 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 45943.78 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817965.816245 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413494.01 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 41349.40 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859315.217351 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372144.61 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 37214.46 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896529.678346 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334930.15 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 33493.01 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930022.693242 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301437.13 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 30143.71 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960166.406649 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271293.42 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 27129.34 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987295.748714 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244164.08 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 24416.41 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011712.156574 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219747.67 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 21974.77 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033686.923647 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197772.90 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 19777.29 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053464.214013 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177995.61 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 17799.56 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071263.775342 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160196.05 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 16019.61 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087283.380538 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144176.45 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 14417.64 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101701.025215 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129758.80 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 12975.88 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114676.905424 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116782.92 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 11678.29 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126355.197612 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105104.63 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 10510.46 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136865.660582 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94594.17 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 9459.42 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146325.077254 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85134.75 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 8513.48 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154838.552259 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76621.28 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 7662.13 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162500.679764 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68959.15 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 6895.91 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169396.594518 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62063.23 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 6206.32 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175602.917797 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55856.91 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 5585.69 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181188.608748 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50271.22 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 5027.12 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186215.730604 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45244.10 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 4524.41 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190740.140274 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40719.69 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 4071.97 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194812.108977 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36647.72 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3664.77 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198476.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32982.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202076.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29382.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205676.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25782.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209276.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22182.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212876.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18582.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216476.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14982.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220076.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11382.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223676.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7782.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227276.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4182.95 sec) 13:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230876.880810 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 582.95 sec) 13:25:00 [GPUGRID] [rr_sim] 2231459.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981348919.20G/439.78G) 13:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852800.74 13:25:00 [---] pick_jobs_to_run() start 13:25:00 [---] pick_jobs_to_run() end 13:25:00 [---] [rr_sim] end 13:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:25:00 [---] [cpu_sched_debug] preliminary job list: 13:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:25:00 [---] [cpu_sched_debug] final job list: 13:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:25:00 [---] [cpu_sched_debug] enforce_run_list: end 13:25:00 [---] choose_project(): 48300.000000 13:25:00 [WUProp@Home] doing RPC to get work 13:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387299.09 cpu 2134112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134112.90 13:25:00 [---] Got 0 tasks 13:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:26:00 [---] [rr_sim] doing sim: CPU sched 13:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981321328.85G/439.78G) 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231397.09 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 223139.71 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223139.709042 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008257.38 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 200825.74 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423965.447180 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807431.64 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 180743.16 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604708.611505 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626688.48 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 162668.85 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767377.459397 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1464019.63 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 146401.96 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913779.422499 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317617.67 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 131761.77 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045541.189292 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185855.90 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 118585.59 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164126.779405 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067270.31 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 106727.03 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270853.810507 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960543.28 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 96054.33 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366908.138499 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864488.95 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 86448.90 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453357.033691 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778040.06 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 77804.01 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531161.039364 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700236.05 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 70023.61 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601184.644470 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630212.45 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 63021.24 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664205.889066 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567191.20 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 56719.12 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720925.009201 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510472.08 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 51047.21 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771972.217324 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459424.87 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 45942.49 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817914.704634 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413482.39 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 41348.24 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859262.943213 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372134.15 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 37213.41 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896476.357934 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334920.73 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 33492.07 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929968.431183 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301428.66 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 30142.87 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960111.297107 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271285.79 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 27128.58 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987239.876439 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244157.21 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 24415.72 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011655.597837 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219741.49 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 21974.15 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033629.747096 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197767.34 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 19776.73 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053406.481428 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177990.61 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 17799.06 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071205.542328 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160191.55 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 16019.15 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087224.697138 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144172.39 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 14417.24 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101641.936466 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129755.15 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 12975.52 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114617.451862 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116779.64 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 11677.96 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126295.415718 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105101.67 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 10510.17 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136805.583189 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94591.51 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 9459.15 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146264.733912 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85132.36 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 8513.24 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154777.969563 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76619.12 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 7661.91 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162439.881649 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68957.21 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 6895.72 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169335.602527 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62061.49 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 6206.15 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175541.751316 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55855.34 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 5585.53 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181127.285227 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50269.81 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 5026.98 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186154.265747 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45242.82 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 4524.28 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190678.548215 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40718.54 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 4071.85 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194750.402435 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36646.69 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3664.67 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198415.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32982.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202015.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29382.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205615.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25782.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209215.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22182.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212815.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18582.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216415.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14982.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220015.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11382.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223615.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7782.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227215.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4182.02 sec) 13:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230815.071234 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 582.02 sec) 13:26:00 [GPUGRID] [rr_sim] 2231397.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981321328.85G/439.78G) 13:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852798.00 13:26:00 [---] pick_jobs_to_run() start 13:26:00 [---] pick_jobs_to_run() end 13:26:00 [---] [rr_sim] end 13:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:26:00 [---] [cpu_sched_debug] preliminary job list: 13:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:26:00 [---] [cpu_sched_debug] final job list: 13:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:26:00 [---] [cpu_sched_debug] enforce_run_list: end 13:26:00 [---] choose_project(): 48360.000000 13:26:00 [WUProp@Home] doing RPC to get work 13:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387239.09 cpu 2134052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2134052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2134052.90 13:26:00 [---] Got 0 tasks 13:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:27:00 [---] [rr_sim] doing sim: CPU sched 13:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981293738.49G/439.78G) 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231334.35 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 223133.44 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223133.435354 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008200.92 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 200820.09 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423953.527173 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807380.83 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 180738.08 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604691.609810 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626642.74 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 162664.27 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767355.884183 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463978.47 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 146397.85 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913753.731119 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317580.62 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 131758.06 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045511.793361 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185822.56 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 118582.26 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164094.049379 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067240.30 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 106724.03 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270818.079795 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960516.27 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 96051.63 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366869.707170 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864464.65 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 86446.46 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453316.171807 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 778018.18 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 77801.82 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531117.989981 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700216.36 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 70021.64 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601139.626337 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630194.73 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 63019.47 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664159.099057 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567175.25 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 56717.53 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720876.624506 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510457.73 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 51045.77 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771922.397409 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459411.96 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 45941.20 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817863.593022 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413470.76 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 41347.08 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859210.669074 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372123.68 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 37212.37 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896423.037521 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334911.32 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 33491.13 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929914.169123 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301420.18 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 30142.02 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960056.187565 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271278.17 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 27127.82 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987184.004163 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244150.35 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 24415.03 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011599.039100 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219735.31 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 21973.53 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033572.570545 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197761.78 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 19776.18 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053348.748844 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177985.60 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 17798.56 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071147.309314 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160187.04 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 16018.70 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087166.013737 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144168.34 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 14416.83 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101582.847717 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129751.51 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 12975.15 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114557.998300 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116776.36 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 11677.64 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126235.633824 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105098.72 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 10509.87 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136745.505796 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94588.85 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 9458.88 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146204.390570 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85129.96 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 8513.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154717.386867 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76616.97 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 7661.70 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162379.083535 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68955.27 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 6895.53 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169274.610535 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62059.74 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 6205.97 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175480.584836 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55853.77 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 5585.38 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181065.961707 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50268.39 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 5026.84 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186092.800890 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45241.55 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 4524.16 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190616.956155 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40717.40 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 4071.74 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194688.695894 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36645.66 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3664.57 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198353.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32981.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201953.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29381.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205553.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25781.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209153.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22181.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212753.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18581.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216353.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14981.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219953.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11381.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223553.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7781.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227153.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4181.09 sec) 13:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230753.261659 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 581.09 sec) 13:27:00 [GPUGRID] [rr_sim] 2231334.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981293738.49G/439.78G) 13:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852795.27 13:27:00 [---] pick_jobs_to_run() start 13:27:00 [---] pick_jobs_to_run() end 13:27:00 [---] [rr_sim] end 13:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:27:00 [---] [cpu_sched_debug] preliminary job list: 13:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:27:00 [---] [cpu_sched_debug] final job list: 13:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:27:00 [---] [cpu_sched_debug] enforce_run_list: end 13:27:00 [---] choose_project(): 48420.000000 13:27:00 [WUProp@Home] doing RPC to get work 13:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387179.09 cpu 2133992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133992.90 13:27:00 [---] Got 0 tasks 13:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:28:00 [---] [rr_sim] doing sim: CPU sched 13:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981266148.13G/439.78G) 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231271.62 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 223127.16 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223127.161666 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008144.45 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 200814.45 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423941.607165 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807330.01 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 180733.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604674.608115 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626597.01 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 162659.70 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767334.308969 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463937.31 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 146393.73 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913728.039738 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317543.58 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 131754.36 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045482.397430 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185789.22 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 118578.92 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164061.319353 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067210.30 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 106721.03 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270782.349084 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960489.27 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 96048.93 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366831.275841 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864440.34 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 86444.03 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453275.309923 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777996.31 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 77799.63 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531074.940597 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700196.68 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 70019.67 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601094.608203 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630177.01 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 63017.70 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664112.309049 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567159.31 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 56715.93 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720828.239810 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510443.38 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 51044.34 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771872.577495 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459399.04 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 45939.90 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817812.481411 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413459.14 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 41345.91 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859158.394936 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372113.22 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 37211.32 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896369.717108 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334901.90 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 33490.19 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929859.907063 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301411.71 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 30141.17 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960001.078023 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271270.54 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 27127.05 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987128.131887 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244143.48 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 24414.35 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011542.480364 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219729.14 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 21972.91 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033515.393993 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197756.22 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 19775.62 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053291.016260 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177980.60 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 17798.06 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071089.076300 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160182.54 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 16018.25 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087107.330336 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144164.29 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 14416.43 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101523.758968 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129747.86 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 12974.79 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114498.544737 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116773.07 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 11677.31 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126175.851930 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105095.76 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 10509.58 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136685.428403 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94586.19 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 9458.62 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146144.047228 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85127.57 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 8512.76 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154656.804171 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76614.81 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 7661.48 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162318.285420 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68953.33 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 6895.33 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169213.618544 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62058.00 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 6205.80 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175419.418356 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55852.20 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 5585.22 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181004.638186 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50266.98 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 5026.70 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186031.336033 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45240.28 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 4524.03 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190555.364096 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40716.25 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 4071.63 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194626.989352 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36644.63 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3664.46 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198291.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32980.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201891.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29380.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205491.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25780.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209091.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22180.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212691.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18580.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216291.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14980.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219891.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11380.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223491.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7780.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227091.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4180.16 sec) 13:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230691.452083 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 580.16 sec) 13:28:00 [GPUGRID] [rr_sim] 2231271.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981266148.13G/439.78G) 13:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852792.53 13:28:00 [---] pick_jobs_to_run() start 13:28:00 [---] pick_jobs_to_run() end 13:28:00 [---] [rr_sim] end 13:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:28:00 [---] [cpu_sched_debug] preliminary job list: 13:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:28:00 [---] [cpu_sched_debug] final job list: 13:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:28:00 [---] [cpu_sched_debug] enforce_run_list: end 13:28:00 [---] choose_project(): 48480.000000 13:28:00 [WUProp@Home] doing RPC to get work 13:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387119.09 cpu 2133932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133932.90 13:28:00 [---] Got 0 tasks 13:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:29:00 [---] [rr_sim] doing sim: CPU sched 13:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981238557.78G/439.78G) 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231208.88 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 223120.89 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223120.887978 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008087.99 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 200808.80 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423929.687158 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807279.19 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 180727.92 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604657.606420 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626551.27 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 162655.13 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767312.733755 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463896.15 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 146389.61 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913702.348358 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317506.53 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 131750.65 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045453.001500 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185755.88 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 118575.59 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1164028.589327 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067180.29 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 106718.03 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270746.618372 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960462.26 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 96046.23 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366792.844513 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864416.04 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 86441.60 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453234.448039 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777974.43 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 77797.44 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1531031.891213 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700176.99 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 70017.70 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601049.590069 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630159.29 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 63015.93 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664065.519040 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567143.36 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 56714.34 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720779.855114 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510429.02 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 51042.90 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771822.757580 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459386.12 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 45938.61 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817761.369800 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413447.51 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 41344.75 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859106.120798 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372102.76 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 37210.28 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896316.396696 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334892.48 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 33489.25 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929805.645004 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301403.23 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 30140.32 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959945.968481 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271262.91 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 27126.29 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987072.259611 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244136.62 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 24413.66 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011485.921627 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219722.96 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 21972.30 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033458.217442 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197750.66 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 19775.07 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053233.283676 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177975.60 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 17797.56 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071030.843286 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160178.04 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 16017.80 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087048.646935 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144160.23 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 14416.02 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101464.670219 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129744.21 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 12974.42 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114439.091175 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116769.79 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 11676.98 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126116.070035 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105092.81 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 10509.28 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136625.351009 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94583.53 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 9458.35 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146083.703886 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85125.18 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 8512.52 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154596.221475 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76612.66 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 7661.27 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162257.487306 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68951.39 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 6895.14 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169152.626553 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62056.25 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 6205.63 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175358.251875 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55850.63 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 5585.06 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180943.314665 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50265.57 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 5026.56 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185969.871177 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45239.01 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 4523.90 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190493.772037 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40715.11 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 4071.51 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194565.282811 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36643.60 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3664.36 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198229.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32979.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201829.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29379.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205429.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25779.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209029.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22179.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212629.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18579.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216229.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14979.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219829.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11379.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223429.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7779.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227029.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4179.24 sec) 13:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230629.642507 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 579.24 sec) 13:29:00 [GPUGRID] [rr_sim] 2231208.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981238557.78G/439.78G) 13:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852789.79 13:29:00 [---] pick_jobs_to_run() start 13:29:00 [---] pick_jobs_to_run() end 13:29:00 [---] [rr_sim] end 13:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:29:00 [---] [cpu_sched_debug] preliminary job list: 13:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:29:00 [---] [cpu_sched_debug] final job list: 13:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:29:00 [---] [cpu_sched_debug] enforce_run_list: end 13:29:00 [---] choose_project(): 48540.000000 13:29:00 [WUProp@Home] doing RPC to get work 13:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 387059.09 cpu 2133872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133872.90 13:29:00 [---] Got 0 tasks 13:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:30:00 [---] [rr_sim] doing sim: CPU sched 13:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981210967.42G/439.78G) 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231146.14 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 223114.61 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223114.614290 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2008031.53 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 200803.15 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423917.767150 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807228.38 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 180722.84 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604640.604725 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626505.54 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 162650.55 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767291.158542 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463854.98 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 146385.50 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913676.656977 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317469.49 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 131746.95 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045423.605569 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185722.54 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 118572.25 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163995.859301 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067150.28 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 106715.03 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270710.887661 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960435.26 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 96043.53 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366754.413184 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864391.73 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 86439.17 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453193.586155 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777952.56 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 77795.26 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530988.841829 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700157.30 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 70015.73 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1601004.571936 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630141.57 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 63014.16 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1664018.729032 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567127.41 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 56712.74 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720731.470418 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510414.67 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 51041.47 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771772.937666 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459373.21 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 45937.32 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817710.258189 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413435.88 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 41343.59 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859053.846659 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372092.30 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 37209.23 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896263.076283 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334883.07 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 33488.31 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929751.382944 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301394.76 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 30139.48 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959890.858939 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271255.28 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 27125.53 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987016.387335 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244129.76 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 24412.98 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011429.362891 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219716.78 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 21971.68 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033401.040891 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197745.10 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 19774.51 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053175.551092 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177970.59 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 17797.06 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070972.610272 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160173.53 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 16017.35 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086989.963534 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144156.18 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 14415.62 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101405.581470 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129740.56 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 12974.06 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114379.637613 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116766.51 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 11676.65 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126056.288141 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105089.85 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 10508.99 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136565.273616 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94580.87 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 9458.09 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146023.360544 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85122.78 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 8512.28 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154535.638779 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76610.50 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 7661.05 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162196.689191 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68949.45 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 6894.95 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169091.634561 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62054.51 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 6205.45 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175297.085395 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55849.06 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 5584.91 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180881.991145 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50264.15 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 5026.42 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185908.406320 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45237.74 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 4523.77 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190432.179977 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40713.96 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 4071.40 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194503.576269 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36642.57 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3664.26 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198167.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32978.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201767.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29378.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205367.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25778.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208967.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22178.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212567.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18578.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216167.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14978.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219767.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11378.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223367.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7778.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226967.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4178.31 sec) 13:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230567.832932 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 578.31 sec) 13:30:00 [GPUGRID] [rr_sim] 2231146.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981210967.42G/439.78G) 13:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852787.06 13:30:00 [---] pick_jobs_to_run() start 13:30:00 [---] pick_jobs_to_run() end 13:30:00 [---] [rr_sim] end 13:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:30:00 [---] [cpu_sched_debug] preliminary job list: 13:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:30:00 [---] [cpu_sched_debug] final job list: 13:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:30:00 [---] [cpu_sched_debug] enforce_run_list: end 13:30:00 [---] choose_project(): 48600.000000 13:30:00 [WUProp@Home] doing RPC to get work 13:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386999.09 cpu 2133812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133812.90 13:30:00 [---] Got 0 tasks 13:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:31:00 [---] [rr_sim] doing sim: CPU sched 13:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981183377.06G/439.78G) 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231083.41 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 223108.34 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223108.340601 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007975.07 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 200797.51 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423905.847142 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807177.56 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 180717.76 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604623.603029 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626459.80 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 162645.98 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767269.583328 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463813.82 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 146381.38 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913650.965596 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317432.44 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 131743.24 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045394.209638 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185689.20 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 118568.92 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163963.129275 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067120.28 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 106712.03 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270675.156949 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960408.25 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 96040.82 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366715.981856 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864367.42 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 86436.74 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453152.724271 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777930.68 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 77793.07 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530945.792445 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700137.61 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 70013.76 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600959.553802 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630123.85 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 63012.39 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663971.939023 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567111.47 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 56711.15 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720683.085722 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510400.32 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 51040.03 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771723.117751 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459360.29 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 45936.03 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817659.146577 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413424.26 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 41342.43 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859001.572521 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372081.83 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 37208.18 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896209.755870 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334873.65 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 33487.37 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929697.120884 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301386.29 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 30138.63 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959835.749397 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271247.66 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 27124.77 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986960.515059 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244122.89 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 24412.29 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011372.804154 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219710.60 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 21971.06 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033343.864340 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197739.54 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 19773.95 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053117.818507 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177965.59 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 17796.56 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070914.377258 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160169.03 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 16016.90 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086931.280133 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144152.13 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 14415.21 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101346.492721 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129736.91 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 12973.69 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114320.184050 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116763.22 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 11676.32 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125996.506247 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105086.90 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 10508.69 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136505.196223 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94578.21 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 9457.82 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145963.017202 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85120.39 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 8512.04 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154475.056083 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76608.35 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 7660.83 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162135.891076 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68947.51 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 6894.75 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169030.642570 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62052.76 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 6205.28 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175235.918914 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55847.49 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 5584.75 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180820.667624 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50262.74 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 5026.27 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185846.941463 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45236.46 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 4523.65 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190370.587918 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40712.82 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 4071.28 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194441.869727 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36641.54 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3664.15 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198106.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32977.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201706.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29377.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205306.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25777.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208906.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22177.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212506.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18577.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216106.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14977.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219706.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11377.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223306.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7777.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226906.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4177.38 sec) 13:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230506.023356 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577.38 sec) 13:31:00 [GPUGRID] [rr_sim] 2231083.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981183377.06G/439.78G) 13:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852784.32 13:31:00 [---] pick_jobs_to_run() start 13:31:00 [---] pick_jobs_to_run() end 13:31:00 [---] [rr_sim] end 13:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:31:00 [---] [cpu_sched_debug] preliminary job list: 13:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:31:00 [---] [cpu_sched_debug] final job list: 13:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:31:00 [---] [cpu_sched_debug] enforce_run_list: end 13:31:00 [---] choose_project(): 48660.000000 13:31:00 [WUProp@Home] doing RPC to get work 13:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386939.09 cpu 2133752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133752.90 13:31:00 [---] Got 0 tasks 13:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:32:00 [---] [rr_sim] doing sim: CPU sched 13:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981183377.06G/439.78G) 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231083.41 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 223108.34 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223108.340601 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007975.07 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 200797.51 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423905.847142 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807177.56 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 180717.76 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604623.603029 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626459.80 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 162645.98 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767269.583328 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463813.82 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 146381.38 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913650.965596 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317432.44 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 131743.24 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045394.209638 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185689.20 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 118568.92 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163963.129275 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067120.28 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 106712.03 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270675.156949 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960408.25 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 96040.82 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366715.981856 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864367.42 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 86436.74 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453152.724271 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777930.68 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 77793.07 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530945.792445 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700137.61 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 70013.76 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600959.553802 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630123.85 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 63012.39 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663971.939023 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567111.47 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 56711.15 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720683.085722 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510400.32 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 51040.03 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771723.117751 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459360.29 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 45936.03 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817659.146577 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413424.26 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 41342.43 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1859001.572521 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372081.83 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 37208.18 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896209.755870 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334873.65 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 33487.37 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929697.120884 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301386.29 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 30138.63 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959835.749397 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271247.66 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 27124.77 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986960.515059 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244122.89 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 24412.29 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011372.804154 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219710.60 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 21971.06 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033343.864340 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197739.54 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 19773.95 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053117.818507 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177965.59 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 17796.56 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070914.377258 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160169.03 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 16016.90 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086931.280133 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144152.13 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 14415.21 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101346.492721 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129736.91 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 12973.69 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114320.184050 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116763.22 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 11676.32 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125996.506247 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105086.90 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 10508.69 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136505.196223 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94578.21 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 9457.82 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145963.017202 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85120.39 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 8512.04 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154475.056083 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76608.35 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 7660.83 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162135.891076 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68947.51 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 6894.75 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169030.642570 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62052.76 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 6205.28 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175235.918914 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55847.49 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 5584.75 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180820.667624 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50262.74 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 5026.27 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185846.941463 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45236.46 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 4523.65 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190370.587918 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40712.82 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 4071.28 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194441.869727 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36641.54 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3664.15 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198106.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32977.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201706.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29377.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205306.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25777.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208906.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22177.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212506.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18577.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216106.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14977.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219706.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11377.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223306.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7777.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226906.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4177.38 sec) 13:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230506.023356 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 577.38 sec) 13:32:00 [GPUGRID] [rr_sim] 2231083.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981183377.06G/439.78G) 13:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852844.32 13:32:00 [---] pick_jobs_to_run() start 13:32:00 [---] pick_jobs_to_run() end 13:32:00 [---] [rr_sim] end 13:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:32:00 [---] [cpu_sched_debug] preliminary job list: 13:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:32:00 [---] [cpu_sched_debug] final job list: 13:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:32:00 [---] [cpu_sched_debug] enforce_run_list: end 13:32:00 [---] choose_project(): 48720.000000 13:32:00 [WUProp@Home] doing RPC to get work 13:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386879.09 cpu 2133752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133752.90 13:32:00 [---] Got 0 tasks 13:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:33:00 [---] [rr_sim] doing sim: CPU sched 13:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981155786.71G/439.78G) 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231020.67 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 223102.07 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223102.066913 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007918.60 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 200791.86 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423893.927135 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807126.74 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 180712.67 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604606.601334 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626414.07 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 162641.41 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767248.008114 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463772.66 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 146377.27 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913625.274216 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317395.39 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 131739.54 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045364.813707 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185655.86 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 118565.59 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163930.399250 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067090.27 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 106709.03 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270639.426238 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960381.24 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 96038.12 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366677.550527 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864343.12 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 86434.31 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453111.862387 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777908.81 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 77790.88 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530902.743062 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700117.93 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 70011.79 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600914.535669 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630106.13 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 63010.61 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663925.149015 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567095.52 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 56709.55 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720634.701026 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510385.97 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 51038.60 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771673.297837 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459347.37 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 45934.74 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817608.034966 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413412.63 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 41341.26 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858949.298383 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372071.37 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 37207.14 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896156.435457 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334864.23 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 33486.42 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929642.858825 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301377.81 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 30137.78 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959780.639855 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271240.03 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 27124.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986904.642783 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244116.03 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 24411.60 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011316.245418 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219704.42 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 21970.44 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033286.687789 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197733.98 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 19773.40 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053060.085923 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177960.58 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 17796.06 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070856.144244 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160164.52 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 16016.45 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086872.596733 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144148.07 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 14414.81 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101287.403972 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129733.27 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 12973.33 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114260.730488 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116759.94 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 11675.99 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125936.724352 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105083.94 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 10508.39 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136445.118830 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94575.55 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 9457.56 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145902.673860 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85118.00 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 8511.80 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154414.473387 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76606.20 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 7660.62 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162075.092962 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68945.58 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 6894.56 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168969.650579 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62051.02 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 6205.10 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175174.752434 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55845.92 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 5584.59 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180759.344103 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50261.33 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 5026.13 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185785.476606 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45235.19 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 4523.52 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190308.995859 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40711.67 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 4071.17 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194380.163186 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36640.51 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3664.05 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198044.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32976.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201644.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29376.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205244.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25776.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208844.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22176.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212444.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18576.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216044.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14976.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219644.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11376.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223244.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7776.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226844.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4176.46 sec) 13:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230444.213780 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576.46 sec) 13:33:00 [GPUGRID] [rr_sim] 2231020.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981155786.71G/439.78G) 13:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852841.58 13:33:00 [---] pick_jobs_to_run() start 13:33:00 [---] pick_jobs_to_run() end 13:33:00 [---] [rr_sim] end 13:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:33:00 [---] [cpu_sched_debug] preliminary job list: 13:33:00 [---] [cpu_sched_debug] final job list: 13:33:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 13:33:00 [---] [work_fetch] Request work fetch: CPUs idle 13:33:00 [---] [cpu_sched_debug] enforce_run_list: end 13:33:00 [---] choose_project(): 48780.000000 13:33:00 [WUProp@Home] doing RPC to get work 13:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386819.09 cpu 2133692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133692.90 13:33:00 [---] Got 0 tasks 13:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:34:00 [---] [rr_sim] doing sim: CPU sched 13:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981155786.71G/439.78G) 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2231020.67 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 223102.07 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223102.066913 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007918.60 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 200791.86 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423893.927135 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807126.74 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 180712.67 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604606.601334 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626414.07 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 162641.41 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767248.008114 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463772.66 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 146377.27 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913625.274216 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317395.39 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 131739.54 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045364.813707 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185655.86 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 118565.59 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163930.399250 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067090.27 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 106709.03 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270639.426238 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960381.24 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 96038.12 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366677.550527 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864343.12 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 86434.31 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453111.862387 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777908.81 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 77790.88 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530902.743062 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700117.93 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 70011.79 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600914.535669 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630106.13 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 63010.61 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663925.149015 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567095.52 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 56709.55 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720634.701026 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510385.97 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 51038.60 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771673.297837 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459347.37 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 45934.74 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817608.034966 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413412.63 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 41341.26 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858949.298383 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372071.37 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 37207.14 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896156.435457 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334864.23 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 33486.42 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929642.858825 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301377.81 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 30137.78 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959780.639855 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271240.03 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 27124.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986904.642783 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244116.03 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 24411.60 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011316.245418 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219704.42 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 21970.44 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033286.687789 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197733.98 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 19773.40 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053060.085923 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177960.58 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 17796.06 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070856.144244 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160164.52 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 16016.45 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086872.596733 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144148.07 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 14414.81 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101287.403972 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129733.27 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 12973.33 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114260.730488 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116759.94 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 11675.99 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125936.724352 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105083.94 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 10508.39 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136445.118830 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94575.55 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 9457.56 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145902.673860 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85118.00 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 8511.80 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154414.473387 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76606.20 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 7660.62 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162075.092962 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68945.58 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 6894.56 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168969.650579 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62051.02 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 6205.10 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175174.752434 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55845.92 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 5584.59 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180759.344103 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50261.33 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 5026.13 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185785.476606 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45235.19 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 4523.52 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190308.995859 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40711.67 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 4071.17 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194380.163186 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36640.51 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3664.05 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198044.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32976.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201644.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29376.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205244.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25776.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208844.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22176.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212444.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18576.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216044.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14976.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219644.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11376.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223244.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7776.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226844.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4176.46 sec) 13:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230444.213780 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 576.46 sec) 13:34:00 [GPUGRID] [rr_sim] 2231020.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981155786.71G/439.78G) 13:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852901.58 13:34:00 [---] pick_jobs_to_run() start 13:34:00 [---] pick_jobs_to_run() end 13:34:00 [---] [rr_sim] end 13:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:34:00 [---] [cpu_sched_debug] preliminary job list: 13:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:34:00 [---] [cpu_sched_debug] final job list: 13:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 13:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:34:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 981155786.71G 13:34:00 [---] [cpu_sched_debug] enforce_run_list: end 13:34:00 [---] choose_project(): 48840.000000 13:34:00 [WUProp@Home] doing RPC to get work 13:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386759.09 cpu 2133692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133692.90 13:34:00 [---] Got 0 tasks 13:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:35:00 [---] [rr_sim] doing sim: CPU sched 13:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981128196.35G/439.78G) 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230957.93 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 223095.79 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223095.793225 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007862.14 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 200786.21 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423882.007127 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807075.93 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 180707.59 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604589.599639 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626368.33 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 162636.83 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767226.432900 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463731.50 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 146373.15 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913599.582835 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317358.35 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 131735.83 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045335.417776 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185622.51 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 118562.25 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163897.669224 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067060.26 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 106706.03 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270603.695526 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960354.24 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 96035.42 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366639.119198 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864318.81 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 86431.88 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453071.000503 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777886.93 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 77788.69 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530859.693678 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700098.24 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 70009.82 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600869.517535 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630088.41 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 63008.84 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663878.359006 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567079.57 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 56707.96 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720586.316331 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510371.62 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 51037.16 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771623.477922 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459334.45 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 45933.45 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817556.923355 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413401.01 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 41340.10 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858897.024244 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372060.91 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 37206.09 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896103.115045 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334854.82 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 33485.48 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929588.596765 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301369.34 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 30136.93 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959725.530313 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271232.40 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 27123.24 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986848.770507 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244109.16 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 24410.92 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011259.686681 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219698.25 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 21969.82 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033229.511238 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197728.42 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 19772.84 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053002.353339 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177955.58 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 17795.56 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070797.911230 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160160.02 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 16016.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086813.913332 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144144.02 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 14414.40 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101228.315223 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129729.62 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 12972.96 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114201.276926 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116756.66 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 11675.67 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125876.942458 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105080.99 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 10508.10 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136385.041437 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94572.89 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 9457.29 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145842.330518 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85115.60 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 8511.56 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154353.890691 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76604.04 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 7660.40 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162014.294847 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68943.64 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 6894.36 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168908.658587 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62049.27 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 6204.93 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175113.585953 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55844.35 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 5584.43 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180698.020583 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50259.91 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 5025.99 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185724.011749 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45233.92 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 4523.39 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190247.403799 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40710.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 4071.05 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194318.456644 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36639.48 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3663.95 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197982.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32975.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201582.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29375.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205182.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25775.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208782.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22175.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212382.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18575.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215982.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14975.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219582.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11375.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223182.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7775.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226782.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4175.53 sec) 13:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230382.404205 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 575.53 sec) 13:35:00 [GPUGRID] [rr_sim] 2230957.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981128196.35G/439.78G) 13:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852898.84 13:35:00 [---] pick_jobs_to_run() start 13:35:00 [---] pick_jobs_to_run() end 13:35:00 [---] [rr_sim] end 13:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:35:00 [---] [cpu_sched_debug] preliminary job list: 13:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 13:35:00 [---] [cpu_sched_debug] final job list: 13:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:35:00 [---] [cpu_sched_debug] enforce_run_list: end 13:35:00 [---] choose_project(): 48900.000000 13:35:00 [WUProp@Home] doing RPC to get work 13:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386699.09 cpu 2133632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133632.90 13:35:00 [---] Got 0 tasks 13:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:36:00 [---] [rr_sim] doing sim: CPU sched 13:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981100606.00G/439.78G) 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230895.20 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 223089.52 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223089.519537 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007805.68 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 200780.57 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423870.087120 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1807025.11 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 180702.51 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604572.597944 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626322.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 162632.26 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767204.857686 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463690.34 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 146369.03 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913573.891454 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317321.30 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 131732.13 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045306.021846 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185589.17 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 118558.92 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163864.939198 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067030.26 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 106703.03 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270567.964815 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960327.23 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 96032.72 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366600.687870 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864294.51 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 86429.45 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1453030.138619 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777865.06 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 77786.51 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530816.644294 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700078.55 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 70007.86 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600824.499401 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630070.70 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 63007.07 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663831.568998 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567063.63 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 56706.36 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720537.931635 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510357.26 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 51035.73 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771573.658008 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459321.54 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 45932.15 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817505.811744 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413389.38 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 41338.94 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858844.750106 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372050.45 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 37205.04 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896049.794632 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334845.40 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 33484.54 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929534.334705 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301360.86 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 30136.09 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959670.420772 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271224.77 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 27122.48 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986792.898231 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244102.30 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 24410.23 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011203.127945 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219692.07 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 21969.21 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033172.334687 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197722.86 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 19772.29 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052944.620755 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177950.57 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 17795.06 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070739.678216 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160155.52 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 16015.55 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086755.229931 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144139.97 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 14414.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101169.226474 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129725.97 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 12972.60 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114141.823364 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116753.37 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 11675.34 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125817.160564 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105078.03 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 10507.80 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136324.964044 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94570.23 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 9457.02 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145781.987176 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85113.21 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 8511.32 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154293.307995 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76601.89 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 7660.19 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161953.496732 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68941.70 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 6894.17 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168847.666596 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62047.53 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 6204.75 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175052.419473 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55842.78 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 5584.28 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180636.697062 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50258.50 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 5025.85 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185662.546893 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45232.65 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 4523.26 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190185.811740 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40709.38 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 4070.94 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194256.750103 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36638.45 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3663.84 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197920.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32974.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201520.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29374.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205120.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25774.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208720.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22174.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212320.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18574.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215920.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14974.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219520.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11374.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223120.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7774.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226720.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4174.60 sec) 13:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230320.594629 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 574.60 sec) 13:36:00 [GPUGRID] [rr_sim] 2230895.20: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981100606.00G/439.78G) 13:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852896.11 13:36:00 [---] pick_jobs_to_run() start 13:36:00 [---] pick_jobs_to_run() end 13:36:00 [---] [rr_sim] end 13:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:36:00 [---] [cpu_sched_debug] preliminary job list: 13:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:36:00 [---] [cpu_sched_debug] final job list: 13:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:36:00 [---] [cpu_sched_debug] enforce_run_list: end 13:36:00 [---] choose_project(): 48960.000000 13:36:00 [WUProp@Home] doing RPC to get work 13:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386639.09 cpu 2133572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133572.90 13:36:00 [---] Got 0 tasks 13:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:37:00 [---] [rr_sim] doing sim: CPU sched 13:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981073015.64G/439.78G) 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230832.46 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 223083.25 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223083.245848 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007749.21 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 200774.92 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423858.167112 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806974.29 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 180697.43 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604555.596249 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626276.86 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 162627.69 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767183.282473 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463649.18 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 146364.92 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913548.200074 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317284.26 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 131728.43 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045276.625915 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185555.83 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 118555.58 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163832.209172 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1067000.25 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 106700.02 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270532.234103 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960300.22 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 96030.02 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366562.256541 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864270.20 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 86427.02 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452989.276735 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777843.18 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 77784.32 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530773.594910 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700058.86 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 70005.89 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600779.481268 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630052.98 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 63005.30 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663784.778989 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567047.68 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 56704.77 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720489.546939 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510342.91 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 51034.29 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771523.838093 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459308.62 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 45930.86 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817454.700132 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413377.76 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 41337.78 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858792.475968 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372039.98 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 37204.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895996.474219 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334835.98 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 33483.60 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929480.072646 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301352.39 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 30135.24 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959615.311230 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271217.15 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 27121.71 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986737.025955 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244095.43 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 24409.54 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011146.569208 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219685.89 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 21968.59 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033115.158136 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197717.30 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 19771.73 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052886.888170 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177945.57 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 17794.56 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070681.445202 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160151.01 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 16015.10 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086696.546530 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144135.91 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 14413.59 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101110.137725 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129722.32 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 12972.23 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114082.369801 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116750.09 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 11675.01 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125757.378670 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105075.08 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 10507.51 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136264.886651 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94567.57 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 9456.76 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145721.643834 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85110.81 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 8511.08 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154232.725299 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76599.73 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 7659.97 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161892.698618 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68939.76 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 6893.98 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168786.674604 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62045.78 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 6204.58 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174991.252992 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55841.21 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 5584.12 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180575.373542 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50257.08 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 5025.71 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185601.082036 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45231.38 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 4523.14 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190124.219681 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40708.24 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 4070.82 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194195.043561 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36637.41 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3663.74 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197858.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32973.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201458.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29373.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205058.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25773.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208658.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22173.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212258.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18573.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215858.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14973.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219458.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11373.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223058.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7773.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226658.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4173.67 sec) 13:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230258.785053 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 573.67 sec) 13:37:00 [GPUGRID] [rr_sim] 2230832.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981073015.64G/439.78G) 13:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852893.37 13:37:00 [---] pick_jobs_to_run() start 13:37:00 [---] pick_jobs_to_run() end 13:37:00 [---] [rr_sim] end 13:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:37:00 [---] [cpu_sched_debug] preliminary job list: 13:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:37:00 [---] [cpu_sched_debug] final job list: 13:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:37:00 [---] [cpu_sched_debug] enforce_run_list: end 13:37:00 [---] choose_project(): 49020.000000 13:37:00 [WUProp@Home] doing RPC to get work 13:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386579.09 cpu 2133512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133512.90 13:37:00 [---] Got 0 tasks 13:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:38:00 [---] [rr_sim] doing sim: CPU sched 13:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981045425.28G/439.78G) 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230769.72 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 223076.97 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223076.972160 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007692.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 200769.27 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423846.247104 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806923.47 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 180692.35 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604538.594554 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626231.13 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 162623.11 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767161.707259 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463608.01 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 146360.80 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913522.508693 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317247.21 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 131724.72 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045247.229984 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185522.49 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 118552.25 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163799.479146 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066970.24 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 106697.02 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270496.503392 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960273.22 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 96027.32 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366523.825213 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864245.90 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 86424.59 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452948.414852 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777821.31 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 77782.13 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530730.545527 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700039.18 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 70003.92 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600734.463134 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630035.26 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 63003.53 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663737.988981 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567031.73 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 56703.17 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720441.162243 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510328.56 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 51032.86 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771474.018179 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459295.70 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 45929.57 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817403.588521 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413366.13 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 41336.61 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858740.201829 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372029.52 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 37202.95 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895943.153807 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334826.57 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 33482.66 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929425.810586 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301343.91 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 30134.39 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959560.201688 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271209.52 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 27120.95 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986681.153679 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244088.57 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 24408.86 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011090.010471 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219679.71 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 21967.97 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033057.981584 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197711.74 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 19771.17 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052829.155586 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177940.57 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 17794.06 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070623.212188 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160146.51 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 16014.65 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086637.863129 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144131.86 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 14413.19 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101051.048977 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129718.67 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 12971.87 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114022.916239 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116746.81 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 11674.68 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125697.596775 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105072.12 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 10507.21 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136204.809258 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94564.91 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 9456.49 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145661.300492 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85108.42 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 8510.84 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154172.142603 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76597.58 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 7659.76 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161831.900503 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68937.82 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 6893.78 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168725.682613 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62044.04 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 6204.40 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174930.086512 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55839.64 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 5583.96 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180514.050021 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50255.67 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 5025.57 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185539.617179 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45230.10 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 4523.01 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190062.627621 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40707.09 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 4070.71 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194133.337019 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36636.38 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3663.64 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197796.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32972.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201396.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29372.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204996.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25772.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208596.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22172.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212196.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18572.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215796.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14972.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219396.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11372.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222996.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7772.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226596.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4172.75 sec) 13:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230196.975478 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 572.75 sec) 13:38:00 [GPUGRID] [rr_sim] 2230769.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981045425.28G/439.78G) 13:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852890.63 13:38:00 [---] pick_jobs_to_run() start 13:38:00 [---] pick_jobs_to_run() end 13:38:00 [---] [rr_sim] end 13:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:38:00 [---] [cpu_sched_debug] preliminary job list: 13:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:38:00 [---] [cpu_sched_debug] final job list: 13:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:38:00 [---] [cpu_sched_debug] enforce_run_list: end 13:38:00 [---] choose_project(): 49080.000000 13:38:00 [WUProp@Home] doing RPC to get work 13:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386519.09 cpu 2133452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133452.90 13:38:00 [---] Got 0 tasks 13:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:39:00 [---] [rr_sim] doing sim: CPU sched 13:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (981017834.93G/439.78G) 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230706.98 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 223070.70 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223070.698472 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007636.29 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 200763.63 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423834.327097 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806872.66 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 180687.27 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604521.592859 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626185.39 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 162618.54 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767140.132045 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463566.85 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 146356.69 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913496.817313 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317210.17 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 131721.02 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045217.834053 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185489.15 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 118548.92 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163766.749120 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066940.24 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 106694.02 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270460.772680 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960246.21 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 96024.62 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366485.393884 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864221.59 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 86422.16 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452907.552968 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777799.43 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 77779.94 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530687.496143 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 700019.49 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 70001.95 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600689.445000 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 630017.54 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 63001.75 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663691.198972 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567015.79 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 56701.58 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720392.777547 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510314.21 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 51031.42 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771424.198264 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459282.79 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 45928.28 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817352.476910 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413354.51 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 41335.45 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858687.927691 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372019.06 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 37201.91 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895889.833394 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334817.15 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 33481.72 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929371.548526 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301335.44 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 30133.54 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959505.092146 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271201.89 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 27120.19 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986625.281403 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244081.70 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 24408.17 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011033.451735 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219673.53 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 21967.35 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033000.805033 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197706.18 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 19770.62 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052771.423002 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177935.56 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 17793.56 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070564.979174 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160142.01 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 16014.20 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086579.179728 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144127.80 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 14412.78 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100991.960228 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129715.02 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 12971.50 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113963.462677 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116743.52 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 11674.35 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125637.814881 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105069.17 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 10506.92 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136144.731865 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94562.25 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 9456.23 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145600.957150 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85106.03 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 8510.60 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154111.559907 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76595.42 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 7659.54 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161771.102389 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68935.88 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 6893.59 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168664.690622 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62042.29 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 6204.23 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174868.920032 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55838.06 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 5583.81 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180452.726500 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50254.26 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 5025.43 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185478.152322 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45228.83 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 4522.88 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190001.035562 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40705.95 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 4070.59 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194071.630478 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36635.35 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3663.54 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197735.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32971.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201335.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29371.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204935.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25771.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208535.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22171.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212135.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18571.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215735.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14971.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219335.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11371.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222935.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7771.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226535.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4171.82 sec) 13:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230135.165902 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 571.82 sec) 13:39:00 [GPUGRID] [rr_sim] 2230706.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (981017834.93G/439.78G) 13:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852887.90 13:39:00 [---] pick_jobs_to_run() start 13:39:00 [---] pick_jobs_to_run() end 13:39:00 [---] [rr_sim] end 13:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:39:00 [---] [cpu_sched_debug] preliminary job list: 13:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:39:00 [---] [cpu_sched_debug] final job list: 13:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:39:00 [---] [cpu_sched_debug] enforce_run_list: end 13:39:00 [---] choose_project(): 49140.000000 13:39:00 [WUProp@Home] doing RPC to get work 13:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386459.09 cpu 2133392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133392.90 13:39:00 [---] Got 0 tasks 13:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:40:00 [---] [rr_sim] doing sim: CPU sched 13:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980990244.57G/439.78G) 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230644.25 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 223064.42 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223064.424784 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007579.82 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 200757.98 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423822.407089 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806821.84 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 180682.18 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604504.591164 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626139.66 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 162613.97 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767118.556831 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463525.69 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 146352.57 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913471.125932 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317173.12 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 131717.31 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045188.438123 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185455.81 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 118545.58 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163734.019094 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066910.23 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 106691.02 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270425.041968 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960219.21 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 96021.92 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366446.962555 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864197.29 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 86419.73 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452866.691084 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777777.56 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 77777.76 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530644.446759 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699999.80 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 69999.98 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600644.426867 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629999.82 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 62999.98 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663644.408964 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566999.84 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 56699.98 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720344.392851 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510299.85 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 51029.99 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771374.378350 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459269.87 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 45926.99 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817301.365299 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413342.88 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 41334.29 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858635.653553 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372008.59 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 37200.86 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895836.512981 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334807.73 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 33480.77 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929317.286467 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301326.96 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 30132.70 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959449.982604 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271194.27 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 27119.43 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986569.409127 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244074.84 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 24407.48 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010976.892998 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219667.35 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 21966.74 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032943.628482 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197700.62 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 19770.06 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052713.690418 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177930.56 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 17793.06 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070506.746160 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160137.50 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 16013.75 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086520.496328 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144123.75 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 14412.38 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100932.871479 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129711.38 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 12971.14 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113904.009114 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116740.24 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 11674.02 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125578.032987 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105066.21 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 10506.62 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136084.654472 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94559.59 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 9455.96 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145540.613808 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85103.63 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 8510.36 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154050.977211 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76593.27 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 7659.33 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161710.304274 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68933.94 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 6893.39 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168603.698630 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62040.55 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 6204.05 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174807.753551 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55836.49 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 5583.65 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180391.402980 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50252.84 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 5025.28 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185416.687466 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45227.56 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 4522.76 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189939.443503 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40704.80 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 4070.48 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194009.923936 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36634.32 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3663.43 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197673.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32970.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201273.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29370.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204873.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25770.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208473.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22170.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212073.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18570.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215673.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14970.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219273.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11370.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222873.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7770.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226473.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4170.89 sec) 13:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230073.356326 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 570.89 sec) 13:40:00 [GPUGRID] [rr_sim] 2230644.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980990244.57G/439.78G) 13:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852885.16 13:40:00 [---] pick_jobs_to_run() start 13:40:00 [---] pick_jobs_to_run() end 13:40:00 [---] [rr_sim] end 13:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:40:00 [---] [cpu_sched_debug] preliminary job list: 13:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:40:00 [---] [cpu_sched_debug] final job list: 13:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:40:00 [---] [cpu_sched_debug] enforce_run_list: end 13:40:00 [---] choose_project(): 49200.000000 13:40:00 [WUProp@Home] doing RPC to get work 13:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386399.09 cpu 2133332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133332.90 13:40:00 [---] Got 0 tasks 13:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:41:00 [---] [rr_sim] doing sim: CPU sched 13:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980962654.21G/439.78G) 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230581.51 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 223058.15 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223058.151096 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007523.36 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 200752.34 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423810.487082 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806771.02 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 180677.10 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604487.589469 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626093.92 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 162609.39 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767096.981618 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463484.53 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 146348.45 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913445.434551 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317136.08 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 131713.61 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045159.042192 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185422.47 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 118542.25 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163701.289068 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066880.22 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 106688.02 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270389.311257 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960192.20 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 96019.22 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366408.531227 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864172.98 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 86417.30 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452825.829200 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777755.68 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 77775.57 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530601.397375 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699980.11 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 69998.01 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600599.408733 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629982.10 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 62998.21 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663597.618955 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566983.89 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 56698.39 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720296.008155 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510285.50 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 51028.55 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771324.558435 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459256.95 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 45925.70 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817250.253687 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413331.26 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 41333.13 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858583.379414 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371998.13 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 37199.81 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895783.192568 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334798.32 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 33479.83 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929263.024407 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301318.49 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 30131.85 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959394.873062 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271186.64 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 27118.66 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986513.536851 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244067.97 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 24406.80 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010920.334262 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219661.18 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 21966.12 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032886.451931 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197695.06 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 19769.51 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052655.957834 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177925.55 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 17792.56 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070448.513146 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160133.00 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 16013.30 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086461.812927 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144119.70 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 14411.97 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100873.782730 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129707.73 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 12970.77 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113844.555552 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116736.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 11673.70 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125518.251093 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105063.26 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 10506.33 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136024.577079 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94556.93 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 9455.69 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145480.270467 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85101.24 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 8510.12 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153990.394515 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76591.12 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 7659.11 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161649.506159 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68932.00 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 6893.20 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168542.706639 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62038.80 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 6203.88 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174746.587071 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55834.92 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 5583.49 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180330.079459 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50251.43 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 5025.14 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185355.222609 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45226.29 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 4522.63 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189877.851443 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40703.66 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 4070.37 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193948.217395 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36633.29 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3663.33 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197611.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32969.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201211.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29369.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204811.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25769.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208411.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22169.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212011.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18569.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215611.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14969.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219211.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11369.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222811.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7769.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226411.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4169.96 sec) 13:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2230011.546751 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569.96 sec) 13:41:00 [GPUGRID] [rr_sim] 2230581.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980962654.21G/439.78G) 13:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852882.42 13:41:00 [---] pick_jobs_to_run() start 13:41:00 [---] pick_jobs_to_run() end 13:41:00 [---] [rr_sim] end 13:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:41:00 [---] [cpu_sched_debug] preliminary job list: 13:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:41:00 [---] [cpu_sched_debug] final job list: 13:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:41:00 [---] [cpu_sched_debug] enforce_run_list: end 13:41:00 [---] choose_project(): 49260.000000 13:41:00 [WUProp@Home] doing RPC to get work 13:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386339.09 cpu 2133272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133272.90 13:41:00 [---] Got 0 tasks 13:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:42:00 [---] [rr_sim] doing sim: CPU sched 13:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980935063.86G/439.78G) 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230518.77 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 223051.88 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223051.877407 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007466.90 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 200746.69 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423798.567074 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806720.21 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 180672.02 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604470.587774 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626048.19 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 162604.82 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767075.406404 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463443.37 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 146344.34 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913419.743171 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317099.03 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 131709.90 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045129.646261 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185389.13 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 118538.91 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163668.559042 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066850.22 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 106685.02 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270353.580545 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960165.19 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 96016.52 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366370.099898 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864148.67 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 86414.87 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452784.967316 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777733.81 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 77773.38 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530558.347991 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699960.43 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 69996.04 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600554.390600 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629964.38 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 62996.44 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663550.828947 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566967.95 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 56696.79 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720247.623460 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510271.15 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 51027.12 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771274.738521 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459244.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 45924.40 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817199.142076 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413319.63 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 41331.96 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858531.105276 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371987.67 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 37198.77 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895729.872156 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334788.90 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 33478.89 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929208.762347 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301310.01 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 30131.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959339.763520 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271179.01 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 27117.90 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986457.664575 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244061.11 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 24406.11 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010863.775525 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219655.00 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 21965.50 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032829.275380 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197689.50 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 19768.95 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052598.225249 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177920.55 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 17792.05 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070390.280132 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160128.49 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 16012.85 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086403.129526 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144115.64 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 14411.56 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100814.693981 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129704.08 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 12970.41 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113785.101990 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116733.67 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 11673.37 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125458.469198 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105060.30 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 10506.03 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135964.499686 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94554.27 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 9455.43 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145419.927125 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85098.85 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 8509.88 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153929.811819 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76588.96 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 7658.90 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161588.708045 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68930.07 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 6893.01 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168481.714648 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62037.06 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 6203.71 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174685.420590 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55833.35 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 5583.34 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180268.755939 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50250.02 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 5025.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185293.757752 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45225.02 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 4522.50 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189816.259384 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40702.51 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 4070.25 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193886.510853 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36632.26 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3663.23 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197549.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32969.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201149.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29369.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204749.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25769.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208349.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22169.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211949.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18569.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215549.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14969.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219149.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11369.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222749.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7769.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226349.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4169.04 sec) 13:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229949.737175 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 569.04 sec) 13:42:00 [GPUGRID] [rr_sim] 2230518.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980935063.86G/439.78G) 13:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852879.69 13:42:00 [---] pick_jobs_to_run() start 13:42:00 [---] pick_jobs_to_run() end 13:42:00 [---] [rr_sim] end 13:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:42:00 [---] [cpu_sched_debug] preliminary job list: 13:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:42:00 [---] [cpu_sched_debug] final job list: 13:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:42:00 [---] [cpu_sched_debug] enforce_run_list: end 13:42:00 [---] choose_project(): 49320.000000 13:42:00 [WUProp@Home] doing RPC to get work 13:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386279.09 cpu 2133212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133212.90 13:42:00 [---] Got 0 tasks 13:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:43:00 [---] [rr_sim] doing sim: CPU sched 13:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980907473.50G/439.78G) 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230456.04 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 223045.60 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223045.603719 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007410.43 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 200741.04 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423786.647066 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806669.39 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 180666.94 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604453.586079 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1626002.45 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 162600.25 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767053.831190 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463402.21 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 146340.22 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913394.051790 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317061.99 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 131706.20 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045100.250330 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185355.79 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 118535.58 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163635.829016 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066820.21 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 106682.02 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270317.849834 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960138.19 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 96013.82 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366331.668570 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864124.37 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 86412.44 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452744.105432 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777711.93 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 77771.19 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530515.298608 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699940.74 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 69994.07 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600509.372466 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629946.66 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 62994.67 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663504.038939 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566952.00 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 56695.20 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720199.238764 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510256.80 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 51025.68 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771224.918607 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459231.12 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 45923.11 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817148.030465 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413308.01 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 41330.80 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858478.831138 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371977.21 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 37197.72 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895676.551743 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334779.49 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 33477.95 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929154.500288 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301301.54 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 30130.15 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959284.653978 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271171.38 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 27117.14 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986401.792299 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244054.24 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 24405.42 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010807.216789 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219648.82 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 21964.88 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032772.098829 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197683.94 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 19768.39 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052540.492665 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177915.54 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 17791.55 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070332.047118 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160123.99 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 16012.40 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086344.446125 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144111.59 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 14411.16 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100755.605232 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129700.43 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 12970.04 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113725.648428 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116730.39 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 11673.04 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125398.687304 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105057.35 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 10505.73 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135904.422293 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94551.61 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 9455.16 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145359.583783 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85096.45 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 8509.65 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153869.229123 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76586.81 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 7658.68 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161527.909930 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68928.13 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 6892.81 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168420.722656 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62035.31 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 6203.53 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174624.254110 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55831.78 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 5583.18 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180207.432418 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50248.60 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 5024.86 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185232.292895 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45223.74 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 4522.37 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189754.667325 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40701.37 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 4070.14 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193824.804311 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36631.23 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3663.12 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197487.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32968.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201087.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29368.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204687.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25768.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208287.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22168.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211887.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18568.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215487.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14968.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219087.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11368.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222687.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7768.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226287.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4168.11 sec) 13:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229887.927599 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 568.11 sec) 13:43:00 [GPUGRID] [rr_sim] 2230456.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980907473.50G/439.78G) 13:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852876.95 13:43:00 [---] pick_jobs_to_run() start 13:43:00 [---] pick_jobs_to_run() end 13:43:00 [---] [rr_sim] end 13:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:43:00 [---] [cpu_sched_debug] preliminary job list: 13:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:43:00 [---] [cpu_sched_debug] final job list: 13:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:43:00 [---] [cpu_sched_debug] enforce_run_list: end 13:43:00 [---] choose_project(): 49380.000000 13:43:00 [WUProp@Home] doing RPC to get work 13:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386219.09 cpu 2133152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133152.90 13:43:00 [---] Got 0 tasks 13:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:44:00 [---] [rr_sim] doing sim: CPU sched 13:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980879883.15G/439.78G) 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230393.30 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 223039.33 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223039.330031 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007353.97 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 200735.40 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423774.727059 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806618.57 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 180661.86 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604436.584384 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625956.72 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 162595.67 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767032.255976 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463361.04 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 146336.10 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913368.360410 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1317024.94 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 131702.49 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045070.854399 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185322.45 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 118532.24 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163603.098990 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066790.20 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 106679.02 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270282.119122 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960111.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 96011.12 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366293.237241 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864100.06 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 86410.01 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452703.243548 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777690.06 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 77769.01 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530472.249224 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699921.05 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 69992.11 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600464.354332 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629928.95 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 62992.89 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663457.248930 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566936.05 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 56693.61 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720150.854068 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510242.45 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 51024.24 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771175.098692 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459218.20 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 45921.82 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817096.918854 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413296.38 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 41329.64 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858426.556999 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371966.74 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 37196.67 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895623.231330 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334770.07 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 33477.01 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929100.238228 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301293.06 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 30129.31 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959229.544436 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271163.76 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 27116.38 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986345.920024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244047.38 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 24404.74 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010750.658052 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219642.64 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 21964.26 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032714.922278 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197678.38 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 19767.84 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052482.760081 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177910.54 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 17791.05 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070273.814104 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160119.49 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 16011.95 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086285.762724 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144107.54 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 14410.75 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100696.516483 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129696.78 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 12969.68 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113666.194865 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116727.11 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 11672.71 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125338.905410 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105054.39 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 10505.44 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135844.344900 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94548.96 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 9454.90 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145299.240441 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85094.06 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 8509.41 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153808.646427 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76584.65 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 7658.47 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161467.111816 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68926.19 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 6892.62 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168359.730665 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62033.57 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 6203.36 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174563.087629 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55830.21 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 5583.02 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180146.108897 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50247.19 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 5024.72 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185170.828038 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45222.47 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 4522.25 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189693.075266 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40700.23 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 4070.02 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193763.097770 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36630.20 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3663.02 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197426.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32967.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201026.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29367.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204626.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25767.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208226.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22167.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211826.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18567.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215426.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14967.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219026.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11367.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222626.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7767.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226226.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4167.18 sec) 13:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229826.118024 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 567.18 sec) 13:44:00 [GPUGRID] [rr_sim] 2230393.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980879883.15G/439.78G) 13:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852874.21 13:44:00 [---] pick_jobs_to_run() start 13:44:00 [---] pick_jobs_to_run() end 13:44:00 [---] [rr_sim] end 13:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:44:00 [---] [cpu_sched_debug] preliminary job list: 13:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:44:00 [---] [cpu_sched_debug] final job list: 13:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:44:00 [---] [cpu_sched_debug] enforce_run_list: end 13:44:00 [---] choose_project(): 49440.000000 13:44:00 [WUProp@Home] doing RPC to get work 13:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386159.09 cpu 2133092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133092.90 13:44:00 [---] Got 0 tasks 13:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:45:00 [---] [rr_sim] doing sim: CPU sched 13:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980852292.79G/439.78G) 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230330.56 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 223033.06 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223033.056343 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007297.51 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 200729.75 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423762.807051 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806567.76 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 180656.78 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604419.582689 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625910.98 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 162591.10 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 767010.680762 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463319.88 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 146331.99 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913342.669029 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316987.89 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 131698.79 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045041.458469 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185289.10 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 118528.91 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163570.368965 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066760.19 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 106676.02 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270246.388411 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960084.18 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 96008.42 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366254.805912 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864075.76 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 86407.58 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452662.381664 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777668.18 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 77766.82 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530429.199840 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699901.36 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 69990.14 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600419.336199 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629911.23 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 62991.12 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663410.458922 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566920.10 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 56692.01 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720102.469372 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510228.09 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 51022.81 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771125.278778 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459205.28 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 45920.53 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1817045.807243 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413284.76 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 41328.48 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858374.282861 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371956.28 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 37195.63 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895569.910918 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334760.65 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 33476.07 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929045.976168 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301284.59 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 30128.46 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959174.434894 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271156.13 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 27115.61 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986290.047748 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244040.52 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 24404.05 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010694.099316 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219636.46 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 21963.65 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032657.745727 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197672.82 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 19767.28 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052425.027497 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177905.54 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 17790.55 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070215.581090 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160114.98 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 16011.50 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086227.079323 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144103.48 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 14410.35 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100637.427734 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129693.14 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 12969.31 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113606.741303 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116723.82 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 11672.38 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125279.123515 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105051.44 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 10505.14 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135784.267507 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94546.30 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 9454.63 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145238.897099 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85091.67 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 8509.17 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153748.063731 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76582.50 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 7658.25 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161406.313701 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68924.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 6892.42 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168298.738674 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62031.82 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 6203.18 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174501.921149 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55828.64 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 5582.86 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180084.785377 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50245.78 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 5024.58 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185109.363182 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45221.20 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 4522.12 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189631.483206 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40699.08 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 4069.91 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193701.391228 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36629.17 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3662.92 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197364.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32966.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200964.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29366.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204564.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25766.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208164.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22166.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211764.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18566.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215364.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14966.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218964.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11366.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222564.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7766.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226164.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4166.25 sec) 13:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229764.308448 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566.25 sec) 13:45:00 [GPUGRID] [rr_sim] 2230330.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980852292.79G/439.78G) 13:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852871.48 13:45:00 [---] pick_jobs_to_run() start 13:45:00 [---] pick_jobs_to_run() end 13:45:00 [---] [rr_sim] end 13:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:45:00 [---] [cpu_sched_debug] preliminary job list: 13:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:45:00 [---] [cpu_sched_debug] final job list: 13:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:45:00 [---] [cpu_sched_debug] enforce_run_list: end 13:45:00 [---] choose_project(): 49500.000000 13:45:00 [WUProp@Home] doing RPC to get work 13:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386099.09 cpu 2133032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2133032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2133032.90 13:45:00 [---] Got 0 tasks 13:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:46:00 [---] [rr_sim] doing sim: CPU sched 13:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980824702.43G/439.78G) 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230267.83 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 223026.78 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223026.782654 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007241.04 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 200724.10 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423750.887043 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806516.94 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 180651.69 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604402.580994 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625865.25 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 162586.52 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766989.105549 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463278.72 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 146327.87 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913316.977648 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316950.85 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 131695.08 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1045012.062538 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185255.76 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 118525.58 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163537.638939 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066730.19 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 106673.02 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270210.657699 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960057.17 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 96005.72 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366216.374584 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864051.45 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 86405.15 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452621.519780 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777646.31 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 77764.63 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530386.150456 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699881.68 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 69988.17 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600374.318065 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629893.51 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 62989.35 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663363.668913 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566904.16 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 56690.42 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720054.084676 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510213.74 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 51021.37 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771075.458863 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459192.37 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 45919.24 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816994.695631 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413273.13 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 41327.31 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858322.008723 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371945.82 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 37194.58 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895516.590505 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334751.24 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 33475.12 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928991.714109 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301276.11 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 30127.61 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959119.325352 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271148.50 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 27114.85 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986234.175472 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244033.65 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 24403.37 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010637.540579 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219630.29 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 21963.03 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032600.569176 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197667.26 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 19766.73 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052367.294912 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177900.53 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 17790.05 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070157.348076 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160110.48 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 16011.05 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086168.395923 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144099.43 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 14409.94 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100578.338985 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129689.49 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 12968.95 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113547.287741 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116720.54 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 11672.05 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125219.341621 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105048.48 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 10504.85 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135724.190114 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94543.64 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 9454.36 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145178.553757 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85089.27 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 8508.93 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153687.481035 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76580.35 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 7658.03 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161345.515586 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68922.31 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 6892.23 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168237.746682 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62030.08 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 6203.01 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174440.754668 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55827.07 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 5582.71 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180023.461856 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50244.36 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 5024.44 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185047.898325 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45219.93 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 4521.99 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189569.891147 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40697.94 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 4069.79 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193639.684687 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36628.14 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3662.81 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197302.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32965.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200902.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29365.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204502.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25765.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208102.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22165.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211702.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18565.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215302.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14965.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218902.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11365.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222502.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7765.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226102.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4165.33 sec) 13:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229702.498872 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565.33 sec) 13:46:00 [GPUGRID] [rr_sim] 2230267.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980824702.43G/439.78G) 13:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852868.74 13:46:00 [---] pick_jobs_to_run() start 13:46:00 [---] pick_jobs_to_run() end 13:46:00 [---] [rr_sim] end 13:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:46:00 [---] [cpu_sched_debug] preliminary job list: 13:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:46:00 [---] [cpu_sched_debug] final job list: 13:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:46:00 [---] [cpu_sched_debug] enforce_run_list: end 13:46:00 [---] choose_project(): 49560.000000 13:46:00 [WUProp@Home] doing RPC to get work 13:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 386039.09 cpu 2132972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132972.90 13:46:00 [---] Got 0 tasks 13:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:47:00 [---] [rr_sim] doing sim: CPU sched 13:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980797112.08G/439.78G) 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230205.09 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 223020.51 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223020.508966 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007184.58 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 200718.46 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423738.967036 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806466.12 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 180646.61 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604385.579299 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625819.51 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 162581.95 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766967.530335 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463237.56 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 146323.76 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913291.286268 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316913.80 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 131691.38 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044982.666607 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185222.42 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 118522.24 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163504.908913 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066700.18 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 106670.02 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270174.926988 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960030.16 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 96003.02 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366177.943255 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864027.15 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 86402.71 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452580.657896 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777624.43 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 77762.44 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530343.101073 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699861.99 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 69986.20 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600329.299932 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629875.79 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 62987.58 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663316.878905 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566888.21 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 56688.82 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1720005.699980 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510199.39 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 51019.94 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1771025.638949 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459179.45 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 45917.95 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816943.584020 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413261.51 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 41326.15 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858269.734584 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371935.36 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 37193.54 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895463.270092 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334741.82 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 33474.18 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928937.452049 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301267.64 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 30126.76 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959064.215811 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271140.87 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 27114.09 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986178.303196 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244026.79 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 24402.68 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010580.981842 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219624.11 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 21962.41 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032543.392624 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197661.70 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 19766.17 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052309.562328 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177895.53 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 17789.55 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070099.115062 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160105.97 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 16010.60 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086109.712522 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144095.38 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 14409.54 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100519.250236 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129685.84 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 12968.58 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113487.834178 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116717.26 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 11671.73 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125159.559727 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105045.53 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 10504.55 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135664.112720 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94540.98 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 9454.10 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145118.210415 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85086.88 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 8508.69 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153626.898339 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76578.19 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 7657.82 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161284.717472 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68920.37 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 6892.04 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168176.754691 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62028.33 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 6202.83 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174379.588188 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55825.50 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 5582.55 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179962.138335 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50242.95 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 5024.30 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184986.433468 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45218.66 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 4521.87 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189508.299088 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40696.79 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 4069.68 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193577.978145 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36627.11 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3662.71 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197240.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32964.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200840.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29364.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204440.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25764.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208040.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22164.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211640.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18564.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215240.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14964.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218840.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11364.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222440.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7764.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226040.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4164.40 sec) 13:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229640.689297 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564.40 sec) 13:47:00 [GPUGRID] [rr_sim] 2230205.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980797112.08G/439.78G) 13:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852866.00 13:47:00 [---] pick_jobs_to_run() start 13:47:00 [---] pick_jobs_to_run() end 13:47:00 [---] [rr_sim] end 13:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:47:00 [---] [cpu_sched_debug] preliminary job list: 13:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:47:00 [---] [cpu_sched_debug] final job list: 13:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:47:00 [---] [cpu_sched_debug] enforce_run_list: end 13:47:00 [---] choose_project(): 49620.000000 13:47:00 [WUProp@Home] doing RPC to get work 13:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385979.09 cpu 2132912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132912.90 13:47:00 [---] Got 0 tasks 13:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:48:00 [---] [rr_sim] doing sim: CPU sched 13:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980769521.72G/439.78G) 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230142.35 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 223014.24 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223014.235278 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007128.12 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 200712.81 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423727.047028 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806415.31 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 180641.53 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604368.577603 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625773.78 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 162577.38 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766945.955121 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463196.40 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 146319.64 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913265.594887 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316876.76 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 131687.68 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044953.270676 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185189.08 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 118518.91 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163472.178887 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066670.17 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 106667.02 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270139.196276 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 960003.16 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 96000.32 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366139.511927 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 864002.84 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 86400.28 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452539.796012 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777602.56 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 77760.26 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530300.051689 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699842.30 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 69984.23 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600284.281798 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629858.07 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 62985.81 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663270.088896 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566872.26 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 56687.23 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719957.315285 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510185.04 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 51018.50 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770975.819034 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459166.53 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 45916.65 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816892.472409 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413249.88 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 41324.99 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858217.460446 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371924.89 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 37192.49 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895409.949679 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334732.40 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 33473.24 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928883.189990 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301259.16 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 30125.92 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959009.106269 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271133.25 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 27113.32 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986122.430920 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244019.92 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 24401.99 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010524.423106 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219617.93 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 21961.79 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032486.216073 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197656.14 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 19765.61 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052251.829744 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177890.52 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 17789.05 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070040.882048 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160101.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 16010.15 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086051.029121 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144091.32 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 14409.13 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100460.161487 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129682.19 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 12968.22 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113428.380616 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116713.97 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 11671.40 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125099.777833 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105042.57 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 10504.26 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135604.035327 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94538.32 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 9453.83 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145057.867073 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85084.49 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 8508.45 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153566.315643 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76576.04 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 7657.60 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161223.919357 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68918.43 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 6891.84 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168115.762699 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62026.59 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 6202.66 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174318.421708 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55823.93 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 5582.39 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179900.814815 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50241.54 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 5024.15 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184924.968611 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45217.38 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 4521.74 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189446.707028 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40695.65 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 4069.56 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193516.271604 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36626.08 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3662.61 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197178.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32963.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200778.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29363.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204378.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25763.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207978.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22163.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211578.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18563.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215178.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14963.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218778.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11363.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222378.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7763.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225978.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4163.47 sec) 13:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229578.879721 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563.47 sec) 13:48:00 [GPUGRID] [rr_sim] 2230142.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980769521.72G/439.78G) 13:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852863.27 13:48:00 [---] pick_jobs_to_run() start 13:48:00 [---] pick_jobs_to_run() end 13:48:00 [---] [rr_sim] end 13:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:48:00 [---] [cpu_sched_debug] preliminary job list: 13:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:48:00 [---] [cpu_sched_debug] final job list: 13:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:48:00 [---] [cpu_sched_debug] enforce_run_list: end 13:48:00 [---] choose_project(): 49680.000000 13:48:00 [WUProp@Home] doing RPC to get work 13:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385919.09 cpu 2132852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132852.90 13:48:00 [---] Got 0 tasks 13:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:49:00 [---] [rr_sim] doing sim: CPU sched 13:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980741931.36G/439.78G) 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230079.62 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 223007.96 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223007.961590 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007071.65 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 200707.17 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423715.127021 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806364.49 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 180636.45 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604351.575908 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625728.04 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 162572.80 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766924.379907 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463155.24 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 146315.52 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913239.903506 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316839.71 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 131683.97 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044923.874746 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185155.74 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 118515.57 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163439.448861 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066640.17 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 106664.02 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270103.465565 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959976.15 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 95997.62 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366101.080598 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863978.54 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 86397.85 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452498.934128 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777580.68 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 77758.07 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530257.002305 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699822.61 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 69982.26 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600239.263664 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629840.35 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 62984.04 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663223.298888 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566856.32 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 56685.63 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719908.930589 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510170.69 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 51017.07 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770925.999120 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459153.62 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 45915.36 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816841.360798 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413238.26 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 41323.83 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858165.186308 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371914.43 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 37191.44 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895356.629267 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334722.99 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 33472.30 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928828.927930 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301250.69 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 30125.07 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958953.996727 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271125.62 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 27112.56 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986066.558644 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244013.06 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 24401.31 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010467.864369 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219611.75 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 21961.18 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032429.039522 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197650.58 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 19765.06 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052194.097160 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177885.52 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 17788.55 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069982.649034 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160096.97 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 16009.70 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085992.345720 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144087.27 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 14408.73 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100401.072738 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129678.54 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 12967.85 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113368.927054 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116710.69 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 11671.07 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125039.995938 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105039.62 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 10503.96 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135543.957934 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94535.66 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 9453.57 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144997.523731 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85082.09 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 8508.21 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153505.732947 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76573.88 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 7657.39 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161163.121243 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68916.49 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 6891.65 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168054.770708 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62024.85 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 6202.48 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174257.255227 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55822.36 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 5582.24 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179839.491294 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50240.12 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 5024.01 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184863.503755 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45216.11 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 4521.61 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189385.114969 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40694.50 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 4069.45 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193454.565062 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36625.05 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3662.51 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197117.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32962.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200717.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29362.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204317.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25762.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207917.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22162.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211517.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18562.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215117.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14962.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218717.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11362.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222317.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7762.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225917.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4162.55 sec) 13:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229517.070146 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562.55 sec) 13:49:00 [GPUGRID] [rr_sim] 2230079.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980741931.36G/439.78G) 13:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852860.53 13:49:00 [---] pick_jobs_to_run() start 13:49:00 [---] pick_jobs_to_run() end 13:49:00 [---] [rr_sim] end 13:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:49:00 [---] [cpu_sched_debug] preliminary job list: 13:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:49:00 [---] [cpu_sched_debug] final job list: 13:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:49:00 [---] [cpu_sched_debug] enforce_run_list: end 13:49:00 [---] choose_project(): 49740.000000 13:49:00 [WUProp@Home] doing RPC to get work 13:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385859.09 cpu 2132792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132792.90 13:49:00 [---] Got 0 tasks 13:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:50:00 [---] [rr_sim] doing sim: CPU sched 13:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980714341.01G/439.78G) 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2230016.88 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 223001.69 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 223001.687902 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2007015.19 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 200701.52 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423703.207013 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806313.67 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 180631.37 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604334.574213 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625682.30 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 162568.23 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766902.804694 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463114.07 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 146311.41 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913214.212126 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316802.67 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 131680.27 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044894.478815 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185122.40 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 118512.24 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163406.718835 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066610.16 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 106661.02 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270067.734853 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959949.14 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 95994.91 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366062.649269 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863954.23 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 86395.42 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452458.072244 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777558.81 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 77755.88 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530213.952921 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699802.93 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 69980.29 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600194.245531 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629822.63 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 62982.26 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663176.508879 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566840.37 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 56684.04 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719860.545893 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510156.33 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 51015.63 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770876.179205 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459140.70 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 45914.07 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816790.249186 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413226.63 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 41322.66 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858112.912169 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371903.97 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 37190.40 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895303.308854 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334713.57 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 33471.36 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928774.665870 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301242.21 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 30124.22 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958898.887185 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271117.99 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 27111.80 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986010.686368 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244006.19 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 24400.62 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010411.305633 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219605.57 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 21960.56 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032371.862971 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197645.02 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 19764.50 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052136.364576 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177880.51 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 17788.05 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069924.416020 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160092.46 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 16009.25 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085933.662319 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144083.22 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 14408.32 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100341.983989 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129674.90 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 12967.49 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113309.473492 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116707.41 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 11670.74 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124980.214044 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105036.66 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 10503.67 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135483.880541 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94533.00 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 9453.30 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144937.180389 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85079.70 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 8507.97 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153445.150251 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76571.73 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 7657.17 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161102.323128 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68914.56 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 6891.46 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167993.778717 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62023.10 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 6202.31 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174196.088747 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55820.79 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 5582.08 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179778.167774 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50238.71 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 5023.87 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184802.038898 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45214.84 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 4521.48 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189323.522910 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40693.36 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 4069.34 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193392.858520 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36624.02 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3662.40 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197055.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32961.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200655.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29361.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204255.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25761.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207855.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22161.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211455.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18561.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215055.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14961.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218655.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11361.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222255.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7761.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225855.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4161.62 sec) 13:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229455.260570 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561.62 sec) 13:50:00 [GPUGRID] [rr_sim] 2230016.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980714341.01G/439.78G) 13:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852857.79 13:50:00 [---] pick_jobs_to_run() start 13:50:00 [---] pick_jobs_to_run() end 13:50:00 [---] [rr_sim] end 13:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:50:00 [---] [cpu_sched_debug] preliminary job list: 13:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:50:00 [---] [cpu_sched_debug] final job list: 13:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:50:00 [---] [cpu_sched_debug] enforce_run_list: end 13:50:00 [---] choose_project(): 49800.000000 13:50:00 [WUProp@Home] doing RPC to get work 13:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385799.09 cpu 2132732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132732.90 13:50:00 [---] Got 0 tasks 13:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:51:00 [---] [rr_sim] doing sim: CPU sched 13:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980686750.65G/439.78G) 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229954.14 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 222995.41 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222995.414213 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006958.73 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 200695.87 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423691.287005 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806262.86 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 180626.29 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604317.572518 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625636.57 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 162563.66 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766881.229480 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463072.91 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 146307.29 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913188.520745 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316765.62 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 131676.56 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044865.082884 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185089.06 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 118508.91 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163373.988809 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066580.15 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 106658.02 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1270032.004142 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959922.14 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 95992.21 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1366024.217941 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863929.92 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 86392.99 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452417.210360 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777536.93 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 77753.69 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530170.903537 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699783.24 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 69978.32 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600149.227397 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629804.91 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 62980.49 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663129.718871 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566824.42 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 56682.44 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719812.161197 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510141.98 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 51014.20 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770826.359291 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459127.78 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 45912.78 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816739.137575 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413215.00 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 41321.50 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858060.638031 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371893.50 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 37189.35 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895249.988441 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334704.15 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 33470.42 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928720.403811 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301233.74 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 30123.37 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958843.777643 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271110.36 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 27111.04 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985954.814092 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243999.33 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 24399.93 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010354.746896 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219599.40 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 21959.94 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032314.686420 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197639.46 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 19763.95 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052078.631991 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177875.51 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 17787.55 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069866.183006 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160087.96 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 16008.80 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085874.978918 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144079.16 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 14407.92 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100282.895240 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129671.25 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 12967.12 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113250.019929 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116704.12 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 11670.41 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124920.432150 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105033.71 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 10503.37 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135423.803148 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94530.34 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 9453.03 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144876.837047 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85077.31 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 8507.73 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153384.567555 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76569.57 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 7656.96 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161041.525013 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68912.62 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 6891.26 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167932.786725 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62021.36 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 6202.14 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174134.922266 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55819.22 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 5581.92 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179716.844253 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50237.30 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 5023.73 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184740.574041 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45213.57 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 4521.36 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189261.930850 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40692.21 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 4069.22 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193331.151979 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36622.99 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3662.30 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196993.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32960.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200593.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29360.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204193.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25760.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207793.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22160.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211393.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18560.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214993.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14960.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218593.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11360.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222193.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7760.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225793.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4160.69 sec) 13:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229393.450994 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560.69 sec) 13:51:00 [GPUGRID] [rr_sim] 2229954.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980686750.65G/439.78G) 13:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852855.05 13:51:00 [---] pick_jobs_to_run() start 13:51:00 [---] pick_jobs_to_run() end 13:51:00 [---] [rr_sim] end 13:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:51:00 [---] [cpu_sched_debug] preliminary job list: 13:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:51:00 [---] [cpu_sched_debug] final job list: 13:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:51:00 [---] [cpu_sched_debug] enforce_run_list: end 13:51:00 [---] choose_project(): 49860.000000 13:51:00 [WUProp@Home] doing RPC to get work 13:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385739.09 cpu 2132672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132672.90 13:51:00 [---] Got 0 tasks 13:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:52:00 [---] [rr_sim] doing sim: CPU sched 13:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980659160.30G/439.78G) 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229891.41 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 222989.14 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222989.140525 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006902.26 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 200690.23 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423679.366998 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806212.04 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 180621.20 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604300.570823 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625590.83 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 162559.08 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766859.654266 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1463031.75 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 146303.18 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913162.829365 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316728.58 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 131672.86 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044835.686953 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185055.72 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 118505.57 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163341.258783 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066550.15 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 106655.01 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269996.273430 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959895.13 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 95989.51 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365985.786612 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863905.62 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 86390.56 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452376.348476 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777515.06 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 77751.51 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530127.854154 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699763.55 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 69976.36 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600104.209264 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629787.20 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 62978.72 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663082.928862 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566808.48 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 56680.85 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719763.776501 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510127.63 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 51012.76 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770776.539376 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459114.87 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 45911.49 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816688.025964 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413203.38 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 41320.34 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1858008.363893 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371883.04 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 37188.30 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895196.668029 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334694.74 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 33469.47 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928666.141751 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301225.26 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 30122.53 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958788.668101 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271102.74 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 27110.27 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985898.941816 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243992.46 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 24399.25 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010298.188160 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219593.22 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 21959.32 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032257.509869 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197633.90 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 19763.39 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052020.899407 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177870.51 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 17787.05 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069807.949992 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160083.46 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 16008.35 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085816.295518 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144075.11 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 14407.51 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100223.806491 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129667.60 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 12966.76 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113190.566367 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116700.84 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 11670.08 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124860.650256 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105030.75 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 10503.08 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135363.725755 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94527.68 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 9452.77 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144816.493705 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85074.91 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 8507.49 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153323.984859 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76567.42 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 7656.74 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160980.726899 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68910.68 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 6891.07 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167871.794734 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62019.61 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 6201.96 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174073.755786 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55817.65 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 5581.76 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179655.520732 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50235.88 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 5023.59 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184679.109184 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45212.30 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 4521.23 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189200.338791 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40691.07 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 4069.11 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193269.445437 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36621.96 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3662.20 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196931.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32959.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200531.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29359.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204131.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25759.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207731.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22159.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211331.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18559.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214931.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14959.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218531.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11359.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222131.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7759.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225731.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4159.76 sec) 13:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229331.641419 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559.76 sec) 13:52:00 [GPUGRID] [rr_sim] 2229891.41: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980659160.30G/439.78G) 13:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852852.32 13:52:00 [---] pick_jobs_to_run() start 13:52:00 [---] pick_jobs_to_run() end 13:52:00 [---] [rr_sim] end 13:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:52:00 [---] [cpu_sched_debug] preliminary job list: 13:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:52:00 [---] [cpu_sched_debug] final job list: 13:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:52:00 [---] [cpu_sched_debug] enforce_run_list: end 13:52:00 [---] choose_project(): 49920.000000 13:52:00 [WUProp@Home] doing RPC to get work 13:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385679.09 cpu 2132612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132612.90 13:52:00 [---] Got 0 tasks 13:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:53:00 [---] [rr_sim] doing sim: CPU sched 13:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980631569.94G/439.78G) 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229828.67 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 222982.87 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222982.866837 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006845.80 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 200684.58 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423667.446990 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806161.22 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 180616.12 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604283.569128 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625545.10 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 162554.51 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766838.079052 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462990.59 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 146299.06 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913137.137984 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316691.53 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 131669.15 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044806.291023 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1185022.38 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 118502.24 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163308.528757 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066520.14 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 106652.01 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269960.542718 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959868.13 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 95986.81 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365947.355284 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863881.31 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 86388.13 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452335.486592 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777493.18 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 77749.32 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530084.804770 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699743.86 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 69974.39 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600059.191130 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629769.48 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 62976.95 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1663036.138854 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566792.53 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 56679.25 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719715.391805 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510113.28 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 51011.33 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770726.719462 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459101.95 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 45910.19 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816636.914353 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413191.75 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 41319.18 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857956.089754 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371872.58 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 37187.26 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895143.347616 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334685.32 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 33468.53 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928611.879691 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301216.79 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 30121.68 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958733.558559 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271095.11 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 27109.51 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985843.069540 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243985.60 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 24398.56 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010241.629423 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219587.04 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 21958.70 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032200.333318 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197628.34 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 19762.83 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051963.166823 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177865.50 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 17786.55 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069749.716978 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160078.95 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 16007.90 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085757.612117 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144071.06 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 14407.11 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100164.717742 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129663.95 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 12966.40 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113131.112805 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116697.56 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 11669.76 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124800.868361 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105027.80 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 10502.78 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135303.648362 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94525.02 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 9452.50 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144756.150363 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85072.52 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 8507.25 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153263.402163 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76565.27 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 7656.53 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160919.928784 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68908.74 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 6890.87 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167810.802743 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62017.87 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 6201.79 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174012.589305 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55816.08 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 5581.61 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179594.197212 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50234.47 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 5023.45 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184617.644328 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45211.02 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 4521.10 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189138.746732 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40689.92 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 4068.99 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193207.738896 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36620.93 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3662.09 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196869.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32958.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200469.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29358.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204069.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25758.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207669.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22158.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211269.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18558.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214869.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14958.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218469.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11358.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222069.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7758.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225669.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4158.84 sec) 13:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229269.831843 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558.84 sec) 13:53:00 [GPUGRID] [rr_sim] 2229828.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980631569.94G/439.78G) 13:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852849.58 13:53:00 [---] pick_jobs_to_run() start 13:53:00 [---] pick_jobs_to_run() end 13:53:00 [---] [rr_sim] end 13:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:53:00 [---] [cpu_sched_debug] preliminary job list: 13:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:53:00 [---] [cpu_sched_debug] final job list: 13:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:53:00 [---] [cpu_sched_debug] enforce_run_list: end 13:53:00 [---] choose_project(): 49980.000000 13:53:00 [WUProp@Home] doing RPC to get work 13:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385619.09 cpu 2132552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132552.90 13:53:00 [---] Got 0 tasks 13:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:54:00 [---] [rr_sim] doing sim: CPU sched 13:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980603979.58G/439.78G) 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229765.93 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 222976.59 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222976.593149 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006789.34 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 200678.93 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423655.526983 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806110.40 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 180611.04 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604266.567433 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625499.36 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 162549.94 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766816.503839 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462949.43 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 146294.94 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913111.446603 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316654.48 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 131665.45 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044776.895092 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184989.04 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 118498.90 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163275.798731 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066490.13 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 106649.01 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269924.812007 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959841.12 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 95984.11 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365908.923955 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863857.01 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 86385.70 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452294.624708 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777471.31 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 77747.13 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1530041.755386 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699724.18 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 69972.42 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1600014.172996 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629751.76 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 62975.18 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662989.348845 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566776.58 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 56677.66 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719667.007110 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510098.92 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 51009.89 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770676.899547 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459089.03 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 45908.90 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816585.802741 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413180.13 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 41318.01 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857903.815616 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371862.12 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 37186.21 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895090.027203 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334675.90 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 33467.59 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928557.617632 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301208.31 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 30120.83 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958678.449017 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271087.48 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 27108.75 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985787.197264 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243978.73 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 24397.87 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010185.070686 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219580.86 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 21958.09 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032143.156767 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197622.77 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 19762.28 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051905.434239 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177860.50 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 17786.05 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069691.483964 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160074.45 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 16007.44 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085698.928716 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144067.00 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 14406.70 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100105.628993 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129660.30 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 12966.03 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113071.659242 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116694.27 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 11669.43 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124741.086467 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105024.85 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 10502.48 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135243.570969 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94522.36 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 9452.24 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144695.807021 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85070.12 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 8507.01 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153202.819468 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76563.11 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 7656.31 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160859.130669 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68906.80 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 6890.68 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167749.810751 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62016.12 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 6201.61 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173951.422825 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55814.51 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 5581.45 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179532.873691 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50233.06 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 5023.31 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184556.179471 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45209.75 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 4520.98 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189077.154672 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40688.78 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 4068.88 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193146.032354 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36619.90 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3661.99 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196808.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32957.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200408.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29357.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204008.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25757.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207608.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22157.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211208.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18557.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214808.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14957.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218408.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11357.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222008.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7757.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225608.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4157.91 sec) 13:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229208.022267 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557.91 sec) 13:54:00 [GPUGRID] [rr_sim] 2229765.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980603979.58G/439.78G) 13:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852846.84 13:54:00 [---] pick_jobs_to_run() start 13:54:00 [---] pick_jobs_to_run() end 13:54:00 [---] [rr_sim] end 13:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:54:00 [---] [cpu_sched_debug] preliminary job list: 13:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:54:00 [---] [cpu_sched_debug] final job list: 13:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:54:00 [---] [cpu_sched_debug] enforce_run_list: end 13:54:00 [---] choose_project(): 50040.000000 13:54:00 [WUProp@Home] doing RPC to get work 13:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385559.09 cpu 2132492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132492.90 13:54:00 [---] Got 0 tasks 13:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:55:00 [---] [rr_sim] doing sim: CPU sched 13:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980576389.23G/439.78G) 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229703.19 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 222970.32 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222970.319461 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006732.88 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 200673.29 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423643.606975 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806059.59 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 180605.96 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604249.565738 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625453.63 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 162545.36 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766794.928625 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462908.27 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 146290.83 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913085.755223 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316617.44 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 131661.74 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044747.499161 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184955.70 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 118495.57 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163243.068705 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066460.13 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 106646.01 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269889.081295 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959814.11 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 95981.41 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365870.492626 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863832.70 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 86383.27 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452253.762824 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777449.43 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 77744.94 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529998.706002 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699704.49 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 69970.45 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599969.154863 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629734.04 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 62973.40 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662942.558837 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566760.64 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 56676.06 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719618.622414 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510084.57 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 51008.46 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770627.079633 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459076.11 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 45907.61 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816534.691130 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413168.50 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 41316.85 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857851.541478 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371851.65 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 37185.17 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895036.706790 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334666.49 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 33466.65 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928503.355572 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301199.84 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 30119.98 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958623.339475 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271079.86 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 27107.99 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985731.324988 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243971.87 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 24397.19 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010128.511950 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219574.68 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 21957.47 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032085.980215 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197617.21 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 19761.72 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051847.701654 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177855.49 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 17785.55 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069633.250949 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160069.94 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 16006.99 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085640.245315 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144062.95 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 14406.29 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100046.540244 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129656.65 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 12965.67 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113012.205680 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116690.99 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 11669.10 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124681.304573 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105021.89 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 10502.19 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135183.493576 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94519.70 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 9451.97 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144635.463679 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85067.73 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 8506.77 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153142.236772 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76560.96 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 7656.10 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160798.332555 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68904.86 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 6890.49 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167688.818760 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62014.38 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 6201.44 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173890.256344 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55812.94 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 5581.29 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179471.550171 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50231.64 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 5023.16 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184494.714614 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45208.48 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 4520.85 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189015.562613 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40687.63 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 4068.76 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193084.325812 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36618.87 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3661.89 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196746.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32956.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200346.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29356.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203946.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25756.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207546.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22156.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211146.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18556.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214746.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14956.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218346.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11356.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221946.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7756.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225546.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4156.98 sec) 13:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229146.212692 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 556.98 sec) 13:55:00 [GPUGRID] [rr_sim] 2229703.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980576389.23G/439.78G) 13:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852844.11 13:55:00 [---] pick_jobs_to_run() start 13:55:00 [---] pick_jobs_to_run() end 13:55:00 [---] [rr_sim] end 13:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:55:00 [---] [cpu_sched_debug] preliminary job list: 13:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:55:00 [---] [cpu_sched_debug] final job list: 13:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:55:00 [---] [cpu_sched_debug] enforce_run_list: end 13:55:00 [---] choose_project(): 50100.000000 13:55:00 [WUProp@Home] doing RPC to get work 13:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385499.09 cpu 2132432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132432.90 13:55:00 [---] Got 0 tasks 13:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:56:00 [---] [rr_sim] doing sim: CPU sched 13:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980548798.87G/439.78G) 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229640.46 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 222964.05 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222964.045772 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006676.41 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 200667.64 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423631.686967 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1806008.77 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 180600.88 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604232.564043 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625407.89 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 162540.79 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766773.353411 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462867.10 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 146286.71 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913060.063842 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316580.39 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 131658.04 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044718.103230 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184922.35 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 118492.24 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163210.338680 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066430.12 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 106643.01 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269853.350584 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959787.11 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 95978.71 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365832.061298 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863808.40 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 86380.84 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452212.900940 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777427.56 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 77742.76 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529955.656619 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699684.80 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 69968.48 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599924.136729 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629716.32 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 62971.63 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662895.768828 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566744.69 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 56674.47 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719570.237718 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510070.22 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 51007.02 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770577.259718 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459063.20 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 45906.32 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816483.579519 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413156.88 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 41315.69 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857799.267339 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371841.19 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 37184.12 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894983.386378 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334657.07 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 33465.71 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928449.093512 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301191.36 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 30119.14 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958568.229933 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271072.23 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 27107.22 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985675.452712 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243965.01 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 24396.50 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010071.953213 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219568.50 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 21956.85 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032028.803664 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197611.65 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 19761.17 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051789.969070 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177850.49 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 17785.05 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069575.017935 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160065.44 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 16006.54 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085581.561914 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144058.90 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 14405.89 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099987.451495 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129653.01 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 12965.30 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112952.752118 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116687.71 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 11668.77 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124621.522678 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105018.94 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 10501.89 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135123.416183 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94517.04 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 9451.70 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144575.120337 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85065.34 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 8506.53 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153081.654076 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76558.80 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 7655.88 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160737.534440 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68902.92 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 6890.29 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167627.826769 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62012.63 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 6201.26 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173829.089864 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55811.37 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 5581.14 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179410.226650 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50230.23 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 5023.02 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184433.249757 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45207.21 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 4520.72 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188953.970554 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40686.49 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 4068.65 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193022.619271 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36617.84 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3661.78 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196684.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32956.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200284.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29356.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203884.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25756.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207484.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22156.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211084.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18556.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214684.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14956.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218284.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11356.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221884.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7756.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225484.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4156.05 sec) 13:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229084.403116 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 556.05 sec) 13:56:00 [GPUGRID] [rr_sim] 2229640.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980548798.87G/439.78G) 13:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852841.37 13:56:00 [---] pick_jobs_to_run() start 13:56:00 [---] pick_jobs_to_run() end 13:56:00 [---] [rr_sim] end 13:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:56:00 [---] [cpu_sched_debug] preliminary job list: 13:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:56:00 [---] [cpu_sched_debug] final job list: 13:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:56:00 [---] [cpu_sched_debug] enforce_run_list: end 13:56:00 [---] choose_project(): 50160.000000 13:56:00 [WUProp@Home] doing RPC to get work 13:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385439.09 cpu 2132372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132372.90 13:56:00 [---] Got 0 tasks 13:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:57:00 [---] [rr_sim] doing sim: CPU sched 13:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980521208.51G/439.78G) 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229577.72 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 222957.77 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222957.772084 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006619.95 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 200661.99 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423619.766960 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805957.95 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 180595.80 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604215.562348 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625362.16 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 162536.22 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766751.778197 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462825.94 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 146282.59 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913034.372462 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316543.35 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 131654.33 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044688.707299 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184889.01 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 118488.90 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163177.608654 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066400.11 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 106640.01 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269817.619872 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959760.10 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 95976.01 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365793.629969 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863784.09 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 86378.41 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452172.039056 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777405.68 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 77740.57 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529912.607235 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699665.11 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 69966.51 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599879.118595 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629698.60 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 62969.86 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662848.978820 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566728.74 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 56672.87 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719521.853022 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510055.87 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 51005.59 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770527.439804 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459050.28 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 45905.03 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816432.467908 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413145.25 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 41314.53 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857746.993201 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371830.73 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 37183.07 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894930.065965 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334647.65 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 33464.77 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928394.831453 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301182.89 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 30118.29 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958513.120391 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271064.60 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 27106.46 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985619.580436 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243958.14 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 24395.81 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010015.394477 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219562.33 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 21956.23 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031971.627113 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197606.09 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 19760.61 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051732.236486 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177845.48 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 17784.55 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069516.784921 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160060.94 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 16006.09 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085522.878513 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144054.84 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 14405.48 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099928.362746 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129649.36 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 12964.94 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112893.298556 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116684.42 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 11668.44 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124561.740784 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105015.98 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 10501.60 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135063.338790 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94514.38 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 9451.44 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144514.776995 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85062.94 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 8506.29 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153021.071380 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76556.65 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 7655.66 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160676.736326 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68900.98 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 6890.10 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167566.834777 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62010.89 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 6201.09 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173767.923384 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55809.80 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 5580.98 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179348.903129 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50228.82 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 5022.88 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184371.784900 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45205.94 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 4520.59 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188892.378494 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40685.34 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 4068.53 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192960.912729 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36616.81 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3661.68 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196622.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32955.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200222.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29355.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203822.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25755.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207422.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22155.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211022.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18555.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214622.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14955.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218222.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11355.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221822.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7755.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225422.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4155.13 sec) 13:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2229022.593540 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 555.13 sec) 13:57:00 [GPUGRID] [rr_sim] 2229577.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980521208.51G/439.78G) 13:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852838.63 13:57:00 [---] pick_jobs_to_run() start 13:57:00 [---] pick_jobs_to_run() end 13:57:00 [---] [rr_sim] end 13:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:57:00 [---] [cpu_sched_debug] preliminary job list: 13:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:57:00 [---] [cpu_sched_debug] final job list: 13:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:57:00 [---] [cpu_sched_debug] enforce_run_list: end 13:57:00 [---] choose_project(): 50220.000000 13:57:00 [WUProp@Home] doing RPC to get work 13:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385379.09 cpu 2132312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132312.90 13:57:00 [---] Got 0 tasks 13:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:58:00 [---] [rr_sim] doing sim: CPU sched 13:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980493618.16G/439.78G) 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229514.98 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 222951.50 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222951.498396 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006563.49 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 200656.35 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423607.846952 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805907.14 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 180590.71 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604198.560653 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625316.42 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 162531.64 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766730.202983 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462784.78 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 146278.48 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 913008.681081 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316506.30 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 131650.63 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044659.311369 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184855.67 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 118485.57 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163144.878628 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066370.11 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 106637.01 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269781.889161 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959733.09 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 95973.31 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365755.198641 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863759.79 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 86375.98 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452131.177172 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777383.81 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 77738.38 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529869.557851 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699645.43 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 69964.54 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599834.100462 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629680.88 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 62968.09 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662802.188812 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566712.80 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 56671.28 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719473.468326 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510041.52 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 51004.15 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770477.619889 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459037.36 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 45903.74 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816381.356296 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413133.63 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 41313.36 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857694.719063 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371820.26 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 37182.03 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894876.745552 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334638.24 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 33463.82 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928340.569393 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301174.41 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 30117.44 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958458.010849 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271056.97 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 27105.70 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985563.708160 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243951.28 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 24395.13 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009958.835740 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219556.15 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 21955.61 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031914.450562 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197600.53 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 19760.05 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051674.503902 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177840.48 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 17784.05 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069458.551907 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160056.43 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 16005.64 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085464.195113 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144050.79 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 14405.08 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099869.273997 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129645.71 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 12964.57 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112833.844993 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116681.14 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 11668.11 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124501.958890 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105013.03 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 10501.30 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135003.261397 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94511.72 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 9451.17 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144454.433653 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85060.55 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 8506.06 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152960.488684 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76554.50 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 7655.45 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160615.938211 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68899.05 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 6889.90 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167505.842786 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62009.14 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 6200.91 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173706.756903 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55808.23 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 5580.82 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179287.579609 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50227.40 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 5022.74 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184310.320044 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45204.66 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 4520.47 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188830.786435 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40684.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 4068.42 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192899.206188 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36615.78 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3661.58 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196560.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32954.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200160.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29354.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203760.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25754.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207360.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22154.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210960.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18554.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214560.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14954.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218160.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11354.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221760.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7754.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225360.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4154.20 sec) 13:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228960.783965 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 554.20 sec) 13:58:00 [GPUGRID] [rr_sim] 2229514.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980493618.16G/439.78G) 13:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852835.90 13:58:00 [---] pick_jobs_to_run() start 13:58:00 [---] pick_jobs_to_run() end 13:58:00 [---] [rr_sim] end 13:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:58:00 [---] [cpu_sched_debug] preliminary job list: 13:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:58:00 [---] [cpu_sched_debug] final job list: 13:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:58:00 [---] [cpu_sched_debug] enforce_run_list: end 13:58:00 [---] choose_project(): 50280.000000 13:58:00 [WUProp@Home] doing RPC to get work 13:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385319.09 cpu 2132252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132252.90 13:58:00 [---] Got 0 tasks 13:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 13:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 13:59:00 [---] [rr_sim] doing sim: CPU sched 13:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980466027.80G/439.78G) 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229452.25 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 222945.22 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222945.224708 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006507.02 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 200650.70 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423595.926945 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805856.32 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 180585.63 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604181.558958 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625270.69 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 162527.07 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766708.627770 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462743.62 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 146274.36 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912982.989700 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316469.26 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 131646.93 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044629.915438 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184822.33 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 118482.23 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163112.148602 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066340.10 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 106634.01 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269746.158449 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959706.09 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 95970.61 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365716.767312 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863735.48 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 86373.55 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452090.315288 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777361.93 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 77736.19 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529826.508467 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699625.74 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 69962.57 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599789.082328 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629663.16 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 62966.32 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662755.398803 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566696.85 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 56669.68 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719425.083630 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510027.16 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 51002.72 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770427.799975 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459024.45 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 45902.44 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816330.244685 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413122.00 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 41312.20 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857642.444924 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371809.80 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 37180.98 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894823.425140 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334628.82 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 33462.88 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928286.307333 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301165.94 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 30116.59 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958402.901308 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271049.35 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 27104.93 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985507.835884 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243944.41 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 24394.44 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009902.277004 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219549.97 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 21955.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031857.274011 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197594.97 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 19759.50 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051616.771318 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177835.48 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 17783.55 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069400.318893 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160051.93 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 16005.19 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085405.511712 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144046.74 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 14404.67 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099810.185248 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129642.06 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 12964.21 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112774.391431 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116677.86 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 11667.79 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124442.176996 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105010.07 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 10501.01 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134943.184004 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94509.06 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 9450.91 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144394.090311 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85058.16 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 8505.82 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152899.905988 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76552.34 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 7655.23 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160555.140096 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68897.11 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 6889.71 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167444.850794 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62007.40 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 6200.74 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173645.590423 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55806.66 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 5580.67 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179226.256088 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50225.99 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 5022.60 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184248.855187 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45203.39 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 4520.34 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188769.194376 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40683.05 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 4068.31 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192837.499646 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36614.75 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3661.47 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196498.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32953.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200098.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29353.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203698.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25753.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207298.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22153.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210898.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18553.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214498.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14953.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218098.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11353.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221698.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7753.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225298.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4153.27 sec) 13:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228898.974389 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 553.27 sec) 13:59:00 [GPUGRID] [rr_sim] 2229452.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980466027.80G/439.78G) 13:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852833.16 13:59:00 [---] pick_jobs_to_run() start 13:59:00 [---] pick_jobs_to_run() end 13:59:00 [---] [rr_sim] end 13:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 13:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 13:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 13:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 13:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 13:59:00 [---] [cpu_sched_debug] preliminary job list: 13:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:59:00 [---] [cpu_sched_debug] final job list: 13:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 13:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 13:59:00 [---] [cpu_sched_debug] enforce_run_list: end 13:59:00 [---] choose_project(): 50340.000000 13:59:00 [WUProp@Home] doing RPC to get work 13:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385259.09 cpu 2132192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132192.90 13:59:00 [---] Got 0 tasks 14:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:00:00 [---] [rr_sim] doing sim: CPU sched 14:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980438437.45G/439.78G) 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229389.51 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 222938.95 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222938.951019 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006450.56 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 200645.06 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423584.006937 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805805.50 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 180580.55 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604164.557263 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625224.95 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 162522.50 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766687.052556 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462702.46 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 146270.25 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912957.298320 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316432.21 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 131643.22 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044600.519507 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184788.99 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 118478.90 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163079.418576 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066310.09 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 106631.01 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269710.427738 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959679.08 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 95967.91 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365678.335983 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863711.17 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 86371.12 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452049.453405 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777340.06 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 77734.01 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529783.459083 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699606.05 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 69960.61 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599744.064195 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629645.45 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 62964.54 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662708.608795 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566680.90 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 56668.09 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719376.698935 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510012.81 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 51001.28 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770377.980061 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459011.53 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 45901.15 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816279.133074 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413110.38 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 41311.04 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857590.170786 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371799.34 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 37179.93 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894770.104727 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334619.41 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 33461.94 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928232.045274 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301157.46 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 30115.75 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958347.791766 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271041.72 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 27104.17 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985451.963609 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243937.55 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 24393.75 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009845.718267 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219543.79 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 21954.38 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031800.097460 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197589.41 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 19758.94 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051559.038733 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177830.47 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 17783.05 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069342.085879 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160047.42 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 16004.74 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085346.828311 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144042.68 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 14404.27 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099751.096499 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129638.41 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 12963.84 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112714.937869 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116674.57 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 11667.46 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124382.395101 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105007.12 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 10500.71 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134883.106611 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94506.40 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 9450.64 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144333.746969 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85055.76 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 8505.58 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152839.323292 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76550.19 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 7655.02 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160494.341982 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68895.17 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 6889.52 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167383.858803 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62005.65 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 6200.57 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173584.423942 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55805.09 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 5580.51 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179164.932567 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50224.58 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 5022.46 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184187.390330 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45202.12 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 4520.21 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188707.602317 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40681.91 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 4068.19 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192775.793104 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36613.72 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3661.37 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196437.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32952.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200037.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29352.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203637.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25752.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207237.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22152.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210837.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18552.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214437.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14952.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218037.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11352.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221637.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7752.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225237.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4152.35 sec) 14:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228837.164813 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 552.35 sec) 14:00:00 [GPUGRID] [rr_sim] 2229389.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980438437.45G/439.78G) 14:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852830.42 14:00:00 [---] pick_jobs_to_run() start 14:00:00 [---] pick_jobs_to_run() end 14:00:00 [---] [rr_sim] end 14:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:00:00 [---] [cpu_sched_debug] preliminary job list: 14:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:00:00 [---] [cpu_sched_debug] final job list: 14:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:00:00 [---] [cpu_sched_debug] enforce_run_list: end 14:00:00 [---] choose_project(): 50400.000000 14:00:00 [WUProp@Home] doing RPC to get work 14:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385199.09 cpu 2132132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132132.90 14:00:00 [---] Got 0 tasks 14:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:01:00 [---] [rr_sim] doing sim: CPU sched 14:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980410847.09G/439.78G) 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229326.77 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 222932.68 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222932.677331 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006394.10 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 200639.41 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423572.086929 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805754.69 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 180575.47 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604147.555568 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625179.22 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 162517.92 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766665.477342 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462661.30 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 146266.13 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912931.606939 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316395.17 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 131639.52 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044571.123576 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184755.65 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 118475.56 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163046.688550 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066280.08 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 106628.01 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269674.697026 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959652.08 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 95965.21 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365639.904655 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863686.87 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 86368.69 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1452008.591521 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777318.18 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 77731.82 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529740.409700 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699586.36 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 69958.64 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599699.046061 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629627.73 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 62962.77 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662661.818786 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566664.95 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 56666.50 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719328.314239 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509998.46 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 50999.85 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770328.160146 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458998.61 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 45899.86 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816228.021463 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413098.75 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 41309.88 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857537.896648 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371788.88 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 37178.89 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894716.784314 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334609.99 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 33461.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928177.783214 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301148.99 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 30114.90 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958292.682224 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271034.09 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 27103.41 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985396.091333 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243930.68 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 24393.07 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009789.159531 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219537.61 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 21953.76 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031742.920909 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197583.85 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 19758.39 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051501.306149 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177825.47 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 17782.55 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069283.852865 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160042.92 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 16004.29 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085288.144910 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144038.63 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 14403.86 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099692.007750 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129634.77 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 12963.48 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112655.484306 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116671.29 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 11667.13 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124322.613207 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105004.16 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 10500.42 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134823.029218 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94503.74 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 9450.37 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144273.403627 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85053.37 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 8505.34 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152778.740596 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76548.03 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 7654.80 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160433.543867 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68893.23 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 6889.32 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167322.866812 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62003.91 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 6200.39 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173523.257462 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55803.52 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 5580.35 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179103.609047 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50223.16 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 5022.32 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184125.925473 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45200.85 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 4520.08 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188646.010257 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40680.76 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 4068.08 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192714.086563 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36612.69 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3661.27 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196375.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32951.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199975.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29351.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203575.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25751.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207175.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22151.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210775.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18551.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214375.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14951.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217975.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11351.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221575.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7751.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225175.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4151.42 sec) 14:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228775.355238 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 551.42 sec) 14:01:00 [GPUGRID] [rr_sim] 2229326.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980410847.09G/439.78G) 14:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852827.69 14:01:00 [---] pick_jobs_to_run() start 14:01:00 [---] pick_jobs_to_run() end 14:01:00 [---] [rr_sim] end 14:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:01:00 [---] [cpu_sched_debug] preliminary job list: 14:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:01:00 [---] [cpu_sched_debug] final job list: 14:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:01:00 [---] [cpu_sched_debug] enforce_run_list: end 14:01:00 [---] choose_project(): 50460.000000 14:01:00 [WUProp@Home] doing RPC to get work 14:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385139.09 cpu 2132072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132072.90 14:01:00 [---] Got 0 tasks 14:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:02:00 [---] [rr_sim] doing sim: CPU sched 14:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980383256.73G/439.78G) 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229264.04 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 222926.40 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222926.403643 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006337.63 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 200633.76 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423560.166922 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805703.87 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 180570.39 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604130.553873 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625133.48 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 162513.35 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766643.902128 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462620.13 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 146262.01 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912905.915558 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316358.12 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 131635.81 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044541.727646 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184722.31 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 118472.23 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1163013.958524 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066250.08 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 106625.01 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269638.966315 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959625.07 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 95962.51 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365601.473326 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863662.56 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 86366.26 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451967.729637 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777296.31 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 77729.63 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529697.360316 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699566.68 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 69956.67 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599654.027927 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629610.01 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 62961.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662615.028778 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566649.01 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 56664.90 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719279.929543 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509984.11 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 50998.41 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770278.340232 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458985.70 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 45898.57 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816176.909851 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413087.13 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 41308.71 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857485.622509 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371778.41 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 37177.84 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894663.463901 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334600.57 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 33460.06 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928123.521154 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301140.52 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 30114.05 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958237.572682 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271026.46 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 27102.65 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985340.219057 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243923.82 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 24392.38 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009732.600794 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219531.44 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 21953.14 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031685.744358 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197578.29 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 19757.83 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051443.573565 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177820.46 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 17782.05 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069225.619851 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160038.42 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 16003.84 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085229.461509 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144034.57 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 14403.46 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099632.919001 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129631.12 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 12963.11 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112596.030744 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116668.01 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 11666.80 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124262.831313 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105001.21 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 10500.12 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134762.951825 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94501.08 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 9450.11 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144213.060285 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85050.98 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 8505.10 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152718.157900 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76545.88 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 7654.59 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160372.745753 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68891.29 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 6889.13 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167261.874820 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62002.16 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 6200.22 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173462.090981 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55801.95 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 5580.19 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179042.285526 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50221.75 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 5022.18 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184064.460617 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45199.58 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 4519.96 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188584.418198 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40679.62 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 4067.96 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192652.380021 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36611.66 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3661.17 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196313.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32950.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199913.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29350.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203513.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25750.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207113.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22150.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210713.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18550.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214313.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14950.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217913.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11350.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221513.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7750.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225113.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4150.49 sec) 14:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228713.545662 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 550.49 sec) 14:02:00 [GPUGRID] [rr_sim] 2229264.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980383256.73G/439.78G) 14:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852824.95 14:02:00 [---] pick_jobs_to_run() start 14:02:00 [---] pick_jobs_to_run() end 14:02:00 [---] [rr_sim] end 14:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:02:00 [---] [cpu_sched_debug] preliminary job list: 14:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:02:00 [---] [cpu_sched_debug] final job list: 14:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:02:00 [---] [cpu_sched_debug] enforce_run_list: end 14:02:00 [---] choose_project(): 50520.000000 14:02:00 [WUProp@Home] doing RPC to get work 14:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385079.09 cpu 2132012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2132012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2132012.90 14:02:00 [---] Got 0 tasks 14:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:03:00 [---] [rr_sim] doing sim: CPU sched 14:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980355666.38G/439.78G) 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229201.30 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 222920.13 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222920.129955 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006281.17 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 200628.12 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423548.246914 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805653.05 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 180565.31 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604113.552177 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625087.75 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 162508.77 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766622.326915 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462578.97 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 146257.90 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912880.224178 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316321.08 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 131632.11 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044512.331715 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184688.97 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 118468.90 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162981.228498 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066220.07 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 106622.01 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269603.235603 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959598.06 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 95959.81 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365563.041998 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863638.26 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 86363.83 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451926.867753 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777274.43 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 77727.44 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529654.310932 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699546.99 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 69954.70 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599609.009794 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629592.29 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 62959.23 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662568.238769 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566633.06 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 56663.31 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719231.544847 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509969.75 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 50996.98 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770228.520317 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458972.78 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 45897.28 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816125.798240 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413075.50 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 41307.55 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857433.348371 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371767.95 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 37176.80 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894610.143489 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334591.16 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 33459.12 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928069.259095 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301132.04 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 30113.20 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958182.463140 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271018.84 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 27101.88 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985284.346781 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243916.95 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 24391.70 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009676.042057 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219525.26 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 21952.53 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031628.567806 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197572.73 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 19757.27 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051385.840981 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177815.46 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 17781.55 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069167.386837 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160033.91 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 16003.39 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085170.778108 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144030.52 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 14403.05 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099573.830252 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129627.47 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 12962.75 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112536.577182 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116664.72 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 11666.47 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124203.049419 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104998.25 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 10499.83 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134702.874431 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94498.43 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 9449.84 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144152.716943 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85048.58 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 8504.86 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152657.575204 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76543.72 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 7654.37 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160311.947638 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68889.35 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 6888.94 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167200.882829 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62000.42 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 6200.04 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173400.924501 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55800.38 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 5580.04 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178980.962006 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50220.34 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 5022.03 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184002.995760 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45198.30 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 4519.83 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188522.826139 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40678.47 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 4067.85 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192590.673480 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36610.63 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3661.06 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196251.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32949.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199851.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29349.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203451.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25749.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207051.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22149.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210651.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18549.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214251.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14949.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217851.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11349.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221451.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7749.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225051.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4149.56 sec) 14:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228651.736086 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 549.56 sec) 14:03:00 [GPUGRID] [rr_sim] 2229201.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980355666.38G/439.78G) 14:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852822.21 14:03:00 [---] pick_jobs_to_run() start 14:03:00 [---] pick_jobs_to_run() end 14:03:00 [---] [rr_sim] end 14:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:03:00 [---] [cpu_sched_debug] preliminary job list: 14:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:03:00 [---] [cpu_sched_debug] final job list: 14:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:03:00 [---] [cpu_sched_debug] enforce_run_list: end 14:03:00 [---] choose_project(): 50580.000000 14:03:00 [WUProp@Home] doing RPC to get work 14:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 385019.09 cpu 2131952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131952.90 14:03:00 [---] Got 0 tasks 14:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:04:00 [---] [rr_sim] doing sim: CPU sched 14:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980328076.02G/439.78G) 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229138.56 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 222913.86 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222913.856267 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006224.71 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 200622.47 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423536.326906 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805602.24 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 180560.22 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604096.550482 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1625042.01 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 162504.20 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766600.751701 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462537.81 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 146253.78 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912854.532797 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316284.03 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 131628.40 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044482.935784 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184655.63 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 118465.56 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162948.498472 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066190.06 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 106619.01 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269567.504892 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959571.06 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 95957.11 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365524.610669 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863613.95 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 86361.40 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451886.005869 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777252.56 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 77725.26 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529611.261548 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699527.30 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 69952.73 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599563.991660 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629574.57 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 62957.46 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662521.448761 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566617.11 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 56661.71 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719183.160151 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509955.40 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 50995.54 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770178.700403 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458959.86 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 45895.99 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816074.686629 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413063.88 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 41306.39 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857381.074233 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371757.49 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 37175.75 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894556.823076 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334581.74 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 33458.17 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928014.997035 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301123.57 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 30112.36 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958127.353598 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271011.21 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 27101.12 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985228.474505 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243910.09 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 24391.01 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009619.483321 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219519.08 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 21951.91 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031571.391255 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197567.17 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 19756.72 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051328.108396 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177810.45 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 17781.05 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069109.153823 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160029.41 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 16002.94 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085112.094708 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144026.47 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 14402.65 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099514.741503 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129623.82 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 12962.38 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112477.123620 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116661.44 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 11666.14 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124143.267524 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104995.30 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 10499.53 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134642.797038 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94495.77 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 9449.58 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144092.373601 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85046.19 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 8504.62 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152596.992508 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76541.57 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 7654.16 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160251.149523 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68887.41 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 6888.74 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167139.890838 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61998.67 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 6199.87 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173339.758020 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55798.80 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 5579.88 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178919.638485 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50218.92 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 5021.89 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183941.530903 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45197.03 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 4519.70 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188461.234079 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40677.33 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 4067.73 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192528.966938 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36609.60 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3660.96 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196189.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32948.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199789.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29348.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203389.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25748.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206989.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22148.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210589.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18548.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214189.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14948.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217789.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11348.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221389.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7748.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224989.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4148.64 sec) 14:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228589.926511 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 548.64 sec) 14:04:00 [GPUGRID] [rr_sim] 2229138.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980328076.02G/439.78G) 14:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852819.48 14:04:00 [---] pick_jobs_to_run() start 14:04:00 [---] pick_jobs_to_run() end 14:04:00 [---] [rr_sim] end 14:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:04:00 [---] [cpu_sched_debug] preliminary job list: 14:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:04:00 [---] [cpu_sched_debug] final job list: 14:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:04:00 [---] [cpu_sched_debug] enforce_run_list: end 14:04:00 [---] choose_project(): 50640.000000 14:04:00 [WUProp@Home] doing RPC to get work 14:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384959.09 cpu 2131892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131892.90 14:04:00 [---] Got 0 tasks 14:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:05:00 [---] [rr_sim] doing sim: CPU sched 14:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980300485.67G/439.78G) 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229075.83 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 222907.58 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222907.582578 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006168.24 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 200616.82 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423524.406899 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805551.42 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 180555.14 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604079.548787 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624996.28 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 162499.63 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766579.176487 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462496.65 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 146249.66 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912828.841417 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316246.98 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 131624.70 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044453.539853 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184622.29 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 118462.23 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162915.768446 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066160.06 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 106616.01 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269531.774180 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959544.05 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 95954.41 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365486.179340 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863589.65 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 86358.96 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451845.143985 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777230.68 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 77723.07 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529568.212165 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699507.61 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 69950.76 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599518.973527 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629556.85 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 62955.69 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662474.658752 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566601.17 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 56660.12 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719134.775455 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509941.05 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 50994.11 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770128.880488 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458946.95 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 45894.69 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1816023.575018 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413052.25 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 41305.23 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857328.800094 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371747.03 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 37174.70 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894503.502663 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334572.32 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 33457.23 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927960.734975 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301115.09 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 30111.51 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958072.244056 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271003.58 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 27100.36 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985172.602229 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243903.22 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 24390.32 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009562.924584 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219512.90 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 21951.29 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031514.214704 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197561.61 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 19756.16 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051270.375812 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177805.45 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 17780.54 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069050.920809 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160024.90 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 16002.49 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085053.411307 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144022.41 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 14402.24 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099455.652754 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129620.17 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 12962.02 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112417.670057 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116658.16 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 11665.82 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124083.485630 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104992.34 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 10499.23 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134582.719645 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94493.11 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 9449.31 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144032.030259 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85043.80 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 8504.38 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152536.409812 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76539.42 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 7653.94 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160190.351409 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68885.47 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 6888.55 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167078.898846 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61996.93 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 6199.69 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173278.591540 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55797.23 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 5579.72 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178858.314964 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50217.51 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 5021.75 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183880.066046 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45195.76 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 4519.58 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188399.642020 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40676.18 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 4067.62 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192467.260396 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36608.57 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3660.86 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196128.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32947.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199728.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29347.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203328.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25747.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206928.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22147.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210528.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18547.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214128.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14947.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217728.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11347.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221328.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7747.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224928.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4147.71 sec) 14:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228528.116935 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 547.71 sec) 14:05:00 [GPUGRID] [rr_sim] 2229075.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980300485.67G/439.78G) 14:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852816.74 14:05:00 [---] pick_jobs_to_run() start 14:05:00 [---] pick_jobs_to_run() end 14:05:00 [---] [rr_sim] end 14:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:05:00 [---] [cpu_sched_debug] preliminary job list: 14:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:05:00 [---] [cpu_sched_debug] final job list: 14:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:05:00 [---] [cpu_sched_debug] enforce_run_list: end 14:05:00 [---] choose_project(): 50700.000000 14:05:00 [WUProp@Home] doing RPC to get work 14:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384899.09 cpu 2131832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131832.90 14:05:00 [---] Got 0 tasks 14:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:06:00 [---] [rr_sim] doing sim: CPU sched 14:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980272895.31G/439.78G) 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2229013.09 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 222901.31 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222901.308890 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006111.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 200611.18 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423512.486891 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805500.60 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 180550.06 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604062.547092 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624950.54 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 162495.05 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766557.601273 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462455.49 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 146245.55 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912803.150036 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316209.94 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 131620.99 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044424.143923 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184588.94 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 118458.89 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162883.038420 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066130.05 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 106613.01 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269496.043469 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959517.05 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 95951.70 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365447.748012 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863565.34 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 86356.53 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451804.282101 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777208.81 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 77720.88 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529525.162781 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699487.93 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 69948.79 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599473.955393 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629539.13 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 62953.91 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662427.868744 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566585.22 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 56658.52 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719086.390760 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509926.70 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 50992.67 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770079.060574 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458934.03 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 45893.40 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815972.463406 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413040.63 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 41304.06 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857276.525956 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371736.56 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 37173.66 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894450.182251 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334562.91 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 33456.29 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927906.472916 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301106.62 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 30110.66 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958017.134514 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270995.95 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 27099.60 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985116.729953 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243896.36 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 24389.64 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009506.365848 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219506.72 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 21950.67 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031457.038153 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197556.05 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 19755.61 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051212.643228 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177800.45 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 17780.04 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068992.687795 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160020.40 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 16002.04 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084994.727906 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144018.36 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 14401.84 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099396.564005 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129616.52 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 12961.65 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112358.216495 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116654.87 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 11665.49 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124023.703736 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104989.39 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 10498.94 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134522.642252 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94490.45 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 9449.04 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143971.686917 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85041.40 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 8504.14 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152475.827116 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76537.26 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 7653.73 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160129.553294 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68883.54 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 6888.35 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167017.906855 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61995.18 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 6199.52 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173217.425060 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55795.66 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 5579.57 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178796.991444 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50216.10 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 5021.61 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183818.601190 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45194.49 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 4519.45 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188338.049961 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40675.04 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 4067.50 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192405.553855 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36607.54 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3660.75 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196066.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32946.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199666.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29346.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203266.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25746.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206866.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22146.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210466.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18546.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214066.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14946.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217666.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11346.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221266.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7746.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224866.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4146.78 sec) 14:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228466.307359 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 546.78 sec) 14:06:00 [GPUGRID] [rr_sim] 2229013.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980272895.31G/439.78G) 14:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852814.00 14:06:00 [---] pick_jobs_to_run() start 14:06:00 [---] pick_jobs_to_run() end 14:06:00 [---] [rr_sim] end 14:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:06:00 [---] [cpu_sched_debug] preliminary job list: 14:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:06:00 [---] [cpu_sched_debug] final job list: 14:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:06:00 [---] [cpu_sched_debug] enforce_run_list: end 14:06:00 [---] choose_project(): 50760.000000 14:06:00 [WUProp@Home] doing RPC to get work 14:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384839.09 cpu 2131772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131772.90 14:06:00 [---] Got 0 tasks 14:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:07:00 [---] [rr_sim] doing sim: CPU sched 14:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980245304.95G/439.78G) 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228950.35 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 222895.04 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222895.035202 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2006055.32 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 200605.53 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423500.566884 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805449.79 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 180544.98 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604045.545397 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624904.81 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 162490.48 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766536.026059 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462414.33 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 146241.43 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912777.458655 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316172.89 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 131617.29 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044394.747992 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184555.60 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 118455.56 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162850.308395 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066100.04 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 106610.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269460.312757 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959490.04 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 95949.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365409.316683 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863541.04 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 86354.10 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451763.420217 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777186.93 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 77718.69 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529482.113397 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699468.24 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 69946.82 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599428.937259 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629521.41 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 62952.14 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662381.078735 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566569.27 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 56656.93 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1719038.006064 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509912.35 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 50991.23 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1770029.240659 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458921.11 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 45892.11 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815921.351795 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413029.00 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 41302.90 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857224.251818 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371726.10 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 37172.61 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894396.861838 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334553.49 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 33455.35 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927852.210856 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301098.14 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 30109.81 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957962.024972 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270988.33 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 27098.83 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985060.857677 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243889.49 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 24388.95 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009449.807111 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219500.54 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 21950.05 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031399.861602 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197550.49 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 19755.05 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051154.910644 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177795.44 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 17779.54 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068934.454781 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160015.90 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 16001.59 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084936.044505 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144014.31 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 14401.43 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099337.475256 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129612.88 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 12961.29 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112298.762933 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116651.59 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 11665.16 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123963.921841 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104986.43 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 10498.64 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134462.564859 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94487.79 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 9448.78 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143911.343575 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85039.01 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 8503.90 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152415.244420 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76535.11 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 7653.51 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160068.755180 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68881.60 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 6888.16 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166956.914864 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61993.44 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 6199.34 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173156.258579 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55794.09 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 5579.41 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178735.667923 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50214.68 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 5021.47 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183757.136333 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45193.22 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 4519.32 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188276.457901 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40673.89 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 4067.39 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192343.847313 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36606.50 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3660.65 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196004.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32945.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199604.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29345.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203204.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25745.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206804.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22145.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210404.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18545.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214004.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14945.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217604.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11345.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221204.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7745.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224804.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4145.85 sec) 14:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228404.497784 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 545.85 sec) 14:07:00 [GPUGRID] [rr_sim] 2228950.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980245304.95G/439.78G) 14:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852811.26 14:07:00 [---] pick_jobs_to_run() start 14:07:00 [---] pick_jobs_to_run() end 14:07:00 [---] [rr_sim] end 14:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:07:00 [---] [cpu_sched_debug] preliminary job list: 14:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:07:00 [---] [cpu_sched_debug] final job list: 14:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:07:00 [---] [cpu_sched_debug] enforce_run_list: end 14:07:00 [---] choose_project(): 50820.000000 14:07:00 [WUProp@Home] doing RPC to get work 14:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384779.09 cpu 2131712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131712.90 14:07:00 [---] Got 0 tasks 14:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:08:00 [---] [rr_sim] doing sim: CPU sched 14:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980217714.60G/439.78G) 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228887.62 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 222888.76 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222888.761514 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005998.85 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 200599.89 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423488.646876 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805398.97 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 180539.90 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604028.543702 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624859.07 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 162485.91 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766514.450846 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462373.16 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 146237.32 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912751.767275 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316135.85 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 131613.58 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044365.352061 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184522.26 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 118452.23 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162817.578369 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066070.04 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 106607.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269424.582045 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959463.03 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 95946.30 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365370.885355 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863516.73 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 86351.67 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451722.558333 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777165.06 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 77716.51 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529439.064013 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699448.55 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 69944.86 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599383.919126 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629503.70 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 62950.37 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662334.288727 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566553.33 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 56655.33 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718989.621368 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509897.99 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 50989.80 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769979.420745 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458908.19 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 45890.82 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815870.240184 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413017.37 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 41301.74 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857171.977679 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371715.64 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 37171.56 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894343.541425 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334544.07 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 33454.41 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927797.948796 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301089.67 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 30108.97 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957906.915430 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270980.70 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 27098.07 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985004.985401 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243882.63 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 24388.26 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009393.248375 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219494.37 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 21949.44 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031342.685051 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197544.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 19754.49 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051097.178059 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177790.44 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 17779.04 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068876.221767 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160011.39 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 16001.14 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084877.361104 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144010.25 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 14401.03 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099278.386508 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129609.23 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 12960.92 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112239.309370 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116648.31 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 11664.83 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123904.139947 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104983.48 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 10498.35 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134402.487466 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94485.13 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 9448.51 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143851.000233 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85036.61 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 8503.66 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152354.661724 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76532.95 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 7653.30 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160007.957065 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68879.66 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 6887.97 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166895.922872 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61991.69 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 6199.17 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173095.092099 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55792.52 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 5579.25 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178674.344403 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50213.27 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 5021.33 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183695.671476 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45191.94 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 4519.19 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188214.865842 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40672.75 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 4067.27 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192282.140772 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36605.47 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3660.55 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195942.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32944.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199542.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29344.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203142.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25744.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206742.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22144.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210342.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18544.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213942.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14944.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217542.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11344.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221142.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7744.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224742.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4144.93 sec) 14:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228342.688208 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 544.93 sec) 14:08:00 [GPUGRID] [rr_sim] 2228887.62: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980217714.60G/439.78G) 14:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852808.53 14:08:00 [---] pick_jobs_to_run() start 14:08:00 [---] pick_jobs_to_run() end 14:08:00 [---] [rr_sim] end 14:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:08:00 [---] [cpu_sched_debug] preliminary job list: 14:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:08:00 [---] [cpu_sched_debug] final job list: 14:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:08:00 [---] [cpu_sched_debug] enforce_run_list: end 14:08:00 [---] choose_project(): 50880.000000 14:08:00 [WUProp@Home] doing RPC to get work 14:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384719.09 cpu 2131652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131652.90 14:08:00 [---] Got 0 tasks 14:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:09:00 [---] [rr_sim] doing sim: CPU sched 14:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980190124.24G/439.78G) 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228824.88 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 222882.49 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222882.487825 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005942.39 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 200594.24 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423476.726868 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805348.15 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 180534.82 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 604011.542007 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624813.34 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 162481.33 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766492.875632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462332.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 146233.20 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912726.075894 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316098.80 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 131609.88 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044335.956130 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184488.92 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 118448.89 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162784.848343 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066040.03 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 106604.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269388.851334 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959436.03 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 95943.60 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365332.454026 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863492.42 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 86349.24 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451681.696449 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777143.18 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 77714.32 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529396.014630 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699428.86 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 69942.89 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599338.900992 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629485.98 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 62948.60 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662287.498718 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566537.38 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 56653.74 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718941.236672 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509883.64 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 50988.36 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769929.600830 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458895.28 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 45889.53 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815819.128573 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413005.75 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 41300.57 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857119.703541 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371705.17 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 37170.52 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894290.221012 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334534.66 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 33453.47 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927743.686737 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301081.19 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 30108.12 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957851.805888 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270973.07 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 27097.31 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984949.113125 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243875.77 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 24387.58 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009336.689638 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219488.19 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 21948.82 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031285.508500 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197539.37 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 19753.94 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051039.445475 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177785.43 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 17778.54 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068817.988753 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160006.89 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 16000.69 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084818.677703 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144006.20 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 14400.62 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099219.297759 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129605.58 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 12960.56 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112179.855808 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116645.02 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 11664.50 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123844.358053 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104980.52 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 10498.05 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134342.410073 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94482.47 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 9448.25 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143790.656891 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85034.22 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 8503.42 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152294.079028 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76530.80 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 7653.08 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159947.158950 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68877.72 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 6887.77 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166834.930881 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61989.95 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 6198.99 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173033.925618 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55790.95 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 5579.10 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178613.020882 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50211.86 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 5021.19 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183634.206619 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45190.67 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 4519.07 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188153.273783 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40671.60 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 4067.16 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192220.434230 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36604.44 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3660.44 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195880.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32944.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199480.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29344.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203080.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25744.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206680.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22144.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210280.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18544.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213880.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14944.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217480.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11344.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221080.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7744.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224680.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4144.00 sec) 14:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228280.878632 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 544.00 sec) 14:09:00 [GPUGRID] [rr_sim] 2228824.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980190124.24G/439.78G) 14:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852805.79 14:09:00 [---] pick_jobs_to_run() start 14:09:00 [---] pick_jobs_to_run() end 14:09:00 [---] [rr_sim] end 14:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:09:00 [---] [cpu_sched_debug] preliminary job list: 14:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:09:00 [---] [cpu_sched_debug] final job list: 14:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:09:00 [---] [cpu_sched_debug] enforce_run_list: end 14:09:00 [---] choose_project(): 50940.000000 14:09:00 [WUProp@Home] doing RPC to get work 14:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384659.09 cpu 2131592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131592.90 14:09:00 [---] Got 0 tasks 14:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:10:00 [---] [rr_sim] doing sim: CPU sched 14:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980162533.88G/439.78G) 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228762.14 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 222876.21 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222876.214137 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005885.93 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 200588.59 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423464.806861 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805297.33 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 180529.73 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603994.540312 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624767.60 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 162476.76 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766471.300418 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462290.84 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 146229.08 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912700.384514 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316061.76 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 131606.18 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044306.560199 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184455.58 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 118445.56 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162752.118317 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1066010.02 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 106601.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269353.120622 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959409.02 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 95940.90 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365294.022697 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863468.12 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 86346.81 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451640.834565 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777121.31 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 77712.13 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529352.965246 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699409.18 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 69940.92 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599293.882858 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629468.26 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 62946.83 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662240.708710 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566521.43 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 56652.14 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718892.851976 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509869.29 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 50986.93 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769879.780916 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458882.36 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 45888.24 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815768.016962 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412994.12 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 41299.41 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857067.429403 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371694.71 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 37169.47 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894236.900600 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334525.24 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 33452.52 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927689.424677 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301072.72 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 30107.27 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957796.696347 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270965.45 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 27096.54 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984893.240849 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243868.90 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 24386.89 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009280.130902 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219482.01 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 21948.20 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031228.331949 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197533.81 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 19753.38 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050981.712891 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177780.43 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 17778.04 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068759.755739 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160002.39 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 16000.24 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084759.994303 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144002.15 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 14400.21 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099160.209010 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129601.93 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 12960.19 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112120.402246 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116641.74 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 11664.17 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123784.576159 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104977.57 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 10497.76 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134282.332680 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94479.81 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 9447.98 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143730.313549 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85031.83 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 8503.18 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152233.496332 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76528.65 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 7652.86 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159886.360836 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68875.78 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 6887.58 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166773.938889 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61988.20 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 6198.82 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172972.759138 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55789.38 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 5578.94 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178551.697361 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50210.44 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 5021.04 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183572.741762 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45189.40 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 4518.94 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188091.681723 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40670.46 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 4067.05 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192158.727688 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36603.41 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3660.34 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195819.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32943.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199419.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29343.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203019.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25743.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206619.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22143.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210219.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18543.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213819.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14943.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217419.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11343.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221019.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7743.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224619.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4143.07 sec) 14:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228219.069057 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 543.07 sec) 14:10:00 [GPUGRID] [rr_sim] 2228762.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980162533.88G/439.78G) 14:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852803.05 14:10:00 [---] pick_jobs_to_run() start 14:10:00 [---] pick_jobs_to_run() end 14:10:00 [---] [rr_sim] end 14:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:10:00 [---] [cpu_sched_debug] preliminary job list: 14:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:10:00 [---] [cpu_sched_debug] final job list: 14:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:10:00 [---] [cpu_sched_debug] enforce_run_list: end 14:10:00 [---] choose_project(): 51000.000000 14:10:00 [WUProp@Home] doing RPC to get work 14:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384599.09 cpu 2131532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131532.90 14:10:00 [---] Got 0 tasks 14:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:11:00 [---] [rr_sim] doing sim: CPU sched 14:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980134943.53G/439.78G) 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228699.40 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 222869.94 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222869.940449 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005829.46 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 200582.95 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423452.886853 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805246.52 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 180524.65 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603977.538617 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624721.87 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 162472.19 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766449.725204 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462249.68 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 146224.97 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912674.693133 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1316024.71 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 131602.47 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044277.164269 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184422.24 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 118442.22 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162719.388291 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065980.02 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 106598.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269317.389911 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959382.01 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 95938.20 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365255.591369 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863443.81 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 86344.38 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451599.972681 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777099.43 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 77709.94 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529309.915862 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699389.49 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 69938.95 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599248.864725 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629450.54 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 62945.05 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662193.918701 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566505.49 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 56650.55 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718844.467280 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509854.94 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 50985.49 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769829.961001 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458869.44 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 45886.94 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815716.905350 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412982.50 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 41298.25 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1857015.155264 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371684.25 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 37168.42 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894183.580187 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334515.82 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 33451.58 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927635.162617 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301064.24 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 30106.42 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957741.586805 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270957.82 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 27095.78 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984837.368573 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243862.04 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 24386.20 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009223.572165 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219475.83 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 21947.58 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031171.155398 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197528.25 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 19752.82 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050923.980307 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177775.42 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 17777.54 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068701.522725 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159997.88 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 15999.79 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084701.310902 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143998.09 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 14399.81 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099101.120261 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129598.28 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 12959.83 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112060.948684 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116638.46 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 11663.85 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123724.794264 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104974.61 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 10497.46 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134222.255287 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94477.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 9447.71 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143669.970207 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85029.43 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 8502.94 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152172.913636 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76526.49 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 7652.65 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159825.562721 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68873.84 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 6887.38 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166712.946898 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61986.46 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 6198.65 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172911.592657 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55787.81 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 5578.78 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178490.373841 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50209.03 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 5020.90 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183511.276906 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45188.13 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 4518.81 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188030.089664 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40669.31 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 4066.93 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192097.021147 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36602.38 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3660.24 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195757.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32942.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199357.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29342.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202957.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25742.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206557.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22142.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210157.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18542.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213757.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14942.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217357.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11342.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220957.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7742.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224557.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4142.15 sec) 14:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228157.259481 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 542.15 sec) 14:11:00 [GPUGRID] [rr_sim] 2228699.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980134943.53G/439.78G) 14:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852800.32 14:11:00 [---] pick_jobs_to_run() start 14:11:00 [---] pick_jobs_to_run() end 14:11:00 [---] [rr_sim] end 14:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:11:00 [---] [cpu_sched_debug] preliminary job list: 14:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:11:00 [---] [cpu_sched_debug] final job list: 14:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:11:00 [---] [cpu_sched_debug] enforce_run_list: end 14:11:00 [---] choose_project(): 51060.000000 14:11:00 [WUProp@Home] doing RPC to get work 14:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384539.09 cpu 2131472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131472.90 14:11:00 [---] Got 0 tasks 14:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:12:00 [---] [rr_sim] doing sim: CPU sched 14:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980107353.17G/439.78G) 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228636.67 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 222863.67 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222863.666761 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005773.00 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 200577.30 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423440.966846 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805195.70 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 180519.57 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603960.536922 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624676.13 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 162467.61 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766428.149991 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462208.52 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 146220.85 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912649.001752 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315987.67 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 131598.77 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044247.768338 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184388.90 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 118438.89 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162686.658265 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065950.01 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 106595.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269281.659199 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959355.01 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 95935.50 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365217.160040 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863419.51 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 86341.95 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451559.110797 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777077.56 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 77707.76 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529266.866478 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699369.80 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 69936.98 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599203.846591 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629432.82 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 62943.28 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662147.128693 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566489.54 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 56648.95 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718796.082584 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509840.59 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 50984.06 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769780.141087 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458856.53 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 45885.65 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815665.793739 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412970.87 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 41297.09 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856962.881126 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371673.79 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 37167.38 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894130.259774 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334506.41 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 33450.64 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927580.900558 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301055.77 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 30105.58 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957686.477263 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270950.19 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 27095.02 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984781.496297 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243855.17 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 24385.52 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009167.013428 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219469.65 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 21946.97 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031113.978846 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197522.69 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 19752.27 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050866.247723 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177770.42 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 17777.04 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068643.289711 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159993.38 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 15999.34 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084642.627501 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143994.04 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 14399.40 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099042.031512 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129594.64 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 12959.46 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112001.495121 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116635.17 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 11663.52 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123665.012370 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104971.66 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 10497.17 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134162.177894 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94474.49 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 9447.45 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143609.626865 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85027.04 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 8502.70 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152112.330940 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76524.34 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 7652.43 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159764.764607 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68871.90 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 6887.19 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166651.954907 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61984.71 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 6198.47 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172850.426177 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55786.24 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 5578.62 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178429.050320 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50207.62 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 5020.76 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183449.812049 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45186.86 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 4518.69 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187968.497605 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40668.17 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 4066.82 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192035.314605 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36601.35 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3660.14 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195695.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32941.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199295.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29341.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202895.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25741.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206495.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22141.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210095.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18541.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213695.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14941.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217295.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11341.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220895.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7741.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224495.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4141.22 sec) 14:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228095.449906 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 541.22 sec) 14:12:00 [GPUGRID] [rr_sim] 2228636.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980107353.17G/439.78G) 14:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852797.58 14:12:00 [---] pick_jobs_to_run() start 14:12:00 [---] pick_jobs_to_run() end 14:12:00 [---] [rr_sim] end 14:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:12:00 [---] [cpu_sched_debug] preliminary job list: 14:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:12:00 [---] [cpu_sched_debug] final job list: 14:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:12:00 [---] [cpu_sched_debug] enforce_run_list: end 14:12:00 [---] choose_project(): 51120.000000 14:12:00 [WUProp@Home] doing RPC to get work 14:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384479.09 cpu 2131412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131412.90 14:12:00 [---] Got 0 tasks 14:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:13:00 [---] [rr_sim] doing sim: CPU sched 14:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980079762.82G/439.78G) 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228573.93 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 222857.39 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222857.393073 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005716.54 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 200571.65 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423429.046838 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805144.88 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 180514.49 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603943.535227 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624630.40 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 162463.04 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766406.574777 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462167.36 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 146216.74 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912623.310372 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315950.62 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 131595.06 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044218.372407 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184355.56 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 118435.56 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162653.928239 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065920.00 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 106592.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269245.928488 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959328.00 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 95932.80 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365178.728712 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863395.20 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 86339.52 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451518.248913 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777055.68 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 77705.57 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529223.817094 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699350.11 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 69935.01 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599158.828458 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629415.10 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 62941.51 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662100.338684 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566473.59 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 56647.36 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718747.697889 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509826.23 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 50982.62 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769730.321172 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458843.61 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 45884.36 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815614.682128 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412959.25 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 41295.92 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856910.606988 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371663.32 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 37166.33 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894076.939361 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334496.99 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 33449.70 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927526.638498 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301047.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 30104.73 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957631.367721 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270942.56 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 27094.26 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984725.624021 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243848.31 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 24384.83 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009110.454692 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219463.48 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 21946.35 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031056.802295 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197517.13 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 19751.71 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050808.515138 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177765.42 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 17776.54 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068585.056697 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159988.87 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 15998.89 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084583.944100 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143989.99 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 14399.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098982.942763 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129590.99 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 12959.10 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111942.041559 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116631.89 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 11663.19 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123605.230476 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104968.70 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 10496.87 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134102.100501 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94471.83 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 9447.18 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143549.283523 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85024.65 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 8502.46 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152051.748244 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76522.18 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 7652.22 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159703.966492 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68869.96 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 6887.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166590.962915 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61982.97 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 6198.30 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172789.259696 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55784.67 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 5578.47 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178367.726799 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50206.20 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 5020.62 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183388.347192 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45185.58 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 4518.56 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187906.905546 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40667.03 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 4066.70 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191973.608064 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36600.32 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3660.03 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195633.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32940.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199233.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29340.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202833.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25740.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206433.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22140.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210033.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18540.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213633.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14940.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217233.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11340.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220833.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7740.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224433.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4140.29 sec) 14:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2228033.640330 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 540.29 sec) 14:13:00 [GPUGRID] [rr_sim] 2228573.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980079762.82G/439.78G) 14:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852794.84 14:13:00 [---] pick_jobs_to_run() start 14:13:00 [---] pick_jobs_to_run() end 14:13:00 [---] [rr_sim] end 14:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:13:00 [---] [cpu_sched_debug] preliminary job list: 14:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:13:00 [---] [cpu_sched_debug] final job list: 14:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:13:00 [---] [cpu_sched_debug] enforce_run_list: end 14:13:00 [---] choose_project(): 51180.000000 14:13:00 [WUProp@Home] doing RPC to get work 14:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384419.09 cpu 2131352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131352.90 14:13:00 [---] Got 0 tasks 14:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:14:00 [---] [rr_sim] doing sim: CPU sched 14:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980052172.46G/439.78G) 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228511.19 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 222851.12 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222851.119384 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005660.07 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 200566.01 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423417.126830 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805094.07 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 180509.41 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603926.533532 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624584.66 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 162458.47 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766384.999563 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462126.19 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 146212.62 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912597.618991 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315913.57 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 131591.36 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044188.976476 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184322.22 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 118432.22 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162621.198213 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065890.00 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 106589.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269210.197776 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959301.00 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 95930.10 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365140.297383 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863370.90 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 86337.09 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451477.387029 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777033.81 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 77703.38 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529180.767711 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699330.43 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 69933.04 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599113.810324 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629397.38 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 62939.74 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662053.548676 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566457.65 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 56645.76 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718699.313193 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509811.88 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 50981.19 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769680.501258 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458830.69 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 45883.07 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815563.570517 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412947.62 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 41294.76 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856858.332849 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371652.86 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 37165.29 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1894023.618949 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334487.57 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 33448.76 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927472.376438 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301038.82 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 30103.88 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957576.258179 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270934.94 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 27093.49 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984669.751745 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243841.44 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 24384.14 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009053.895955 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219457.30 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 21945.73 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030999.625744 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197511.57 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 19751.16 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050750.782554 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177760.41 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 17776.04 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068526.823683 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159984.37 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 15998.44 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084525.260699 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143985.93 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 14398.59 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098923.854014 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129587.34 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 12958.73 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111882.587997 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116628.61 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 11662.86 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123545.448581 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104965.75 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 10496.57 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134042.023108 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94469.17 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 9446.92 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143488.940181 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85022.25 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 8502.23 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151991.165548 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76520.03 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 7652.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159643.168377 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68868.03 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 6886.80 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166529.970924 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61981.22 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 6198.12 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172728.093216 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55783.10 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 5578.31 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178306.403279 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50204.79 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 5020.48 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183326.882335 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45184.31 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 4518.43 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187845.313486 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40665.88 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 4066.59 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191911.901522 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36599.29 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3659.93 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195571.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32939.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199171.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29339.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202771.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25739.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206371.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22139.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209971.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18539.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213571.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14939.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217171.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11339.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220771.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7739.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224371.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4139.36 sec) 14:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227971.830754 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 539.36 sec) 14:14:00 [GPUGRID] [rr_sim] 2228511.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980052172.46G/439.78G) 14:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852792.11 14:14:00 [---] pick_jobs_to_run() start 14:14:00 [---] pick_jobs_to_run() end 14:14:00 [---] [rr_sim] end 14:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:14:00 [---] [cpu_sched_debug] preliminary job list: 14:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:14:00 [---] [cpu_sched_debug] final job list: 14:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:14:00 [---] [cpu_sched_debug] enforce_run_list: end 14:14:00 [---] choose_project(): 51240.000000 14:14:00 [WUProp@Home] doing RPC to get work 14:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384359.09 cpu 2131292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131292.90 14:14:00 [---] Got 0 tasks 14:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:15:00 [---] [rr_sim] doing sim: CPU sched 14:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (980024582.10G/439.78G) 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228448.46 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 222844.85 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222844.845696 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005603.61 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 200560.36 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423405.206823 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1805043.25 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 180504.33 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603909.531837 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624538.93 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 162453.89 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766363.424349 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462085.03 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 146208.50 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912571.927610 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315876.53 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 131587.65 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044159.580546 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184288.88 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 118428.89 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162588.468187 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065859.99 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 106586.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269174.467065 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959273.99 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 95927.40 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365101.866054 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863346.59 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 86334.66 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451436.525145 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 777011.93 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 77701.19 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529137.718327 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699310.74 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 69931.07 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599068.792190 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629379.66 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 62937.97 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1662006.758668 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566441.70 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 56644.17 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718650.928497 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509797.53 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 50979.75 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769630.681343 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458817.78 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 45881.78 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815512.458905 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412936.00 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 41293.60 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856806.058711 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371642.40 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 37164.24 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893970.298536 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334478.16 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 33447.82 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927418.114379 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301030.34 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 30103.03 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957521.148637 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270927.31 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 27092.73 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984613.879469 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243834.58 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 24383.46 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008997.337219 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219451.12 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 21945.11 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030942.449193 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197506.01 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 19750.60 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050693.049970 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177755.41 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 17775.54 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068468.590669 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159979.87 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 15997.99 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084466.577298 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143981.88 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 14398.19 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098864.765265 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129583.69 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 12958.37 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111823.134434 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116625.32 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 11662.53 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123485.666687 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104962.79 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 10496.28 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133981.945715 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94466.51 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 9446.65 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143428.596839 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85019.86 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 8501.99 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151930.582852 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76517.87 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 7651.79 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159582.370263 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68866.09 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 6886.61 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166468.978933 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61979.48 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 6197.95 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172666.926736 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55781.53 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 5578.15 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178245.079758 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50203.38 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 5020.34 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183265.417479 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45183.04 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 4518.30 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187783.721427 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40664.74 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 4066.47 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191850.194980 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36598.26 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3659.83 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195510.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32938.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199110.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29338.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202710.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25738.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206310.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22138.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209910.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18538.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213510.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14938.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217110.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11338.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220710.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7738.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224310.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4138.44 sec) 14:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227910.021179 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 538.44 sec) 14:15:00 [GPUGRID] [rr_sim] 2228448.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (980024582.10G/439.78G) 14:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852789.37 14:15:00 [---] pick_jobs_to_run() start 14:15:00 [---] pick_jobs_to_run() end 14:15:00 [---] [rr_sim] end 14:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:15:00 [---] [cpu_sched_debug] preliminary job list: 14:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:15:00 [---] [cpu_sched_debug] final job list: 14:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:15:00 [---] [cpu_sched_debug] enforce_run_list: end 14:15:00 [---] choose_project(): 51300.000000 14:15:00 [WUProp@Home] doing RPC to get work 14:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384299.09 cpu 2131232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131232.90 14:15:00 [---] Got 0 tasks 14:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:16:00 [---] [rr_sim] doing sim: CPU sched 14:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979996991.75G/439.78G) 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228385.72 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 222838.57 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222838.572008 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005547.15 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 200554.71 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423393.286815 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804992.43 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 180499.24 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603892.530142 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624493.19 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 162449.32 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766341.849135 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462043.87 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 146204.39 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912546.236230 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315839.48 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 131583.95 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044130.184615 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184255.54 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 118425.55 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162555.738161 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065829.98 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 106583.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269138.736353 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959246.98 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 95924.70 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365063.434726 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863322.29 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 86332.23 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451395.663261 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776990.06 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 77699.01 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529094.668943 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699291.05 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 69929.11 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1599023.774057 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629361.95 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 62936.19 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661959.968659 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566425.75 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 56642.58 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718602.543801 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509783.18 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 50978.32 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769580.861429 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458804.86 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 45880.49 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815461.347294 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412924.37 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 41292.44 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856753.784573 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371631.94 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 37163.19 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893916.978123 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334468.74 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 33446.87 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927363.852319 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301021.87 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 30102.19 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957466.039095 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270919.68 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 27091.97 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984558.007194 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243827.71 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 24382.77 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008940.778482 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219444.94 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 21944.49 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030885.272642 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197500.45 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 19750.04 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050635.317386 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177750.40 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 17775.04 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068410.357655 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159975.36 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 15997.54 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084407.893898 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143977.83 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 14397.78 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098805.676516 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129580.04 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 12958.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111763.680872 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116622.04 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 11662.20 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123425.884793 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104959.84 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 10495.98 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133921.868322 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94463.85 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 9446.39 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143368.253497 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85017.47 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 8501.75 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151870.000156 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76515.72 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 7651.57 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159521.572148 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68864.15 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 6886.41 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166407.986941 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61977.73 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 6197.77 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172605.760255 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55779.96 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 5578.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178183.756238 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50201.96 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 5020.20 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183203.952622 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45181.77 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 4518.18 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187722.129368 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40663.59 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 4066.36 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191788.488439 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36597.23 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3659.72 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195448.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32937.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199048.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29337.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202648.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25737.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206248.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22137.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209848.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18537.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213448.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14937.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217048.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11337.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220648.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7737.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224248.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4137.51 sec) 14:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227848.211603 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 537.51 sec) 14:16:00 [GPUGRID] [rr_sim] 2228385.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979996991.75G/439.78G) 14:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852786.63 14:16:00 [---] pick_jobs_to_run() start 14:16:00 [---] pick_jobs_to_run() end 14:16:00 [---] [rr_sim] end 14:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:16:00 [---] [cpu_sched_debug] preliminary job list: 14:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:16:00 [---] [cpu_sched_debug] final job list: 14:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:16:00 [---] [cpu_sched_debug] enforce_run_list: end 14:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:17:00 [---] [rr_sim] doing sim: CPU sched 14:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979969401.39G/439.78G) 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228322.98 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 222832.30 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222832.298320 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005490.68 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 200549.07 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423381.366808 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804941.62 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 180494.16 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603875.528447 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624447.45 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 162444.75 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766320.273922 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1462002.71 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 146200.27 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912520.544849 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315802.44 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 131580.24 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044100.788684 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184222.19 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 118422.22 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162523.008135 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065799.98 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 106580.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269103.005642 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959219.98 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 95922.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1365025.003397 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863297.98 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 86329.80 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451354.801377 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776968.18 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 77696.82 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529051.619559 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699271.36 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 69927.14 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598978.755923 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629344.23 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 62934.42 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661913.178651 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566409.80 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 56640.98 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718554.159105 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509768.82 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 50976.88 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769531.041515 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458791.94 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 45879.19 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815410.235683 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412912.75 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 41291.27 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856701.510434 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371621.47 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 37162.15 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893863.657711 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334459.33 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 33445.93 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927309.590259 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301013.39 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 30101.34 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957410.929553 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270912.05 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 27091.21 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984502.134918 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243820.85 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 24382.08 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008884.219746 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219438.76 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 21943.88 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030828.096091 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197494.89 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 19749.49 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050577.584801 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177745.40 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 17774.54 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068352.124641 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159970.86 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 15997.09 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084349.210497 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143973.77 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 14397.38 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098746.587767 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129576.40 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 12957.64 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111704.227310 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116618.76 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 11661.88 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123366.102899 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104956.88 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 10495.69 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133861.790929 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94461.19 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 9446.12 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143307.910155 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85015.07 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 8501.51 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151809.417460 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76513.57 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 7651.36 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159460.774033 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68862.21 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 6886.22 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166346.994950 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61975.99 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 6197.60 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172544.593775 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55778.39 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 5577.84 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178122.432717 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50200.55 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 5020.06 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183142.487765 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45180.50 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 4518.05 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187660.537308 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40662.45 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 4066.24 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191726.781897 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36596.20 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3659.62 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195386.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32936.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198986.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29336.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202586.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25736.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206186.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22136.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209786.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18536.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213386.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14936.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216986.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11336.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220586.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7736.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224186.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4136.58 sec) 14:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227786.402027 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 536.58 sec) 14:17:00 [GPUGRID] [rr_sim] 2228322.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979969401.39G/439.78G) 14:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852783.90 14:17:00 [---] pick_jobs_to_run() start 14:17:00 [---] pick_jobs_to_run() end 14:17:00 [---] [rr_sim] end 14:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:17:00 [---] [cpu_sched_debug] preliminary job list: 14:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:17:00 [---] [cpu_sched_debug] final job list: 14:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:17:00 [---] [cpu_sched_debug] enforce_run_list: end 14:17:00 [---] choose_project(): 51420.000000 14:17:00 [WUProp@Home] doing RPC to get work 14:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384179.09 cpu 2131112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131112.90 14:17:00 [---] Got 0 tasks 14:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:18:00 [---] [rr_sim] doing sim: CPU sched 14:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979941811.03G/439.78G) 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228260.25 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 222826.02 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222826.024632 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005434.22 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 200543.42 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423369.446800 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804890.80 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 180489.08 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603858.526751 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624401.72 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 162440.17 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766298.698708 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461961.55 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 146196.15 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912494.853469 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315765.39 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 131576.54 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044071.392753 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184188.85 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 118418.89 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162490.278110 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065769.97 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 106577.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269067.274930 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959192.97 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 95919.30 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364986.572069 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863273.67 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 86327.37 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451313.939493 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776946.31 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 77694.63 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1529008.570176 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699251.68 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 69925.17 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598933.737790 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629326.51 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 62932.65 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661866.388642 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566393.86 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 56639.39 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718505.774409 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509754.47 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 50975.45 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769481.221600 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458779.02 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 45877.90 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815359.124072 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412901.12 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 41290.11 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856649.236296 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371611.01 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 37161.10 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893810.337298 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334449.91 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 33444.99 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927255.328200 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301004.92 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 30100.49 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957355.820011 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270904.43 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 27090.44 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984446.262642 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243813.98 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 24381.40 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008827.661009 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219432.59 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 21943.26 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030770.919540 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197489.33 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 19748.93 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050519.852217 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177740.39 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 17774.04 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068293.891627 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159966.35 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 15996.64 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084290.527096 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143969.72 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 14396.97 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098687.499018 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129572.75 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 12957.27 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111644.773748 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116615.47 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 11661.55 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123306.321004 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104953.93 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 10495.39 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133801.713535 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94458.53 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 9445.85 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143247.566813 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85012.68 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 8501.27 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151748.834764 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76511.41 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 7651.14 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159399.975919 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68860.27 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 6886.03 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166286.002959 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61974.24 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 6197.42 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172483.427294 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55776.82 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 5577.68 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178061.109196 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50199.14 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 5019.91 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183081.022908 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45179.22 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 4517.92 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187598.945249 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40661.30 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 4066.13 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191665.075356 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36595.17 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3659.52 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195324.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32935.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198924.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29335.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202524.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25735.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206124.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22135.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209724.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18535.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213324.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14935.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216924.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11335.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220524.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7735.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224124.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4135.65 sec) 14:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227724.592452 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 535.65 sec) 14:18:00 [GPUGRID] [rr_sim] 2228260.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979941811.03G/439.78G) 14:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852781.16 14:18:00 [---] pick_jobs_to_run() start 14:18:00 [---] pick_jobs_to_run() end 14:18:00 [---] [rr_sim] end 14:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:18:00 [---] [cpu_sched_debug] preliminary job list: 14:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:18:00 [---] [cpu_sched_debug] final job list: 14:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:18:00 [---] [cpu_sched_debug] enforce_run_list: end 14:18:00 [---] choose_project(): 51480.000000 14:18:00 [WUProp@Home] doing RPC to get work 14:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384119.09 cpu 2131052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2131052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2131052.90 14:18:00 [---] Got 0 tasks 14:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:19:00 [---] [rr_sim] doing sim: CPU sched 14:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979914220.68G/439.78G) 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228197.51 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 222819.75 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222819.750943 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005377.76 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 200537.78 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423357.526792 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804839.98 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 180484.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603841.525056 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624355.98 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 162435.60 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766277.123494 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461920.39 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 146192.04 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912469.162088 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315728.35 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 131572.83 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044041.996823 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184155.51 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 118415.55 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162457.548084 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065739.96 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 106574.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1269031.544219 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959165.97 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 95916.60 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364948.140740 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863249.37 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 86324.94 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451273.077609 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776924.43 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 77692.44 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528965.520792 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699231.99 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 69923.20 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598888.719656 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629308.79 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 62930.88 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661819.598634 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566377.91 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 56637.79 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718457.389714 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509740.12 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 50974.01 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769431.401686 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458766.11 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 45876.61 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815308.012460 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412889.50 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 41288.95 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856596.962158 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371600.55 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 37160.05 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893757.016885 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334440.49 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 33444.05 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927201.066140 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300996.44 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 30099.64 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957300.710469 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270896.80 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 27089.68 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984390.390366 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243807.12 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 24380.71 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008771.102272 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219426.41 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 21942.64 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030713.742989 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197483.77 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 19748.38 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050462.119633 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177735.39 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 17773.54 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068235.658613 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159961.85 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 15996.19 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084231.843695 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143965.67 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 14396.57 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098628.410269 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129569.10 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 12956.91 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111585.320185 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116612.19 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 11661.22 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123246.539110 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104950.97 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 10495.10 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133741.636142 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94455.87 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 9445.59 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143187.223472 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85010.29 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 8501.03 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151688.252068 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76509.26 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 7650.93 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159339.177804 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68858.33 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 6885.83 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166225.010967 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61972.50 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 6197.25 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172422.260814 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55775.25 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 5577.52 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177999.785676 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50197.72 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 5019.77 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183019.558051 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45177.95 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 4517.80 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187537.353190 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40660.16 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 4066.02 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191603.368814 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36594.14 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3659.41 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195262.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32934.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198862.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29334.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202462.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25734.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206062.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22134.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209662.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18534.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213262.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14934.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216862.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11334.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220462.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7734.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224062.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4134.73 sec) 14:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227662.782876 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 534.73 sec) 14:19:00 [GPUGRID] [rr_sim] 2228197.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979914220.68G/439.78G) 14:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852778.42 14:19:00 [---] pick_jobs_to_run() start 14:19:00 [---] pick_jobs_to_run() end 14:19:00 [---] [rr_sim] end 14:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:19:00 [---] [cpu_sched_debug] preliminary job list: 14:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:19:00 [---] [cpu_sched_debug] final job list: 14:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:19:00 [---] [cpu_sched_debug] enforce_run_list: end 14:19:00 [---] choose_project(): 51540.000000 14:19:00 [WUProp@Home] doing RPC to get work 14:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 384059.09 cpu 2130992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130992.90 14:19:00 [---] Got 0 tasks 14:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:20:00 [---] [rr_sim] doing sim: CPU sched 14:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979886630.32G/439.78G) 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228134.77 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 222813.48 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222813.477255 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005321.30 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 200532.13 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423345.606785 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804789.17 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 180478.92 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603824.523361 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624310.25 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 162431.02 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766255.548280 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461879.22 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 146187.92 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912443.470707 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315691.30 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 131569.13 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1044012.600892 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184122.17 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 118412.22 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162424.818058 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065709.95 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 106571.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268995.813507 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959138.96 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 95913.90 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364909.709411 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863225.06 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 86322.51 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451232.215725 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776902.56 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 77690.26 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528922.471408 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699212.30 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 69921.23 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598843.701522 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629291.07 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 62929.11 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661772.808625 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566361.96 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 56636.20 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718409.005018 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509725.77 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 50972.58 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769381.581771 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458753.19 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 45875.32 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815256.900849 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412877.87 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 41287.79 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856544.688019 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371590.08 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 37159.01 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893703.696472 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334431.08 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 33443.11 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927146.804080 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300987.97 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 30098.80 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957245.600927 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270889.17 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 27088.92 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984334.518090 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243800.25 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 24380.03 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008714.543536 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219420.23 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 21942.02 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030656.566437 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197478.21 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 19747.82 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050404.387049 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177730.39 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 17773.04 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068177.425599 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159957.35 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 15995.73 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084173.160294 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143961.61 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 14396.16 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098569.321520 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129565.45 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 12956.55 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111525.866623 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116608.91 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 11660.89 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123186.757216 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104948.02 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 10494.80 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133681.558749 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94453.21 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 9445.32 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143126.880130 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85007.89 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 8500.79 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151627.669372 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76507.10 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 7650.71 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159278.379690 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68856.39 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 6885.64 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166164.018976 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61970.75 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 6197.08 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172361.094333 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55773.68 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 5577.37 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177938.462155 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50196.31 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 5019.63 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182958.093195 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45176.68 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 4517.67 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187475.761130 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40659.01 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 4065.90 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191541.662272 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36593.11 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3659.31 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195200.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32933.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198800.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29333.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202400.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25733.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206000.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22133.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209600.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18533.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213200.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14933.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216800.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11333.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220400.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7733.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224000.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4133.80 sec) 14:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227600.973300 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 533.80 sec) 14:20:00 [GPUGRID] [rr_sim] 2228134.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979886630.32G/439.78G) 14:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852775.69 14:20:00 [---] pick_jobs_to_run() start 14:20:00 [---] pick_jobs_to_run() end 14:20:00 [---] [rr_sim] end 14:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:20:00 [---] [cpu_sched_debug] preliminary job list: 14:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:20:00 [---] [cpu_sched_debug] final job list: 14:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:20:00 [---] [cpu_sched_debug] enforce_run_list: end 14:20:00 [---] choose_project(): 51600.000000 14:20:00 [WUProp@Home] doing RPC to get work 14:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383999.09 cpu 2130932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130932.90 14:20:00 [---] Got 0 tasks 14:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:21:00 [---] [rr_sim] doing sim: CPU sched 14:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979859039.97G/439.78G) 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228072.04 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 222807.20 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222807.203567 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005264.83 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 200526.48 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423333.686777 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804738.35 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 180473.83 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603807.521666 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624264.51 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 162426.45 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766233.973067 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461838.06 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 146183.81 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912417.779327 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315654.26 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 131565.43 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043983.204961 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184088.83 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 118408.88 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162392.088032 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065679.95 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 106567.99 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268960.082796 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959111.95 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 95911.20 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364871.278083 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863200.76 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 86320.08 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451191.353841 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776880.68 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 77688.07 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528879.422024 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699192.61 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 69919.26 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598798.683389 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629273.35 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 62927.34 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661726.018617 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566346.02 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 56634.60 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718360.620322 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509711.42 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 50971.14 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769331.761857 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458740.27 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 45874.03 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815205.789238 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412866.25 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 41286.62 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856492.413881 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371579.62 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 37157.96 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893650.376060 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334421.66 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 33442.17 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927092.542021 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300979.49 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 30097.95 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957190.491386 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270881.54 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 27088.15 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984278.645814 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243793.39 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 24379.34 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008657.984799 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219414.05 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 21941.41 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030599.389886 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197472.65 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 19747.26 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050346.654465 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177725.38 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 17772.54 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068119.192585 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159952.84 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 15995.28 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084114.476893 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143957.56 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 14395.76 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098510.232771 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129561.80 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 12956.18 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111466.413061 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116605.62 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 11660.56 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123126.975322 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104945.06 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 10494.51 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133621.481356 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94450.55 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 9445.06 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143066.536788 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85005.50 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 8500.55 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151567.086676 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76504.95 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 7650.49 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159217.581575 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68854.45 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 6885.45 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166103.026984 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61969.01 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 6196.90 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172299.927853 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55772.11 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 5577.21 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177877.138634 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50194.90 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 5019.49 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182896.628338 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45175.41 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 4517.54 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187414.169071 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40657.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 4065.79 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191479.955731 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36592.08 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3659.21 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195139.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32932.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198739.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29332.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202339.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25732.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205939.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22132.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209539.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18532.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213139.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14932.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216739.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11332.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220339.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7732.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223939.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4132.87 sec) 14:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227539.163725 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 532.87 sec) 14:21:00 [GPUGRID] [rr_sim] 2228072.04: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979859039.97G/439.78G) 14:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852772.95 14:21:00 [---] pick_jobs_to_run() start 14:21:00 [---] pick_jobs_to_run() end 14:21:00 [---] [rr_sim] end 14:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:21:00 [---] [cpu_sched_debug] preliminary job list: 14:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:21:00 [---] [cpu_sched_debug] final job list: 14:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:21:00 [---] [cpu_sched_debug] enforce_run_list: end 14:21:00 [---] choose_project(): 51660.000000 14:21:00 [WUProp@Home] doing RPC to get work 14:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383939.09 cpu 2130872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130872.90 14:21:00 [---] Got 0 tasks 14:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:22:00 [---] [rr_sim] doing sim: CPU sched 14:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979831449.61G/439.78G) 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2228009.30 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 222800.93 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222800.929879 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005208.37 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 200520.84 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423321.766769 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804687.53 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 180468.75 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603790.519971 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624218.78 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 162421.88 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766212.397853 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461796.90 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 146179.69 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912392.087946 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315617.21 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 131561.72 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043953.809030 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184055.49 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 118405.55 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162359.358006 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065649.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 106564.99 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268924.352084 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959084.95 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 95908.49 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364832.846754 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863176.45 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 86317.65 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451150.491958 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776858.81 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 77685.88 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528836.372640 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699172.93 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 69917.29 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598753.665255 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629255.63 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 62925.56 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661679.228608 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566330.07 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 56633.01 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718312.235626 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509697.06 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 50969.71 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769281.941942 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458727.36 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 45872.74 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815154.677627 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412854.62 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 41285.46 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856440.139743 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371569.16 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 37156.92 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893597.055647 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334412.24 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 33441.22 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927038.279961 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300971.02 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 30097.10 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957135.381844 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270873.92 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 27087.39 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984222.773538 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243786.53 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 24378.65 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008601.426063 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219407.87 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 21940.79 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030542.213335 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197467.09 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 19746.71 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050288.921880 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177720.38 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 17772.04 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068060.959571 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159948.34 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 15994.83 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084055.793493 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143953.51 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 14395.35 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098451.144022 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129558.15 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 12955.82 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111406.959498 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116602.34 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 11660.23 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123067.193427 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104942.11 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 10494.21 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133561.403963 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94447.89 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 9444.79 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143006.193446 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85003.11 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 8500.31 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151506.503980 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76502.79 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 7650.28 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159156.783460 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68852.52 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 6885.25 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166042.034993 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61967.26 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 6196.73 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172238.761372 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55770.54 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 5577.05 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177815.815114 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50193.48 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 5019.35 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182835.163481 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45174.14 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 4517.41 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187352.577012 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40656.72 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 4065.67 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191418.249189 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36591.05 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3659.10 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195077.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32931.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198677.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29331.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202277.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25731.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205877.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22131.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209477.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18531.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213077.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14931.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216677.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11331.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220277.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7731.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223877.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4131.94 sec) 14:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227477.354149 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 531.94 sec) 14:22:00 [GPUGRID] [rr_sim] 2228009.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979831449.61G/439.78G) 14:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852770.21 14:22:00 [---] pick_jobs_to_run() start 14:22:00 [---] pick_jobs_to_run() end 14:22:00 [---] [rr_sim] end 14:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:22:00 [---] [cpu_sched_debug] preliminary job list: 14:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:22:00 [---] [cpu_sched_debug] final job list: 14:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:22:00 [---] [cpu_sched_debug] enforce_run_list: end 14:22:00 [---] choose_project(): 51720.000000 14:22:00 [WUProp@Home] doing RPC to get work 14:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383879.09 cpu 2130812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130812.90 14:22:00 [---] Got 0 tasks 14:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:23:00 [---] [rr_sim] doing sim: CPU sched 14:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979803859.25G/439.78G) 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227946.56 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 222794.66 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222794.656190 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005151.91 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 200515.19 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423309.846762 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804636.72 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 180463.67 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603773.518276 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624173.04 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 162417.30 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766190.822639 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461755.74 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 146175.57 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912366.396566 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315580.17 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 131558.02 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043924.413099 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1184022.15 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 118402.21 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162326.627980 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065619.93 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 106561.99 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268888.621372 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959057.94 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 95905.79 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364794.415426 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863152.15 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 86315.21 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451109.630074 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776836.93 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 77683.69 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528793.323257 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699153.24 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 69915.32 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598708.647121 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629237.91 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 62923.79 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661632.438600 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566314.12 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 56631.41 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718263.850930 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509682.71 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 50968.27 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769232.122028 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458714.44 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 45871.44 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815103.566015 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412843.00 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 41284.30 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856387.865604 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371558.70 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 37155.87 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893543.735234 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334402.83 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 33440.28 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926984.017901 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300962.54 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 30096.25 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957080.272302 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270866.29 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 27086.63 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984166.901262 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243779.66 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 24377.97 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008544.867326 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219401.69 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 21940.17 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030485.036784 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197461.53 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 19746.15 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050231.189296 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177715.37 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 17771.54 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068002.726557 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159943.84 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 15994.38 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083997.110092 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143949.45 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 14394.95 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098392.055273 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129554.51 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 12955.45 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111347.505936 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116599.06 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 11659.91 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123007.411533 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104939.15 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 10493.92 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133501.326570 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94445.24 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 9444.52 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142945.850104 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85000.71 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 8500.07 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151445.921284 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76500.64 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 7650.06 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159095.985346 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68850.58 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 6885.06 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165981.043002 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61965.52 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 6196.55 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172177.594892 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55768.97 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 5576.90 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177754.491593 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50192.07 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 5019.21 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182773.698624 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45172.86 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 4517.29 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187290.984952 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40655.58 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 4065.56 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191356.542648 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36590.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3659.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195015.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32931.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198615.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29331.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202215.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25731.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205815.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22131.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209415.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18531.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213015.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14931.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216615.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11331.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220215.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7731.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223815.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4131.02 sec) 14:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227415.544573 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 531.02 sec) 14:23:00 [GPUGRID] [rr_sim] 2227946.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979803859.25G/439.78G) 14:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852767.47 14:23:00 [---] pick_jobs_to_run() start 14:23:00 [---] pick_jobs_to_run() end 14:23:00 [---] [rr_sim] end 14:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:23:00 [---] [cpu_sched_debug] preliminary job list: 14:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:23:00 [---] [cpu_sched_debug] final job list: 14:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:23:00 [---] [cpu_sched_debug] enforce_run_list: end 14:23:00 [---] choose_project(): 51780.000000 14:23:00 [WUProp@Home] doing RPC to get work 14:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383819.09 cpu 2130752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130752.90 14:23:00 [---] Got 0 tasks 14:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:24:00 [---] [rr_sim] doing sim: CPU sched 14:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979776268.90G/439.78G) 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227883.83 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 222788.38 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222788.382502 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005095.44 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 200509.54 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423297.926754 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804585.90 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 180458.59 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603756.516581 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624127.31 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 162412.73 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766169.247425 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461714.58 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 146171.46 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912340.705185 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315543.12 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 131554.31 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043895.017169 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183988.81 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 118398.88 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162293.897954 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065589.93 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 106558.99 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268852.890661 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959030.93 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 95903.09 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364755.984097 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863127.84 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 86312.78 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451068.768190 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776815.06 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 77681.51 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528750.273873 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699133.55 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 69913.36 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598663.628988 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629220.20 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 62922.02 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661585.648591 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566298.18 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 56629.82 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718215.466234 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509668.36 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 50966.84 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769182.302113 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458701.52 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 45870.15 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815052.454404 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412831.37 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 41283.14 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856335.591466 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371548.23 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 37154.82 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893490.414822 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334393.41 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 33439.34 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926929.755842 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300954.07 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 30095.41 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957025.162760 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270858.66 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 27085.87 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984111.028986 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243772.80 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 24377.28 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008488.308590 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219395.52 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 21939.55 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030427.860233 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197455.96 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 19745.60 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050173.456712 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177710.37 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 17771.04 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067944.493543 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159939.33 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 15993.93 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083938.426691 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143945.40 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 14394.54 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098332.966524 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129550.86 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 12955.09 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111288.052374 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116595.77 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 11659.58 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122947.629639 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104936.20 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 10493.62 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133441.249177 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94442.58 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 9444.26 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142885.506762 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84998.32 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 8499.83 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151385.338588 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76498.49 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 7649.85 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159035.187231 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68848.64 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 6884.86 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165920.051010 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61963.77 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 6196.38 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172116.428412 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55767.40 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 5576.74 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177693.168073 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50190.66 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 5019.07 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182712.233768 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45171.59 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 4517.16 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187229.392893 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40654.43 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 4065.44 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191294.836106 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36588.99 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3658.90 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194953.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32930.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198553.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29330.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202153.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25730.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205753.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22130.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209353.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18530.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212953.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14930.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216553.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11330.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220153.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7730.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223753.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4130.09 sec) 14:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227353.734998 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 530.09 sec) 14:24:00 [GPUGRID] [rr_sim] 2227883.83: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979776268.90G/439.78G) 14:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852764.74 14:24:00 [---] pick_jobs_to_run() start 14:24:00 [---] pick_jobs_to_run() end 14:24:00 [---] [rr_sim] end 14:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:24:00 [---] [cpu_sched_debug] preliminary job list: 14:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:24:00 [---] [cpu_sched_debug] final job list: 14:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:24:00 [---] [cpu_sched_debug] enforce_run_list: end 14:24:00 [---] choose_project(): 51840.000000 14:24:00 [WUProp@Home] doing RPC to get work 14:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383759.09 cpu 2130692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130692.90 14:24:00 [---] Got 0 tasks 14:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:25:00 [---] [rr_sim] doing sim: CPU sched 14:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979748678.54G/439.78G) 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227821.09 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 222782.11 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222782.108814 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2005038.98 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 200503.90 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423286.006747 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804535.08 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 180453.51 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603739.514886 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624081.57 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 162408.16 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766147.672211 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461673.42 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 146167.34 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912315.013804 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315506.07 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 131550.61 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043865.621238 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183955.47 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 118395.55 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162261.167928 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065559.92 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 106555.99 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268817.159949 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 959003.93 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 95900.39 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364717.552768 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863103.54 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 86310.35 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1451027.906306 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776793.18 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 77679.32 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528707.224489 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699113.86 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 69911.39 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598618.610854 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629202.48 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 62920.25 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661538.858583 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566282.23 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 56628.22 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718167.081539 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509654.01 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 50965.40 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769132.482199 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458688.61 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 45868.86 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1815001.342793 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412819.75 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 41281.97 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856283.317328 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371537.77 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 37153.78 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893437.094409 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334383.99 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 33438.40 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926875.493782 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300945.59 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 30094.56 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956970.053218 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270851.03 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 27085.10 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984055.156710 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243765.93 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 24376.59 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008431.749853 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219389.34 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 21938.93 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030370.683682 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197450.40 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 19745.04 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050115.724128 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177705.36 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 17770.54 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067886.260529 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159934.83 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 15993.48 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083879.743290 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143941.34 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 14394.13 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098273.877775 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129547.21 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 12954.72 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111228.598812 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116592.49 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 11659.25 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122887.847744 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104933.24 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 10493.32 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133381.171784 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94439.92 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 9443.99 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142825.163420 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84995.92 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 8499.59 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151324.755892 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76496.33 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 7649.63 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158974.389117 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68846.70 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 6884.67 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165859.059019 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61962.03 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 6196.20 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172055.261931 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55765.83 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 5576.58 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177631.844552 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50189.24 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 5018.92 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182650.768911 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45170.32 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 4517.03 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187167.800834 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40653.29 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 4065.33 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191233.129564 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36587.96 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3658.80 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194891.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32929.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198491.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29329.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202091.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25729.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205691.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22129.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209291.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18529.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212891.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14929.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216491.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11329.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220091.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7729.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223691.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4129.16 sec) 14:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227291.925422 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 529.16 sec) 14:25:00 [GPUGRID] [rr_sim] 2227821.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979748678.54G/439.78G) 14:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852762.00 14:25:00 [---] pick_jobs_to_run() start 14:25:00 [---] pick_jobs_to_run() end 14:25:00 [---] [rr_sim] end 14:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:25:00 [---] [cpu_sched_debug] preliminary job list: 14:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:25:00 [---] [cpu_sched_debug] final job list: 14:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:25:00 [---] [cpu_sched_debug] enforce_run_list: end 14:25:00 [---] choose_project(): 51900.000000 14:25:00 [WUProp@Home] doing RPC to get work 14:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383699.09 cpu 2130632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130632.90 14:25:00 [---] Got 0 tasks 14:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:26:00 [---] [rr_sim] doing sim: CPU sched 14:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979721088.18G/439.78G) 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227758.35 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 222775.84 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222775.835126 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004982.52 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 200498.25 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423274.086739 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804484.26 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 180448.43 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603722.513191 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1624035.84 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 162403.58 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766126.096998 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461632.25 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 146163.23 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912289.322424 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315469.03 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 131546.90 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043836.225307 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183922.13 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 118392.21 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162228.437902 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065529.91 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 106552.99 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268781.429238 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958976.92 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 95897.69 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364679.121440 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863079.23 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 86307.92 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450987.044422 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776771.31 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 77677.13 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528664.175105 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699094.18 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 69909.42 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598573.592721 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629184.76 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 62918.48 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661492.068574 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566266.28 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 56626.63 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718118.696843 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509639.65 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 50963.97 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769082.662284 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458675.69 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 45867.57 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814950.231182 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412808.12 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 41280.81 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856231.043189 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371527.31 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 37152.73 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893383.773996 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334374.58 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 33437.46 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926821.231722 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300937.12 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 30093.71 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956914.943676 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270843.41 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 27084.34 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983999.284434 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243759.07 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 24375.91 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008375.191117 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219383.16 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 21938.32 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030313.507131 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197444.84 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 19744.48 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050057.991543 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177700.36 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 17770.04 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067828.027515 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159930.32 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 15993.03 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083821.059889 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143937.29 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 14393.73 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098214.789026 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129543.56 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 12954.36 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111169.145249 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116589.21 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 11658.92 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122828.065850 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104930.29 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 10493.03 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133321.094391 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94437.26 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 9443.73 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142764.820078 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84993.53 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 8499.35 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151264.173196 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76494.18 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 7649.42 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158913.591002 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68844.76 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 6884.48 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165798.067028 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61960.28 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 6196.03 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171994.095451 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55764.26 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 5576.43 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177570.521031 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50187.83 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 5018.78 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182589.304054 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45169.05 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 4516.90 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187106.208774 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40652.14 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 4065.21 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191171.423023 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36586.93 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3658.69 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194830.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32928.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198430.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29328.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202030.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25728.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205630.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22128.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209230.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18528.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212830.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14928.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216430.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11328.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220030.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7728.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223630.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4128.24 sec) 14:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227230.115846 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 528.24 sec) 14:26:00 [GPUGRID] [rr_sim] 2227758.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979721088.18G/439.78G) 14:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852759.26 14:26:00 [---] pick_jobs_to_run() start 14:26:00 [---] pick_jobs_to_run() end 14:26:00 [---] [rr_sim] end 14:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:26:00 [---] [cpu_sched_debug] preliminary job list: 14:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:26:00 [---] [cpu_sched_debug] final job list: 14:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:26:00 [---] [cpu_sched_debug] enforce_run_list: end 14:26:00 [---] choose_project(): 51960.000000 14:26:00 [WUProp@Home] doing RPC to get work 14:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383639.09 cpu 2130572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130572.90 14:26:00 [---] Got 0 tasks 14:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:27:00 [---] [rr_sim] doing sim: CPU sched 14:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979693497.83G/439.78G) 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227695.61 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 222769.56 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222769.561438 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004926.05 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 200492.61 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423262.166731 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804433.45 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 180443.34 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603705.511496 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623990.10 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 162399.01 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766104.521784 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461591.09 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 146159.11 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912263.631043 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315431.98 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 131543.20 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043806.829376 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183888.78 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 118388.88 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162195.707876 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065499.91 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 106549.99 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268745.698526 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958949.92 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 95894.99 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364640.690111 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863054.92 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 86305.49 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450946.182538 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776749.43 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 77674.94 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528621.125722 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699074.49 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 69907.45 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598528.574587 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629167.04 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 62916.70 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661445.278566 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566250.34 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 56625.03 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718070.312147 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509625.30 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 50962.53 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1769032.842370 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458662.77 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 45866.28 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814899.119570 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412796.49 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 41279.65 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856178.769051 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371516.85 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 37151.68 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893330.453583 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334365.16 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 33436.52 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926766.969663 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300928.64 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 30092.86 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956859.834134 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270835.78 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 27083.58 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983943.412158 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243752.20 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 24375.22 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008318.632380 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219376.98 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 21937.70 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030256.330580 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197439.28 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 19743.93 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050000.258959 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177695.36 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 17769.54 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067769.794501 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159925.82 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 15992.58 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083762.376488 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143933.24 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 14393.32 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098155.700277 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129539.91 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 12953.99 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111109.691687 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116585.92 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 11658.59 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122768.283956 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104927.33 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 10492.73 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133261.016998 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94434.60 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 9443.46 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142704.476736 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84991.14 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 8499.11 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151203.590500 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76492.02 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 7649.20 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158852.792887 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68842.82 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 6884.28 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165737.075036 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61958.54 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 6195.85 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171932.928970 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55762.69 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 5576.27 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177509.197511 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50186.42 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 5018.64 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182527.839197 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45167.78 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 4516.78 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187044.616715 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40651.00 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 4065.10 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191109.716481 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36585.90 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3658.59 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194768.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32927.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198368.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29327.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201968.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25727.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205568.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22127.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209168.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18527.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212768.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14927.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216368.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11327.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219968.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7727.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223568.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4127.31 sec) 14:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227168.306271 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 527.31 sec) 14:27:00 [GPUGRID] [rr_sim] 2227695.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979693497.83G/439.78G) 14:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852756.53 14:27:00 [---] pick_jobs_to_run() start 14:27:00 [---] pick_jobs_to_run() end 14:27:00 [---] [rr_sim] end 14:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:27:00 [---] [cpu_sched_debug] preliminary job list: 14:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:27:00 [---] [cpu_sched_debug] final job list: 14:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:27:00 [---] [cpu_sched_debug] enforce_run_list: end 14:27:00 [---] choose_project(): 52020.000000 14:27:00 [WUProp@Home] doing RPC to get work 14:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383579.09 cpu 2130512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130512.90 14:27:00 [---] Got 0 tasks 14:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:28:00 [---] [rr_sim] doing sim: CPU sched 14:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979665907.47G/439.78G) 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227632.88 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 222763.29 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222763.287749 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004869.59 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 200486.96 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423250.246724 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804382.63 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 180438.26 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603688.509801 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623944.37 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 162394.44 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766082.946570 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461549.93 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 146154.99 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912237.939662 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315394.94 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 131539.49 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043777.433446 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183855.44 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 118385.54 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162162.977850 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065469.90 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 106546.99 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268709.967815 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958922.91 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 95892.29 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364602.258783 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863030.62 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 86303.06 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450905.320654 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776727.56 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 77672.76 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528578.076338 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699054.80 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 69905.48 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598483.556453 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629149.32 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 62914.93 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661398.488557 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566234.39 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 56623.44 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1718021.927451 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509610.95 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 50961.10 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768983.022455 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458649.86 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 45864.99 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814848.007959 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412784.87 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 41278.49 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856126.494913 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371506.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 37150.64 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893277.133171 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334355.74 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 33435.57 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926712.707603 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300920.17 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 30092.02 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956804.724592 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270828.15 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 27082.82 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983887.539882 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243745.34 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 24374.53 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008262.073643 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219370.80 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 21937.08 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030199.154028 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197433.72 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 19743.37 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049942.526375 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177690.35 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 17769.04 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067711.561487 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159921.32 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 15992.13 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083703.693088 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143929.18 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 14392.92 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098096.611528 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129536.27 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 12953.63 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111050.238125 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116582.64 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 11658.26 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122708.502062 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104924.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 10492.44 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133200.939605 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94431.94 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 9443.19 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142644.133394 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84988.74 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 8498.87 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151143.007804 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76489.87 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 7648.99 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158791.994773 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68840.88 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 6884.09 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165676.083045 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61956.79 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 6195.68 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171871.762490 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55761.12 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 5576.11 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177447.873990 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50185.00 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 5018.50 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182466.374341 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45166.50 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 4516.65 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186983.024656 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40649.85 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 4064.99 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191048.009940 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36584.87 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3658.49 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194706.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32926.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198306.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29326.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201906.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25726.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205506.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22126.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209106.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18526.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212706.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14926.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216306.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11326.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219906.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7726.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223506.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4126.38 sec) 14:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227106.496695 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 526.38 sec) 14:28:00 [GPUGRID] [rr_sim] 2227632.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979665907.47G/439.78G) 14:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852753.79 14:28:00 [---] pick_jobs_to_run() start 14:28:00 [---] pick_jobs_to_run() end 14:28:00 [---] [rr_sim] end 14:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:28:00 [---] [cpu_sched_debug] preliminary job list: 14:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:28:00 [---] [cpu_sched_debug] final job list: 14:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:28:00 [---] [cpu_sched_debug] enforce_run_list: end 14:28:00 [---] choose_project(): 52080.000000 14:28:00 [WUProp@Home] doing RPC to get work 14:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383519.09 cpu 2130452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130452.90 14:28:00 [---] Got 0 tasks 14:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:29:00 [---] [rr_sim] doing sim: CPU sched 14:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979638317.12G/439.78G) 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227570.14 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 222757.01 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222757.014061 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004813.13 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 200481.31 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423238.326716 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804331.81 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 180433.18 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603671.508106 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623898.63 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 162389.86 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766061.371356 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461508.77 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 146150.88 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912212.248282 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315357.89 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 131535.79 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043748.037515 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183822.10 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 118382.21 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162130.247825 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065439.89 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 106543.99 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268674.237103 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958895.90 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 95889.59 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364563.827454 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 863006.31 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 86300.63 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450864.458770 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776705.68 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 77670.57 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528535.026954 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699035.11 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 69903.51 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598438.538320 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629131.60 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 62913.16 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661351.698549 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566218.44 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 56621.84 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717973.542755 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509596.60 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 50959.66 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768933.202541 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458636.94 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 45863.69 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814796.896348 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412773.24 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 41277.32 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856074.220774 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371495.92 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 37149.59 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893223.812758 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334346.33 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 33434.63 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926658.445543 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300911.70 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 30091.17 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956749.615050 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270820.53 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 27082.05 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983831.667606 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243738.47 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 24373.85 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008205.514907 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219364.63 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 21936.46 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030141.977477 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197428.16 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 19742.82 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049884.793791 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177685.35 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 17768.53 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067653.328473 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159916.81 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 15991.68 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083645.009687 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143925.13 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 14392.51 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098037.522779 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129532.62 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 12953.26 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110990.784562 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116579.36 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 11657.94 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122648.720167 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104921.42 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 10492.14 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133140.862212 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94429.28 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 9442.93 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142583.790052 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84986.35 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 8498.64 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151082.425108 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76487.72 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 7648.77 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158731.196658 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68838.94 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 6883.89 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165615.091053 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61955.05 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 6195.50 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171810.596009 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55759.54 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 5575.95 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177386.550470 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50183.59 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 5018.36 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182404.909484 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45165.23 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 4516.52 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186921.432597 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40648.71 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 4064.87 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190986.303398 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36583.84 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3658.38 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194644.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32925.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198244.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29325.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201844.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25725.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205444.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22125.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209044.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18525.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212644.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14925.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216244.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11325.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219844.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7725.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223444.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4125.45 sec) 14:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2227044.687119 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 525.45 sec) 14:29:00 [GPUGRID] [rr_sim] 2227570.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979638317.12G/439.78G) 14:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852751.05 14:29:00 [---] pick_jobs_to_run() start 14:29:00 [---] pick_jobs_to_run() end 14:29:00 [---] [rr_sim] end 14:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:29:00 [---] [cpu_sched_debug] preliminary job list: 14:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:29:00 [---] [cpu_sched_debug] final job list: 14:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:29:00 [---] [cpu_sched_debug] enforce_run_list: end 14:29:00 [---] choose_project(): 52140.000000 14:29:00 [WUProp@Home] doing RPC to get work 14:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383459.09 cpu 2130392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130392.90 14:29:00 [---] Got 0 tasks 14:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:30:00 [---] [rr_sim] doing sim: CPU sched 14:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979610726.76G/439.78G) 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227507.40 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 222750.74 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222750.740373 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004756.66 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 200475.67 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423226.406709 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804281.00 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 180428.10 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603654.506411 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623852.90 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 162385.29 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766039.796143 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461467.61 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 146146.76 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912186.556901 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315320.85 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 131532.08 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043718.641584 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183788.76 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 118378.88 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162097.517799 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065409.89 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 106540.99 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268638.506392 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958868.90 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 95886.89 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364525.396125 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862982.01 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 86298.20 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450823.596886 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776683.81 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 77668.38 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528491.977570 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 699015.43 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 69901.54 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598393.520186 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629113.88 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 62911.39 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661304.908540 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566202.50 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 56620.25 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717925.158059 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509582.25 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 50958.22 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768883.382626 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458624.02 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 45862.40 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814745.784737 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412761.62 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 41276.16 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1856021.946636 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371485.46 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 37148.55 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893170.492345 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334336.91 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 33433.69 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926604.183484 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300903.22 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 30090.32 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956694.505508 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270812.90 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 27081.29 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983775.795330 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243731.61 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 24373.16 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008148.956170 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219358.45 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 21935.84 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030084.800926 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197422.60 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 19742.26 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049827.061207 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177680.34 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 17768.03 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067595.095459 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159912.31 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 15991.23 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083586.326286 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143921.08 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 14392.11 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097978.434030 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129528.97 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 12952.90 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110931.331000 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116576.07 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 11657.61 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122588.938273 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104918.47 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 10491.85 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133080.784819 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94426.62 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 9442.66 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142523.446710 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84983.96 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 8498.40 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151021.842412 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76485.56 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 7648.56 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158670.398544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68837.01 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 6883.70 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165554.099062 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61953.30 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 6195.33 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171749.429529 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55757.97 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 5575.80 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177325.226949 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50182.18 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 5018.22 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182343.444627 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45163.96 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 4516.40 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186859.840537 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40647.56 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 4064.76 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190924.596856 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36582.81 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3658.28 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194582.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32924.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198182.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29324.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201782.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25724.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205382.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22124.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208982.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18524.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212582.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14924.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216182.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11324.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219782.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7724.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223382.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4124.53 sec) 14:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226982.877544 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 524.53 sec) 14:30:00 [GPUGRID] [rr_sim] 2227507.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979610726.76G/439.78G) 14:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852748.32 14:30:00 [---] pick_jobs_to_run() start 14:30:00 [---] pick_jobs_to_run() end 14:30:00 [---] [rr_sim] end 14:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:30:00 [---] [cpu_sched_debug] preliminary job list: 14:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:30:00 [---] [cpu_sched_debug] final job list: 14:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:30:00 [---] [cpu_sched_debug] enforce_run_list: end 14:30:00 [---] choose_project(): 52200.000000 14:30:00 [WUProp@Home] doing RPC to get work 14:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383399.09 cpu 2130332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130332.90 14:30:00 [---] Got 0 tasks 14:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:31:00 [---] [rr_sim] doing sim: CPU sched 14:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979583136.40G/439.78G) 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227444.67 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 222744.47 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222744.466685 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004700.20 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 200470.02 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423214.486701 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804230.18 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 180423.02 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603637.504716 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623807.16 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 162380.72 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 766018.220929 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461426.45 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 146142.64 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912160.865521 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315283.80 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 131528.38 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043689.245653 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183755.42 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 118375.54 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162064.787773 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065379.88 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 106537.99 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268602.775680 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958841.89 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 95884.19 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364486.964797 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862957.70 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 86295.77 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450782.735002 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776661.93 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 77666.19 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528448.928186 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698995.74 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 69899.57 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598348.502053 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629096.16 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 62909.62 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661258.118532 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566186.55 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 56618.65 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717876.773364 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509567.89 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 50956.79 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768833.562712 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458611.10 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 45861.11 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814694.673125 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412749.99 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 41275.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855969.672498 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371474.99 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 37147.50 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893117.171933 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334327.49 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 33432.75 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926549.921424 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300894.75 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 30089.47 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956639.395966 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270805.27 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 27080.53 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983719.923054 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243724.74 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 24372.47 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008092.397434 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219352.27 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 21935.23 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030027.624375 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197417.04 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 19741.70 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049769.328622 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177675.34 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 17767.53 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067536.862445 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159907.80 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 15990.78 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083527.642885 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143917.02 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 14391.70 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097919.345281 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129525.32 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 12952.53 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110871.877438 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116572.79 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 11657.28 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122529.156379 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104915.51 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 10491.55 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133020.707426 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94423.96 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 9442.40 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142463.103368 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84981.56 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 8498.16 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150961.259716 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76483.41 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 7648.34 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158609.600429 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68835.07 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 6883.51 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165493.107071 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61951.56 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 6195.16 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171688.263048 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55756.40 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 5575.64 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177263.903428 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50180.76 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 5018.08 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182281.979770 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45162.69 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 4516.27 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186798.248478 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40646.42 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 4064.64 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190862.890315 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36581.78 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3658.18 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194521.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32923.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198121.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29323.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201721.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25723.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205321.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22123.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208921.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18523.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212521.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14923.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216121.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11323.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219721.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7723.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223321.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4123.60 sec) 14:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226921.067968 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 523.60 sec) 14:31:00 [GPUGRID] [rr_sim] 2227444.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979583136.40G/439.78G) 14:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852745.58 14:31:00 [---] pick_jobs_to_run() start 14:31:00 [---] pick_jobs_to_run() end 14:31:00 [---] [rr_sim] end 14:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:31:00 [---] [cpu_sched_debug] preliminary job list: 14:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:31:00 [---] [cpu_sched_debug] final job list: 14:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:31:00 [---] [cpu_sched_debug] enforce_run_list: end 14:31:00 [---] choose_project(): 52260.000000 14:31:00 [WUProp@Home] doing RPC to get work 14:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383339.09 cpu 2130272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130272.90 14:31:00 [---] Got 0 tasks 14:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:32:00 [---] [rr_sim] doing sim: CPU sched 14:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979555546.05G/439.78G) 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227381.93 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 222738.19 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222738.192997 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004643.74 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 200464.37 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423202.566693 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804179.36 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 180417.94 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603620.503021 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623761.43 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 162376.14 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765996.645715 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461385.28 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 146138.53 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912135.174140 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315246.76 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 131524.68 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043659.849723 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183722.08 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 118372.21 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1162032.057747 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065349.87 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 106534.99 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268567.044969 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958814.88 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 95881.49 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364448.533468 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862933.40 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 86293.34 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450741.873118 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776640.06 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 77664.01 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528405.878803 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698976.05 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 69897.61 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598303.483919 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629078.45 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 62907.84 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661211.328524 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566170.60 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 56617.06 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717828.388668 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509553.54 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 50955.35 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768783.742797 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458598.19 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 45859.82 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814643.561514 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412738.37 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 41273.84 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855917.398359 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371464.53 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 37146.45 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893063.851520 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334318.08 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 33431.81 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926495.659364 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300886.27 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 30088.63 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956584.286424 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270797.64 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 27079.76 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983664.050779 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243717.88 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 24371.79 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008035.838697 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219346.09 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 21934.61 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029970.447824 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197411.48 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 19741.15 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049711.596038 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177670.33 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 17767.03 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067478.629431 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159903.30 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 15990.33 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083468.959484 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143912.97 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 14391.30 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097860.256532 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129521.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 12952.17 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110812.423876 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116569.51 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 11656.95 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122469.374485 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104912.56 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 10491.26 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132960.630033 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94421.30 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 9442.13 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142402.760026 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84979.17 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 8497.92 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150900.677020 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76481.25 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 7648.13 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158548.802314 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68833.13 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 6883.31 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165432.115079 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61949.81 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 6194.98 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171627.096568 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55754.83 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 5575.48 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177202.579908 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50179.35 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 5017.94 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182220.514913 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45161.42 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 4516.14 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186736.656419 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40645.27 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 4064.53 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190801.183773 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36580.75 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3658.07 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194459.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32922.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198059.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29322.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201659.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25722.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205259.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22122.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208859.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18522.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212459.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14922.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216059.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11322.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219659.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7722.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223259.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4122.67 sec) 14:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226859.258392 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 522.67 sec) 14:32:00 [GPUGRID] [rr_sim] 2227381.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979555546.05G/439.78G) 14:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852742.84 14:32:00 [---] pick_jobs_to_run() start 14:32:00 [---] pick_jobs_to_run() end 14:32:00 [---] [rr_sim] end 14:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:32:00 [---] [cpu_sched_debug] preliminary job list: 14:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:32:00 [---] [cpu_sched_debug] final job list: 14:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:32:00 [---] [cpu_sched_debug] enforce_run_list: end 14:32:00 [---] choose_project(): 52320.000000 14:32:00 [WUProp@Home] doing RPC to get work 14:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383279.09 cpu 2130212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130212.90 14:32:00 [---] Got 0 tasks 14:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:33:00 [---] [rr_sim] doing sim: CPU sched 14:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979527955.69G/439.78G) 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227319.19 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 222731.92 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222731.919308 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004587.27 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 200458.73 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423190.646686 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804128.55 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 180412.85 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603603.501325 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623715.69 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 162371.57 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765975.070501 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461344.12 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 146134.41 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912109.482759 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315209.71 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 131520.97 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043630.453792 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183688.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 118368.87 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161999.327721 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065319.87 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 106531.99 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268531.314257 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958787.88 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 95878.79 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364410.102140 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862909.09 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 86290.91 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450701.011234 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776618.18 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 77661.82 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528362.829419 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698956.36 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 69895.64 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598258.465785 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629060.73 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 62906.07 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661164.538515 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566154.65 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 56615.47 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717780.003972 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509539.19 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 50953.92 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768733.922883 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458585.27 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 45858.53 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814592.449903 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412726.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 41272.67 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855865.124221 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371454.07 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 37145.41 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1893010.531107 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334308.66 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 33430.87 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926441.397305 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300877.80 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 30087.78 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956529.176883 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270790.02 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 27079.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983608.178503 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243711.01 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 24371.10 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007979.279961 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219339.91 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 21933.99 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029913.271273 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197405.92 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 19740.59 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049653.863454 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177665.33 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 17766.53 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067420.396417 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159898.80 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 15989.88 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083410.276083 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143908.92 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 14390.89 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097801.167783 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129518.03 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 12951.80 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110752.970313 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116566.22 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 11656.62 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122409.592590 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104909.60 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 10490.96 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132900.552640 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94418.64 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 9441.86 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142342.416684 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84976.78 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 8497.68 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150840.094324 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76479.10 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 7647.91 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158488.004200 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68831.19 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 6883.12 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165371.123088 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61948.07 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 6194.81 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171565.930088 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55753.26 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 5575.33 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177141.256387 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50177.94 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 5017.79 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182159.050057 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45160.14 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 4516.01 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186675.064359 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40644.13 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 4064.41 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190739.477232 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36579.72 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3657.97 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194397.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32921.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197997.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29321.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201597.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25721.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205197.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22121.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208797.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18521.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212397.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14921.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215997.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11321.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219597.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7721.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223197.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4121.74 sec) 14:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226797.448817 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 521.74 sec) 14:33:00 [GPUGRID] [rr_sim] 2227319.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979527955.69G/439.78G) 14:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852740.11 14:33:00 [---] pick_jobs_to_run() start 14:33:00 [---] pick_jobs_to_run() end 14:33:00 [---] [rr_sim] end 14:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:33:00 [---] [cpu_sched_debug] preliminary job list: 14:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:33:00 [---] [cpu_sched_debug] final job list: 14:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:33:00 [---] [cpu_sched_debug] enforce_run_list: end 14:33:00 [---] choose_project(): 52380.000000 14:33:00 [WUProp@Home] doing RPC to get work 14:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383219.09 cpu 2130152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130152.90 14:33:00 [---] Got 0 tasks 14:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:34:00 [---] [rr_sim] doing sim: CPU sched 14:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979500365.34G/439.78G) 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227256.46 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 222725.65 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222725.645620 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004530.81 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 200453.08 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423178.726678 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804077.73 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 180407.77 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603586.499630 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623669.96 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 162367.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765953.495287 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461302.96 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 146130.30 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912083.791379 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315172.66 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 131517.27 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043601.057861 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183655.40 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 118365.54 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161966.597695 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065289.86 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 106528.99 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268495.583546 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958760.87 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 95876.09 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364371.670811 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862884.79 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 86288.48 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450660.149350 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776596.31 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 77659.63 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528319.780035 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698936.68 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 69893.67 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598213.447652 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629043.01 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 62904.30 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661117.748507 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566138.71 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 56613.87 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717731.619276 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509524.84 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 50952.48 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768684.102968 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458572.35 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 45857.24 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814541.338292 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412715.12 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 41271.51 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855812.850083 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371443.61 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 37144.36 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892957.210694 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334299.25 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 33429.92 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926387.135245 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300869.32 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 30086.93 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956474.067341 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270782.39 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 27078.24 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983552.306227 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243704.15 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 24370.41 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007922.721224 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219333.73 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 21933.37 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029856.094722 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197400.36 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 19740.04 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049596.130870 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177660.33 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 17766.03 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067362.163403 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159894.29 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 15989.43 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083351.592683 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143904.86 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 14390.49 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097742.079034 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129514.38 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 12951.44 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110693.516751 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116562.94 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 11656.29 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122349.810696 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104906.65 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 10490.66 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132840.475246 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94415.98 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 9441.60 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142282.073342 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84974.38 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 8497.44 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150779.511628 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76476.94 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 7647.69 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158427.206085 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68829.25 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 6882.93 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165310.131097 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61946.33 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 6194.63 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171504.763607 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55751.69 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 5575.17 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177079.932866 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50176.52 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 5017.65 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182097.585200 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45158.87 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 4515.89 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186613.472300 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40642.98 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 4064.30 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190677.770690 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36578.69 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3657.87 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194335.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32920.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197935.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29320.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201535.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25720.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205135.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22120.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208735.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18520.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212335.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14920.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215935.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11320.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219535.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7720.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223135.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4120.82 sec) 14:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226735.639241 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 520.82 sec) 14:34:00 [GPUGRID] [rr_sim] 2227256.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979500365.34G/439.78G) 14:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852737.37 14:34:00 [---] pick_jobs_to_run() start 14:34:00 [---] pick_jobs_to_run() end 14:34:00 [---] [rr_sim] end 14:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:34:00 [---] [cpu_sched_debug] preliminary job list: 14:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:34:00 [---] [cpu_sched_debug] final job list: 14:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:34:00 [---] [cpu_sched_debug] enforce_run_list: end 14:34:00 [---] choose_project(): 52440.000000 14:34:00 [WUProp@Home] doing RPC to get work 14:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383159.09 cpu 2130092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130092.90 14:34:00 [---] Got 0 tasks 14:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:35:00 [---] [rr_sim] doing sim: CPU sched 14:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979472774.98G/439.78G) 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227193.72 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 222719.37 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222719.371932 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004474.35 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 200447.43 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423166.806671 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1804026.91 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 180402.69 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603569.497935 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623624.22 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 162362.42 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765931.920074 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461261.80 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 146126.18 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912058.099998 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315135.62 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 131513.56 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043571.661930 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183622.06 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 118362.21 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161933.867669 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065259.85 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 106525.99 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268459.852834 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958733.87 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 95873.39 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364333.239482 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862860.48 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 86286.05 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450619.287466 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776574.43 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 77657.44 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528276.730651 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698916.99 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 69891.70 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598168.429518 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629025.29 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 62902.53 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661070.958498 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566122.76 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 56612.28 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717683.234580 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509510.48 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 50951.05 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768634.283054 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458559.44 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 45855.94 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814490.226680 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412703.49 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 41270.35 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855760.575944 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371433.14 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 37143.31 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892903.890282 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334289.83 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 33428.98 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926332.873185 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300860.85 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 30086.08 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956418.957799 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270774.76 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 27077.48 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983496.433951 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243697.29 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 24369.73 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007866.162488 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219327.56 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 21932.76 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029798.918171 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197394.80 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 19739.48 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049538.398285 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177655.32 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 17765.53 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067303.930389 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159889.79 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 15988.98 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083292.909282 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143900.81 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 14390.08 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097682.990285 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129510.73 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 12951.07 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110634.063189 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116559.66 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 11655.97 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122290.028802 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104903.69 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 10490.37 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132780.397853 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94413.32 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 9441.33 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142221.730000 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84971.99 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 8497.20 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150718.928932 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76474.79 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 7647.48 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158366.407970 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68827.31 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 6882.73 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165249.139105 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61944.58 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 6194.46 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171443.597127 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55750.12 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 5575.01 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177018.609346 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50175.11 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 5017.51 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182036.120343 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45157.60 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 4515.76 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186551.880241 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40641.84 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 4064.18 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190616.064148 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36577.66 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3657.77 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194273.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32919.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197873.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29319.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201473.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25719.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205073.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22119.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208673.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18519.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212273.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14919.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215873.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11319.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219473.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7719.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223073.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4119.89 sec) 14:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226673.829665 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 519.89 sec) 14:35:00 [GPUGRID] [rr_sim] 2227193.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979472774.98G/439.78G) 14:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852734.63 14:35:00 [---] pick_jobs_to_run() start 14:35:00 [---] pick_jobs_to_run() end 14:35:00 [---] [rr_sim] end 14:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:35:00 [---] [cpu_sched_debug] preliminary job list: 14:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:35:00 [---] [cpu_sched_debug] final job list: 14:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:35:00 [---] [cpu_sched_debug] enforce_run_list: end 14:35:00 [---] choose_project(): 52500.000000 14:35:00 [WUProp@Home] doing RPC to get work 14:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383099.09 cpu 2130032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2130032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2130032.90 14:35:00 [---] Got 0 tasks 14:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:36:00 [---] [rr_sim] doing sim: CPU sched 14:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979445184.62G/439.78G) 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227130.98 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 222713.10 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222713.098244 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004417.88 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 200441.79 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423154.886663 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803976.10 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 180397.61 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603552.496240 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623578.49 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 162357.85 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765910.344860 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461220.64 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 146122.06 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912032.408618 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315098.57 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 131509.86 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043542.265999 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183588.72 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 118358.87 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161901.137643 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065229.84 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 106522.98 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268424.122122 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958706.86 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 95870.69 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364294.808154 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862836.17 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 86283.62 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450578.425582 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776552.56 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 77655.26 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528233.681268 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698897.30 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 69889.73 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598123.411384 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 629007.57 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 62900.76 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1661024.168490 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566106.81 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 56610.68 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717634.849884 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509496.13 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 50949.61 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768584.463140 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458546.52 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 45854.65 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814439.115069 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412691.87 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 41269.19 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855708.301806 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371422.68 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 37142.27 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892850.569869 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334280.41 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 33428.04 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926278.611126 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300852.37 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 30085.24 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956363.848257 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270767.13 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 27076.71 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983440.561675 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243690.42 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 24369.04 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007809.603751 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219321.38 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 21932.14 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029741.741620 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197389.24 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 19738.92 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049480.665701 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177650.32 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 17765.03 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067245.697375 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159885.29 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 15988.53 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083234.225881 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143896.76 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 14389.68 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097623.901536 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129507.08 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 12950.71 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110574.609626 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116556.37 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 11655.64 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122230.246907 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104900.74 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 10490.07 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132720.320460 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94410.66 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 9441.07 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142161.386658 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84969.60 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 8496.96 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150658.346236 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76472.64 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 7647.26 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158305.609856 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68825.37 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 6882.54 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165188.147114 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61942.84 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 6194.28 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171382.430646 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55748.55 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 5574.86 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176957.285825 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50173.70 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 5017.37 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181974.655486 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45156.33 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 4515.63 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186490.288181 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40640.69 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 4064.07 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190554.357607 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36576.62 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3657.66 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194212.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32918.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197812.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29318.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201412.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25718.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205012.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22118.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208612.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18518.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212212.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14918.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215812.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11318.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219412.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7718.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223012.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4118.96 sec) 14:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226612.020090 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518.96 sec) 14:36:00 [GPUGRID] [rr_sim] 2227130.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979445184.62G/439.78G) 14:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852731.90 14:36:00 [---] pick_jobs_to_run() start 14:36:00 [---] pick_jobs_to_run() end 14:36:00 [---] [rr_sim] end 14:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:36:00 [---] [cpu_sched_debug] preliminary job list: 14:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:36:00 [---] [cpu_sched_debug] final job list: 14:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:36:00 [---] [cpu_sched_debug] enforce_run_list: end 14:36:00 [---] choose_project(): 52560.000000 14:36:00 [WUProp@Home] doing RPC to get work 14:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 383039.09 cpu 2129972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129972.90 14:36:00 [---] Got 0 tasks 14:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:37:00 [---] [rr_sim] doing sim: CPU sched 14:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979417594.27G/439.78G) 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227068.25 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 222706.82 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222706.824555 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004361.42 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 200436.14 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423142.966655 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803925.28 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 180392.53 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603535.494545 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623532.75 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 162353.28 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765888.769646 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461179.48 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 146117.95 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 912006.717237 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315061.53 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 131506.15 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043512.870069 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183555.38 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 118355.54 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161868.407617 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065199.84 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 106519.98 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268388.391411 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958679.85 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 95867.99 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364256.376825 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862811.87 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 86281.19 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450537.563698 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776530.68 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 77653.07 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528190.631884 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698877.61 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 69887.76 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598078.393251 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628989.85 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 62898.99 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660977.378481 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566090.87 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 56609.09 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717586.465189 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509481.78 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 50948.18 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768534.643225 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458533.60 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 45853.36 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814388.003458 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412680.24 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 41268.02 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855656.027668 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371412.22 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 37141.22 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892797.249456 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334271.00 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 33427.10 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926224.349066 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300843.90 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 30084.39 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956308.738715 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270759.51 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 27075.95 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983384.689399 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243683.56 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 24368.36 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007753.045014 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219315.20 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 21931.52 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029684.565068 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197383.68 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 19738.37 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049422.933117 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177645.31 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 17764.53 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067187.464361 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159880.78 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 15988.08 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083175.542480 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143892.70 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 14389.27 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097564.812787 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129503.43 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 12950.34 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110515.156064 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116553.09 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 11655.31 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122170.465013 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104897.78 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 10489.78 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132660.243067 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94408.00 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 9440.80 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142101.043316 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84967.20 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 8496.72 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150597.763540 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76470.48 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 7647.05 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158244.811741 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68823.43 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 6882.34 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165127.155123 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61941.09 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 6194.11 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171321.264166 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55746.98 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 5574.70 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176895.962305 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50172.28 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 5017.23 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181913.190630 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45155.05 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 4515.51 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186428.696122 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40639.55 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 4063.95 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190492.651065 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36575.59 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3657.56 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194150.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32918.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197750.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29318.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201350.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25718.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204950.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22118.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208550.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18518.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212150.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14918.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215750.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11318.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219350.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7718.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222950.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4118.04 sec) 14:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226550.210514 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 518.04 sec) 14:37:00 [GPUGRID] [rr_sim] 2227068.25: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979417594.27G/439.78G) 14:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852729.16 14:37:00 [---] pick_jobs_to_run() start 14:37:00 [---] pick_jobs_to_run() end 14:37:00 [---] [rr_sim] end 14:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:37:00 [---] [cpu_sched_debug] preliminary job list: 14:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:37:00 [---] [cpu_sched_debug] final job list: 14:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:37:00 [---] [cpu_sched_debug] enforce_run_list: end 14:37:00 [---] choose_project(): 52620.000000 14:37:00 [WUProp@Home] doing RPC to get work 14:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382979.09 cpu 2129912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129912.90 14:37:00 [---] Got 0 tasks 14:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:38:00 [---] [rr_sim] doing sim: CPU sched 14:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979390003.91G/439.78G) 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2227005.51 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 222700.55 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222700.550867 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004304.96 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 200430.50 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423131.046648 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803874.46 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 180387.45 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603518.492850 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623487.02 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 162348.70 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765867.194432 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461138.31 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 146113.83 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911981.025856 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1315024.48 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 131502.45 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043483.474138 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183522.03 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 118352.20 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161835.677591 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065169.83 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 106516.98 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268352.660699 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958652.85 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 95865.28 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364217.945497 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862787.56 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 86278.76 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450496.701814 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776508.81 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 77650.88 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528147.582500 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698857.93 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 69885.79 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1598033.375117 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628972.13 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 62897.21 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660930.588473 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566074.92 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 56607.49 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717538.080493 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509467.43 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 50946.74 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768484.823311 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458520.69 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 45852.07 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814336.891847 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412668.62 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 41266.86 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855603.753529 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371401.76 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 37140.18 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892743.929044 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334261.58 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 33426.16 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926170.087006 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300835.42 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 30083.54 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956253.629173 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270751.88 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 27075.19 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983328.817123 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243676.69 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 24367.67 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007696.486278 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219309.02 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 21930.90 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029627.388517 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197378.12 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 19737.81 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049365.200533 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177640.31 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 17764.03 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067129.231347 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159876.28 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 15987.63 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083116.859079 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143888.65 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 14388.86 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097505.724039 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129499.78 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 12949.98 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110455.702502 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116549.81 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 11654.98 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122110.683119 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104894.83 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 10489.48 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132600.165674 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94405.34 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 9440.53 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142040.699974 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84964.81 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 8496.48 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150537.180844 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76468.33 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 7646.83 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158184.013627 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68821.50 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 6882.15 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165066.163131 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61939.35 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 6193.93 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171260.097685 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55745.41 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 5574.54 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176834.638784 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50170.87 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 5017.09 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181851.725773 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45153.78 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 4515.38 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186367.104063 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40638.40 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 4063.84 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190430.944524 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36574.56 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3657.46 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194088.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32917.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197688.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29317.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201288.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25717.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204888.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22117.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208488.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18517.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212088.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14917.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215688.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11317.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219288.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7717.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222888.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4117.11 sec) 14:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226488.400939 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 517.11 sec) 14:38:00 [GPUGRID] [rr_sim] 2227005.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979390003.91G/439.78G) 14:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852726.42 14:38:00 [---] pick_jobs_to_run() start 14:38:00 [---] pick_jobs_to_run() end 14:38:00 [---] [rr_sim] end 14:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:38:00 [---] [cpu_sched_debug] preliminary job list: 14:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:38:00 [---] [cpu_sched_debug] final job list: 14:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:38:00 [---] [cpu_sched_debug] enforce_run_list: end 14:38:00 [---] choose_project(): 52680.000000 14:38:00 [WUProp@Home] doing RPC to get work 14:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382919.09 cpu 2129852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129852.90 14:38:00 [---] Got 0 tasks 14:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:39:00 [---] [rr_sim] doing sim: CPU sched 14:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979362413.55G/439.78G) 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226942.77 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 222694.28 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222694.277179 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004248.49 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 200424.85 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423119.126640 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803823.65 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 180382.36 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603501.491155 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623441.28 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 162344.13 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765845.619219 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461097.15 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 146109.72 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911955.334476 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314987.44 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 131498.74 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043454.078207 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183488.69 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 118348.87 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161802.947565 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065139.82 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 106513.98 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268316.929988 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958625.84 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 95862.58 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364179.514168 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862763.26 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 86276.33 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450455.839930 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776486.93 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 77648.69 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528104.533116 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698838.24 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 69883.82 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597988.356984 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628954.41 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 62895.44 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660883.798464 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566058.97 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 56605.90 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717489.695797 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509453.08 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 50945.31 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768435.003396 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458507.77 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 45850.78 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814285.780236 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412656.99 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 41265.70 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855551.479391 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371391.29 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 37139.13 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892690.608631 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334252.16 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 33425.22 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926115.824947 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300826.95 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 30082.69 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956198.519631 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270744.25 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 27074.43 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983272.944847 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243669.83 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 24366.98 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007639.927541 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219302.84 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 21930.28 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029570.211966 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197372.56 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 19737.26 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049307.467949 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177635.30 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 17763.53 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067070.998333 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159871.77 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 15987.18 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083058.175678 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143884.60 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 14388.46 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097446.635290 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129496.14 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 12949.61 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110396.248940 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116546.52 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 11654.65 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122050.901225 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104891.87 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 10489.19 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132540.088281 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94402.68 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 9440.27 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141980.356632 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84962.42 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 8496.24 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150476.598148 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76466.17 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 7646.62 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158123.215512 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68819.56 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 6881.96 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165005.171140 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61937.60 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 6193.76 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171198.931205 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55743.84 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 5574.38 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176773.315263 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50169.46 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 5016.95 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181790.260916 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45152.51 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 4515.25 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186305.512003 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40637.26 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 4063.73 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190369.237982 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36573.53 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3657.35 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194026.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32916.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197626.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29316.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201226.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25716.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204826.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22116.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208426.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18516.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212026.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14916.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215626.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11316.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219226.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7716.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222826.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4116.18 sec) 14:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226426.591363 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 516.18 sec) 14:39:00 [GPUGRID] [rr_sim] 2226942.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979362413.55G/439.78G) 14:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852723.68 14:39:00 [---] pick_jobs_to_run() start 14:39:00 [---] pick_jobs_to_run() end 14:39:00 [---] [rr_sim] end 14:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:39:00 [---] [cpu_sched_debug] preliminary job list: 14:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:39:00 [---] [cpu_sched_debug] final job list: 14:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:39:00 [---] [cpu_sched_debug] enforce_run_list: end 14:39:00 [---] choose_project(): 52740.000000 14:39:00 [WUProp@Home] doing RPC to get work 14:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382859.09 cpu 2129792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129792.90 14:39:00 [---] Got 0 tasks 14:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:40:00 [---] [rr_sim] doing sim: CPU sched 14:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979334823.20G/439.78G) 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226880.03 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 222688.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222688.003491 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004192.03 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 200419.20 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423107.206632 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803772.83 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 180377.28 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603484.489460 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623395.55 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 162339.55 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765824.044005 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461055.99 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 146105.60 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911929.643095 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314950.39 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 131495.04 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043424.682276 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183455.35 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 118345.54 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161770.217540 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065109.82 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 106510.98 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268281.199276 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958598.84 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 95859.88 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364141.082839 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862738.95 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 86273.90 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450414.978046 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776465.06 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 77646.51 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528061.483732 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698818.55 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 69881.86 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597943.338850 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628936.70 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 62893.67 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660837.008456 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566043.03 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 56604.30 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717441.311101 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509438.72 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 50943.87 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768385.183482 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458494.85 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 45849.49 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814234.668624 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412645.37 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 41264.54 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855499.205253 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371380.83 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 37138.08 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892637.288218 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334242.75 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 33424.27 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926061.562887 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300818.47 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 30081.85 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956143.410089 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270736.62 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 27073.66 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983217.072571 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243662.96 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 24366.30 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007583.368805 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219296.67 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 21929.67 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029513.035415 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197367.00 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 19736.70 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049249.735364 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177630.30 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 17763.03 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067012.765319 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159867.27 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 15986.73 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082999.492278 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143880.54 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 14388.05 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097387.546541 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129492.49 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 12949.25 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110336.795377 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116543.24 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 11654.32 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121991.119330 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104888.92 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 10488.89 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132480.010888 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94400.02 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 9440.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141920.013290 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84960.02 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 8496.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150416.015452 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76464.02 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 7646.40 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158062.417397 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68817.62 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 6881.76 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164944.179148 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61935.86 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 6193.59 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171137.764724 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55742.27 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 5574.23 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176711.991743 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50168.04 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 5016.80 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181728.796059 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45151.24 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 4515.12 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186243.919944 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40636.11 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 4063.61 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190307.531440 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36572.50 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3657.25 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193964.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32915.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197564.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29315.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201164.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25715.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204764.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22115.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208364.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18515.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211964.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14915.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215564.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11315.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219164.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7715.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222764.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4115.25 sec) 14:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226364.781787 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 515.25 sec) 14:40:00 [GPUGRID] [rr_sim] 2226880.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979334823.20G/439.78G) 14:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852720.95 14:40:00 [---] pick_jobs_to_run() start 14:40:00 [---] pick_jobs_to_run() end 14:40:00 [---] [rr_sim] end 14:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:40:00 [---] [cpu_sched_debug] preliminary job list: 14:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:40:00 [---] [cpu_sched_debug] final job list: 14:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:40:00 [---] [cpu_sched_debug] enforce_run_list: end 14:40:00 [---] choose_project(): 52800.000000 14:40:00 [WUProp@Home] doing RPC to get work 14:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382799.09 cpu 2129732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129732.90 14:40:00 [---] Got 0 tasks 14:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:41:00 [---] [rr_sim] doing sim: CPU sched 14:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979307232.84G/439.78G) 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226817.30 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 222681.73 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222681.729803 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004135.57 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 200413.56 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423095.286625 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803722.01 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 180372.20 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603467.487765 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623349.81 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 162334.98 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765802.468791 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461014.83 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 146101.48 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911903.951714 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314913.35 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 131491.33 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043395.286346 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183422.01 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 118342.20 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161737.487514 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065079.81 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 106507.98 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268245.468565 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958571.83 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 95857.18 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364102.651511 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862714.65 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 86271.46 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450374.116162 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776443.18 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 77644.32 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528018.434349 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698798.86 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 69879.89 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597898.320716 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628918.98 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 62891.90 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660790.218447 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566027.08 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 56602.71 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717392.926405 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509424.37 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 50942.44 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768335.363567 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458481.93 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 45848.19 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814183.557013 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412633.74 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 41263.37 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855446.931114 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371370.37 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 37137.04 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892583.967805 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334233.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 33423.33 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926007.300827 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300810.00 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 30081.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956088.300547 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270729.00 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 27072.90 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983161.200295 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243656.10 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 24365.61 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007526.810068 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219290.49 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 21929.05 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029455.858864 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197361.44 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 19736.14 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049192.002780 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177625.30 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 17762.53 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066954.532305 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159862.77 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 15986.28 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082940.808877 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143876.49 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 14387.65 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097328.457792 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129488.84 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 12948.88 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110277.341815 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116539.96 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 11654.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121931.337436 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104885.96 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 10488.60 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132419.933495 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94397.36 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 9439.74 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141859.669948 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84957.63 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 8495.76 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150355.432756 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76461.87 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 7646.19 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158001.619283 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68815.68 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 6881.57 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164883.187157 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61934.11 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 6193.41 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171076.598244 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55740.70 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 5574.07 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176650.668222 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50166.63 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 5016.66 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181667.331202 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45149.97 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 4515.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186182.327885 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40634.97 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 4063.50 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190245.824899 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36571.47 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3657.15 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193902.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32914.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197502.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29314.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201102.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25714.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204702.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22114.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208302.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18514.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211902.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14914.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215502.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11314.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219102.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7714.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222702.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4114.33 sec) 14:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226302.972212 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514.33 sec) 14:41:00 [GPUGRID] [rr_sim] 2226817.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979307232.84G/439.78G) 14:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852718.21 14:41:00 [---] pick_jobs_to_run() start 14:41:00 [---] pick_jobs_to_run() end 14:41:00 [---] [rr_sim] end 14:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:41:00 [---] [cpu_sched_debug] preliminary job list: 14:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:41:00 [---] [cpu_sched_debug] final job list: 14:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:41:00 [---] [cpu_sched_debug] enforce_run_list: end 14:41:00 [---] choose_project(): 52860.000000 14:41:00 [WUProp@Home] doing RPC to get work 14:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382739.09 cpu 2129672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129672.90 14:41:00 [---] Got 0 tasks 14:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:42:00 [---] [rr_sim] doing sim: CPU sched 14:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979307232.84G/439.78G) 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226817.30 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 222681.73 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222681.729803 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004135.57 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 200413.56 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423095.286625 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803722.01 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 180372.20 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603467.487765 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623349.81 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 162334.98 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765802.468791 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1461014.83 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 146101.48 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911903.951714 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314913.35 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 131491.33 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043395.286346 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183422.01 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 118342.20 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161737.487514 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065079.81 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 106507.98 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268245.468565 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958571.83 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 95857.18 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364102.651511 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862714.65 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 86271.46 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450374.116162 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776443.18 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 77644.32 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1528018.434349 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698798.86 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 69879.89 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597898.320716 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628918.98 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 62891.90 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660790.218447 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566027.08 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 56602.71 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717392.926405 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509424.37 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 50942.44 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768335.363567 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458481.93 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 45848.19 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814183.557013 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412633.74 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 41263.37 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855446.931114 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371370.37 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 37137.04 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892583.967805 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334233.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 33423.33 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926007.300827 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300810.00 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 30081.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956088.300547 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270729.00 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 27072.90 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983161.200295 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243656.10 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 24365.61 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007526.810068 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219290.49 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 21929.05 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029455.858864 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197361.44 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 19736.14 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049192.002780 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177625.30 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 17762.53 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066954.532305 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159862.77 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 15986.28 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082940.808877 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143876.49 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 14387.65 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097328.457792 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129488.84 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 12948.88 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110277.341815 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116539.96 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 11654.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121931.337436 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104885.96 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 10488.60 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132419.933495 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94397.36 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 9439.74 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141859.669948 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84957.63 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 8495.76 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150355.432756 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76461.87 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 7646.19 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158001.619283 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68815.68 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 6881.57 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164883.187157 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61934.11 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 6193.41 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171076.598244 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55740.70 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 5574.07 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176650.668222 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50166.63 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 5016.66 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181667.331202 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45149.97 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 4515.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186182.327885 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40634.97 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 4063.50 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190245.824899 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36571.47 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3657.15 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193902.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32914.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197502.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29314.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201102.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25714.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204702.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22114.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208302.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18514.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211902.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14914.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215502.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11314.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219102.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7714.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222702.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4114.33 sec) 14:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226302.972212 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 514.33 sec) 14:42:00 [GPUGRID] [rr_sim] 2226817.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979307232.84G/439.78G) 14:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852778.21 14:42:00 [---] pick_jobs_to_run() start 14:42:00 [---] pick_jobs_to_run() end 14:42:00 [---] [rr_sim] end 14:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:42:00 [---] [cpu_sched_debug] preliminary job list: 14:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:42:00 [---] [cpu_sched_debug] final job list: 14:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:42:00 [---] [cpu_sched_debug] enforce_run_list: end 14:42:00 [---] choose_project(): 52920.000000 14:42:00 [WUProp@Home] doing RPC to get work 14:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382679.09 cpu 2129672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129672.90 14:42:00 [---] Got 0 tasks 14:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:43:00 [---] [rr_sim] doing sim: CPU sched 14:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979279642.49G/439.78G) 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226754.56 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 222675.46 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222675.456114 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004079.11 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 200407.91 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423083.366617 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803671.19 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 180367.12 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603450.486070 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623304.08 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 162330.41 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765780.893577 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460973.67 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 146097.37 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911878.260334 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314876.30 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 131487.63 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043365.890415 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183388.67 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 118338.87 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161704.757488 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065049.80 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 106504.98 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268209.737853 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958544.82 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 95854.48 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364064.220182 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862690.34 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 86269.03 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450333.254278 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776421.31 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 77642.13 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527975.384965 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698779.18 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 69877.92 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597853.302583 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628901.26 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 62890.13 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660743.428439 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566011.13 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 56601.11 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717344.541709 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509410.02 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 50941.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768285.543653 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458469.02 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 45846.90 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814132.445402 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412622.12 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 41262.21 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855394.656976 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371359.90 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 37135.99 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892530.647393 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334223.91 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 33422.39 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925953.038768 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300801.52 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 30080.15 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956033.191005 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270721.37 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 27072.14 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983105.328019 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243649.23 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 24364.92 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007470.251332 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219284.31 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 21928.43 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029398.682313 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197355.88 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 19735.59 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049134.270196 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177620.29 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 17762.03 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066896.299291 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159858.26 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 15985.83 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082882.125476 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143872.44 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 14387.24 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097269.369043 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129485.19 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 12948.52 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110217.888253 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116536.67 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 11653.67 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121871.555542 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104883.01 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 10488.30 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132359.856102 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94394.71 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 9439.47 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141799.326606 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84955.23 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 8495.52 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150294.850060 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76459.71 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 7645.97 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157940.821168 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68813.74 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 6881.37 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164822.195166 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61932.37 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 6193.24 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171015.431764 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55739.13 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 5573.91 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176589.344702 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50165.22 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 5016.52 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181605.866346 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45148.69 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 4514.87 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186120.735825 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40633.83 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 4063.38 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190184.118357 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36570.44 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3657.04 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193841.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32913.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197441.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29313.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201041.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25713.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204641.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22113.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208241.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18513.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211841.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14913.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215441.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11313.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219041.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7713.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222641.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4113.40 sec) 14:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226241.162636 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513.40 sec) 14:43:00 [GPUGRID] [rr_sim] 2226754.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979279642.49G/439.78G) 14:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852775.47 14:43:00 [---] pick_jobs_to_run() start 14:43:00 [---] pick_jobs_to_run() end 14:43:00 [---] [rr_sim] end 14:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:43:00 [---] [cpu_sched_debug] preliminary job list: 14:43:00 [---] [cpu_sched_debug] final job list: 14:43:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 14:43:00 [---] [work_fetch] Request work fetch: CPUs idle 14:43:00 [---] [cpu_sched_debug] enforce_run_list: end 14:43:00 [---] choose_project(): 52980.000000 14:43:00 [WUProp@Home] doing RPC to get work 14:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382619.09 cpu 2129612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129612.90 14:43:00 [---] Got 0 tasks 14:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:44:00 [---] [rr_sim] doing sim: CPU sched 14:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979279642.49G/439.78G) 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226754.56 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 222675.46 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222675.456114 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004079.11 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 200407.91 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423083.366617 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803671.19 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 180367.12 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603450.486070 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623304.08 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 162330.41 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765780.893577 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460973.67 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 146097.37 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911878.260334 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314876.30 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 131487.63 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043365.890415 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183388.67 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 118338.87 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161704.757488 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065049.80 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 106504.98 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268209.737853 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958544.82 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 95854.48 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364064.220182 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862690.34 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 86269.03 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450333.254278 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776421.31 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 77642.13 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527975.384965 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698779.18 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 69877.92 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597853.302583 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628901.26 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 62890.13 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660743.428439 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 566011.13 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 56601.11 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717344.541709 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509410.02 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 50941.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768285.543653 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458469.02 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 45846.90 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814132.445402 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412622.12 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 41262.21 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855394.656976 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371359.90 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 37135.99 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892530.647393 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334223.91 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 33422.39 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925953.038768 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300801.52 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 30080.15 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956033.191005 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270721.37 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 27072.14 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983105.328019 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243649.23 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 24364.92 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007470.251332 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219284.31 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 21928.43 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029398.682313 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197355.88 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 19735.59 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049134.270196 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177620.29 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 17762.03 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066896.299291 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159858.26 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 15985.83 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082882.125476 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143872.44 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 14387.24 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097269.369043 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129485.19 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 12948.52 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110217.888253 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116536.67 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 11653.67 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121871.555542 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104883.01 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 10488.30 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132359.856102 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94394.71 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 9439.47 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141799.326606 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84955.23 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 8495.52 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150294.850060 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76459.71 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 7645.97 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157940.821168 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68813.74 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 6881.37 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164822.195166 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61932.37 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 6193.24 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171015.431764 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55739.13 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 5573.91 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176589.344702 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50165.22 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 5016.52 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181605.866346 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45148.69 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 4514.87 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186120.735825 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40633.83 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 4063.38 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190184.118357 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36570.44 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3657.04 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193841.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32913.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197441.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29313.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201041.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25713.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204641.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22113.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208241.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18513.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211841.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14913.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215441.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11313.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219041.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7713.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222641.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4113.40 sec) 14:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226241.162636 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 513.40 sec) 14:44:00 [GPUGRID] [rr_sim] 2226754.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979279642.49G/439.78G) 14:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852835.47 14:44:00 [---] pick_jobs_to_run() start 14:44:00 [---] pick_jobs_to_run() end 14:44:00 [---] [rr_sim] end 14:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:44:00 [---] [cpu_sched_debug] preliminary job list: 14:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:44:00 [---] [cpu_sched_debug] final job list: 14:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:44:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 979279642.49G 14:44:00 [---] [cpu_sched_debug] enforce_run_list: end 14:44:00 [---] choose_project(): 53040.000000 14:44:00 [WUProp@Home] doing RPC to get work 14:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382559.09 cpu 2129612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129612.90 14:44:00 [---] Got 0 tasks 14:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:45:00 [---] [rr_sim] doing sim: CPU sched 14:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979252052.13G/439.78G) 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226691.82 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 222669.18 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222669.182426 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2004022.64 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 200402.26 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423071.446610 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803620.38 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 180362.04 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603433.484375 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623258.34 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 162325.83 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765759.318363 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460932.51 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 146093.25 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911852.568953 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314839.26 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 131483.93 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043336.494484 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183355.33 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 118335.53 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161672.027462 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1065019.80 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 106501.98 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268174.007142 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958517.82 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 95851.78 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1364025.788854 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862666.04 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 86266.60 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450292.392394 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776399.43 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 77639.94 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527932.335581 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698759.49 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 69875.95 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597808.284449 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628883.54 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 62888.35 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660696.638430 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565995.19 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 56599.52 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717296.157013 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509395.67 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 50939.57 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768235.723738 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458456.10 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 45845.61 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814081.333791 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412610.49 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 41261.05 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855342.382838 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371349.44 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 37134.94 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892477.326980 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334214.50 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 33421.45 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925898.776708 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300793.05 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 30079.30 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955978.081463 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270713.74 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 27071.37 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983049.455743 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243642.37 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 24364.24 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007413.692595 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219278.13 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 21927.81 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029341.505762 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197350.32 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 19735.03 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049076.537612 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177615.29 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 17761.53 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066838.066277 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159853.76 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 15985.38 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082823.442075 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143868.38 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 14386.84 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097210.280294 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129481.54 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 12948.15 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110158.434690 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116533.39 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 11653.34 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121811.773648 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104880.05 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 10488.01 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132299.778709 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94392.05 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 9439.20 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141738.983264 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84952.84 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 8495.28 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150234.267364 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76457.56 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 7645.76 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157880.023054 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68811.80 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 6881.18 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164761.203174 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61930.62 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 6193.06 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170954.265283 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55737.56 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 5573.76 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176528.021181 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50163.80 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 5016.38 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181544.401489 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45147.42 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 4514.74 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186059.143766 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40632.68 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 4063.27 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190122.411816 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36569.41 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3656.94 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193779.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32912.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197379.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29312.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200979.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25712.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204579.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22112.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208179.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18512.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211779.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14912.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215379.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11312.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218979.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7712.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222579.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4112.47 sec) 14:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226179.353060 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 512.47 sec) 14:45:00 [GPUGRID] [rr_sim] 2226691.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979252052.13G/439.78G) 14:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852832.74 14:45:00 [---] pick_jobs_to_run() start 14:45:00 [---] pick_jobs_to_run() end 14:45:00 [---] [rr_sim] end 14:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:45:00 [---] [cpu_sched_debug] preliminary job list: 14:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 14:45:00 [---] [cpu_sched_debug] final job list: 14:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:45:00 [---] [cpu_sched_debug] enforce_run_list: end 14:45:00 [---] choose_project(): 53100.000000 14:45:00 [WUProp@Home] doing RPC to get work 14:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382499.09 cpu 2129552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129552.90 14:45:00 [---] Got 0 tasks 14:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:46:00 [---] [rr_sim] doing sim: CPU sched 14:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979224461.77G/439.78G) 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226629.09 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 222662.91 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222662.908738 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003966.18 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 200396.62 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423059.526602 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803569.56 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 180356.96 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603416.482680 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623212.60 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 162321.26 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765737.743150 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460891.34 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 146089.13 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911826.877573 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314802.21 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 131480.22 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043307.098553 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183321.99 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 118332.20 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161639.297436 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064989.79 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 106498.98 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268138.276430 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958490.81 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 95849.08 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363987.357525 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862641.73 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 86264.17 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450251.530510 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776377.56 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 77637.76 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527889.286197 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698739.80 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 69873.98 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597763.266316 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628865.82 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 62886.58 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660649.848422 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565979.24 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 56597.92 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717247.772318 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509381.32 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 50938.13 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768185.903824 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458443.18 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 45844.32 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1814030.222179 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412598.87 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 41259.89 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855290.108699 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371338.98 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 37133.90 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892424.006567 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334205.08 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 33420.51 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925844.514648 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300784.57 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 30078.46 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955922.971922 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270706.12 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 27070.61 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982993.583467 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243635.50 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 24363.55 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007357.133858 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219271.95 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 21927.20 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029284.329211 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197344.76 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 19734.48 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049018.805027 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177610.28 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 17761.03 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066779.833263 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159849.25 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 15984.93 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082764.758674 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143864.33 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 14386.43 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097151.191545 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129477.90 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 12947.79 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110098.981128 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116530.11 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 11653.01 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121751.991753 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104877.10 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 10487.71 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132239.701316 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94389.39 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 9438.94 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141678.639922 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84950.45 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 8495.04 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150173.684668 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76455.40 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 7645.54 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157819.224939 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68809.86 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 6880.99 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164700.211183 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61928.88 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 6192.89 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170893.098803 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55735.99 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 5573.60 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176466.697660 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50162.39 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 5016.24 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181482.936632 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45146.15 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 4514.62 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185997.551707 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40631.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 4063.15 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190060.705274 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36568.38 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3656.84 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193717.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32911.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197317.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29311.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200917.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25711.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204517.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22111.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208117.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18511.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211717.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14911.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215317.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11311.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218917.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7711.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222517.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4111.54 sec) 14:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226117.543485 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 511.54 sec) 14:46:00 [GPUGRID] [rr_sim] 2226629.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979224461.77G/439.78G) 14:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852830.00 14:46:00 [---] pick_jobs_to_run() start 14:46:00 [---] pick_jobs_to_run() end 14:46:00 [---] [rr_sim] end 14:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:46:00 [---] [cpu_sched_debug] preliminary job list: 14:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:46:00 [---] [cpu_sched_debug] final job list: 14:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:46:00 [---] [cpu_sched_debug] enforce_run_list: end 14:46:00 [---] choose_project(): 53160.000000 14:46:00 [WUProp@Home] doing RPC to get work 14:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382439.09 cpu 2129492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129492.90 14:46:00 [---] Got 0 tasks 14:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:47:00 [---] [rr_sim] doing sim: CPU sched 14:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979196871.42G/439.78G) 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226566.35 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 222656.64 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222656.635050 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003909.72 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 200390.97 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423047.606594 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803518.74 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 180351.87 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603399.480985 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623166.87 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 162316.69 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765716.167936 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460850.18 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 146085.02 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911801.186192 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314765.16 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 131476.52 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043277.702623 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183288.65 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 118328.86 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161606.567410 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064959.78 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 106495.98 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268102.545719 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958463.80 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 95846.38 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363948.926196 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862617.42 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 86261.74 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450210.668627 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776355.68 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 77635.57 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527846.236814 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698720.11 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 69872.01 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597718.248182 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628848.10 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 62884.81 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660603.058413 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565963.29 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 56596.33 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717199.387622 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509366.96 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 50936.70 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768136.083909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458430.27 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 45843.03 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813979.110568 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412587.24 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 41258.72 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855237.834561 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371328.52 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 37132.85 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892370.686155 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334195.66 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 33419.57 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925790.252589 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300776.10 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 30077.61 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955867.862380 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270698.49 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 27069.85 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982937.711191 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243628.64 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 24362.86 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007300.575122 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219265.78 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 21926.58 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029227.152659 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197339.20 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 19733.92 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048961.072443 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177605.28 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 17760.53 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066721.600249 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159844.75 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 15984.48 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082706.075273 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143860.28 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 14386.03 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097092.102796 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129474.25 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 12947.42 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110039.527566 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116526.82 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 11652.68 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121692.209859 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104874.14 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 10487.41 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132179.623923 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94386.73 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 9438.67 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141618.296580 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84948.05 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 8494.81 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150113.101972 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76453.25 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 7645.32 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157758.426824 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68807.92 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 6880.79 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164639.219192 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61927.13 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 6192.71 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170831.932322 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55734.42 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 5573.44 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176405.374140 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50160.98 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 5016.10 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181421.471775 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45144.88 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 4514.49 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185935.959648 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40630.39 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 4063.04 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189998.998733 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36567.35 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3656.74 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193655.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32910.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197255.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29310.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200855.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25710.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204455.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22110.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208055.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18510.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211655.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14910.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215255.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11310.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218855.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7710.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222455.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4110.62 sec) 14:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2226055.733909 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 510.62 sec) 14:47:00 [GPUGRID] [rr_sim] 2226566.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979196871.42G/439.78G) 14:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852827.26 14:47:00 [---] pick_jobs_to_run() start 14:47:00 [---] pick_jobs_to_run() end 14:47:00 [---] [rr_sim] end 14:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:47:00 [---] [cpu_sched_debug] preliminary job list: 14:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:47:00 [---] [cpu_sched_debug] final job list: 14:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:47:00 [---] [cpu_sched_debug] enforce_run_list: end 14:47:00 [---] choose_project(): 53220.000000 14:47:00 [WUProp@Home] doing RPC to get work 14:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382379.09 cpu 2129432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129432.90 14:47:00 [---] Got 0 tasks 14:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:48:00 [---] [rr_sim] doing sim: CPU sched 14:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979169281.06G/439.78G) 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226503.61 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 222650.36 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222650.361361 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003853.25 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 200385.33 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423035.686587 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803467.93 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 180346.79 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603382.479290 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623121.13 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 162312.11 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765694.592722 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460809.02 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 146080.90 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911775.494811 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314728.12 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 131472.81 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043248.306692 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183255.31 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 118325.53 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161573.837384 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064929.78 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 106492.98 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268066.815007 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958436.80 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 95843.68 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363910.494868 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862593.12 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 86259.31 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450169.806743 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776333.81 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 77633.38 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527803.187430 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698700.43 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 69870.04 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597673.230048 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628830.38 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 62883.04 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660556.268405 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565947.35 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 56594.73 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717151.002926 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509352.61 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 50935.26 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768086.263995 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458417.35 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 45841.73 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813927.998957 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412575.61 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 41257.56 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855185.560423 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371318.05 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 37131.81 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892317.365742 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334186.25 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 33418.62 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925735.990529 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300767.62 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 30076.76 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955812.752838 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270690.86 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 27069.09 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982881.838915 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243621.77 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 24362.18 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007244.016385 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219259.60 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 21925.96 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029169.976108 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197333.64 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 19733.36 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048903.339859 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177600.27 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 17760.03 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066663.367235 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159840.25 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 15984.02 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082647.391873 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143856.22 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 14385.62 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097033.014047 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129470.60 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 12947.06 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109980.074004 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116523.54 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 11652.35 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121632.427965 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104871.19 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 10487.12 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132119.546530 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94384.07 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 9438.41 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141557.953238 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84945.66 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 8494.57 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150052.519276 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76451.09 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 7645.11 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157697.628710 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68805.98 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 6880.60 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164578.227200 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61925.39 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 6192.54 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170770.765842 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55732.85 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 5573.28 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176344.050619 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50159.56 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 5015.96 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181360.006919 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45143.61 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 4514.36 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185874.367588 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40629.25 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 4062.92 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189937.292191 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36566.32 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3656.63 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193593.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32909.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197193.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29309.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200793.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25709.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204393.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22109.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207993.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18509.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211593.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14909.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215193.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11309.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218793.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7709.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222393.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4109.69 sec) 14:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225993.924333 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509.69 sec) 14:48:00 [GPUGRID] [rr_sim] 2226503.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979169281.06G/439.78G) 14:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852824.53 14:48:00 [---] pick_jobs_to_run() start 14:48:00 [---] pick_jobs_to_run() end 14:48:00 [---] [rr_sim] end 14:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:48:00 [---] [cpu_sched_debug] preliminary job list: 14:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:48:00 [---] [cpu_sched_debug] final job list: 14:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:48:00 [---] [cpu_sched_debug] enforce_run_list: end 14:48:00 [---] choose_project(): 53280.000000 14:48:00 [WUProp@Home] doing RPC to get work 14:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382319.09 cpu 2129372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129372.90 14:48:00 [---] Got 0 tasks 14:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:49:00 [---] [rr_sim] doing sim: CPU sched 14:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979141690.70G/439.78G) 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226440.88 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 222644.09 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222644.087673 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003796.79 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 200379.68 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423023.766579 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803417.11 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 180341.71 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603365.477595 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623075.40 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 162307.54 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765673.017508 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460767.86 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 146076.79 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911749.803431 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314691.07 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 131469.11 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043218.910761 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183221.97 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 118322.20 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161541.107358 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064899.77 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 106489.98 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1268031.084296 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958409.79 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 95840.98 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363872.063539 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862568.81 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 86256.88 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450128.944859 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776311.93 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 77631.19 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527760.138046 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698680.74 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 69868.07 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597628.211915 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628812.66 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 62881.27 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660509.478396 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565931.40 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 56593.14 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717102.618230 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509338.26 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 50933.83 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1768036.444080 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458404.43 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 45840.44 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813876.887346 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412563.99 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 41256.40 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855133.286284 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371307.59 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 37130.76 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892264.045329 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334176.83 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 33417.68 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925681.728469 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300759.15 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 30075.91 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955757.643296 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270683.23 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 27068.32 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982825.966639 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243614.91 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 24361.49 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007187.457649 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219253.42 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 21925.34 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029112.799557 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197328.08 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 19732.81 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048845.607275 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177595.27 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 17759.53 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066605.134221 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159835.74 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 15983.57 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082588.708472 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143852.17 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 14385.22 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096973.925298 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129466.95 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 12946.70 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109920.620441 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116520.26 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 11652.03 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121572.646070 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104868.23 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 10486.82 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132059.469137 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94381.41 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 9438.14 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141497.609896 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84943.27 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 8494.33 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149991.936580 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76448.94 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 7644.89 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157636.830595 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68804.05 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 6880.40 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164517.235209 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61923.64 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 6192.36 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170709.599361 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55731.28 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 5573.13 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176282.727098 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50158.15 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 5015.81 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181298.542062 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45142.33 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 4514.23 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185812.775529 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40628.10 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 4062.81 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189875.585649 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36565.29 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3656.53 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193532.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32908.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197132.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29308.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200732.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25708.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204332.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22108.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207932.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18508.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211532.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14908.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215132.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11308.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218732.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7708.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222332.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4108.76 sec) 14:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225932.114758 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508.76 sec) 14:49:00 [GPUGRID] [rr_sim] 2226440.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979141690.70G/439.78G) 14:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852821.79 14:49:00 [---] pick_jobs_to_run() start 14:49:00 [---] pick_jobs_to_run() end 14:49:00 [---] [rr_sim] end 14:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:49:00 [---] [cpu_sched_debug] preliminary job list: 14:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:49:00 [---] [cpu_sched_debug] final job list: 14:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:49:00 [---] [cpu_sched_debug] enforce_run_list: end 14:49:00 [---] choose_project(): 53340.000000 14:49:00 [WUProp@Home] doing RPC to get work 14:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382259.09 cpu 2129312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129312.90 14:49:00 [---] Got 0 tasks 14:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:50:00 [---] [rr_sim] doing sim: CPU sched 14:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979114100.35G/439.78G) 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226378.14 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 222637.81 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222637.813985 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003740.33 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 200374.03 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 423011.846572 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803366.29 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 180336.63 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603348.475899 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1623029.66 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 162302.97 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765651.442295 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460726.70 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 146072.67 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911724.112050 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314654.03 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 131465.40 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043189.514830 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183188.63 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 118318.86 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161508.377332 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064869.76 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 106486.98 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267995.353584 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958382.79 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 95838.28 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363833.632211 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862544.51 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 86254.45 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450088.082975 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776290.06 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 77629.01 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527717.088662 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698661.05 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 69866.11 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597583.193781 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628794.95 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 62879.49 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660462.688388 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565915.45 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 56591.55 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717054.233534 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509323.91 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 50932.39 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767986.624166 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458391.52 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 45839.15 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813825.775734 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412552.36 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 41255.24 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855081.012146 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371297.13 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 37129.71 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892210.724916 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334167.41 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 33416.74 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925627.466410 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300750.67 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 30075.07 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955702.533754 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270675.61 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 27067.56 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982770.094364 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243608.05 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 24360.80 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007130.898912 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219247.24 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 21924.72 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029055.623006 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197322.52 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 19732.25 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048787.874691 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177590.27 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 17759.03 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066546.901207 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159831.24 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 15983.12 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082530.025071 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143848.11 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 14384.81 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096914.836549 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129463.30 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 12946.33 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109861.166879 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116516.97 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 11651.70 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121512.864176 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104865.28 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 10486.53 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131999.391744 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94378.75 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 9437.87 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141437.266554 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84940.87 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 8494.09 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149931.353884 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76446.79 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 7644.68 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157576.032481 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68802.11 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 6880.21 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164456.243218 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61921.90 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 6192.19 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170648.432881 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55729.71 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 5572.97 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176221.403578 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50156.74 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 5015.67 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181237.077205 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45141.06 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 4514.11 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185751.183470 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40626.96 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 4062.70 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189813.879108 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36564.26 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3656.43 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193470.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32907.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197070.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29307.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200670.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25707.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204270.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22107.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207870.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18507.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211470.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14907.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215070.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11307.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218670.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7707.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222270.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4107.83 sec) 14:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225870.305182 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507.83 sec) 14:50:00 [GPUGRID] [rr_sim] 2226378.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979114100.35G/439.78G) 14:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852819.05 14:50:00 [---] pick_jobs_to_run() start 14:50:00 [---] pick_jobs_to_run() end 14:50:00 [---] [rr_sim] end 14:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:50:00 [---] [cpu_sched_debug] preliminary job list: 14:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:50:00 [---] [cpu_sched_debug] final job list: 14:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:50:00 [---] [cpu_sched_debug] enforce_run_list: end 14:50:00 [---] choose_project(): 53400.000000 14:50:00 [WUProp@Home] doing RPC to get work 14:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382199.09 cpu 2129252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129252.90 14:50:00 [---] Got 0 tasks 14:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:51:00 [---] [rr_sim] doing sim: CPU sched 14:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979086509.99G/439.78G) 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226315.40 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 222631.54 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222631.540297 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003683.86 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 200368.39 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422999.926564 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803315.48 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 180331.55 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603331.474204 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622983.93 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 162298.39 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765629.867081 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460685.54 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 146068.55 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911698.420670 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314616.98 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 131461.70 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043160.118899 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183155.28 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 118315.53 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161475.647306 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064839.76 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 106483.98 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267959.622873 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958355.78 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 95835.58 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363795.200882 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862520.20 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 86252.02 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450047.221091 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776268.18 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 77626.82 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527674.039278 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698641.36 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 69864.14 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597538.175647 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628777.23 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 62877.72 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660415.898380 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565899.50 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 56589.95 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1717005.848838 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509309.55 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 50930.96 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767936.804251 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458378.60 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 45837.86 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813774.664123 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412540.74 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 41254.07 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1855028.738008 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371286.66 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 37128.67 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892157.404504 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334158.00 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 33415.80 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925573.204350 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300742.20 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 30074.22 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955647.424212 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270667.98 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 27066.80 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982714.222088 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243601.18 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 24360.12 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007074.340176 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219241.06 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 21924.11 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028998.446455 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197316.96 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 19731.70 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048730.142106 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177585.26 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 17758.53 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066488.668192 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159826.73 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 15982.67 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082471.341670 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143844.06 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 14384.41 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096855.747800 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129459.66 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 12945.97 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109801.713317 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116513.69 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 11651.37 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121453.082282 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104862.32 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 10486.23 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131939.314351 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94376.09 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 9437.61 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141376.923212 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84938.48 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 8493.85 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149870.771188 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76444.63 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 7644.46 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157515.234366 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68800.17 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 6880.02 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164395.251226 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61920.15 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 6192.02 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170587.266400 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55728.14 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 5572.81 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176160.080057 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50155.32 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 5015.53 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181175.612348 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45139.79 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 4513.98 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185689.591410 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40625.81 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 4062.58 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189752.172566 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36563.23 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3656.32 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193408.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32906.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197008.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29306.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200608.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25706.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204208.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22106.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207808.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18506.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211408.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14906.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215008.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11306.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218608.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7706.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222208.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4106.91 sec) 14:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225808.495606 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506.91 sec) 14:51:00 [GPUGRID] [rr_sim] 2226315.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979086509.99G/439.78G) 14:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852816.32 14:51:00 [---] pick_jobs_to_run() start 14:51:00 [---] pick_jobs_to_run() end 14:51:00 [---] [rr_sim] end 14:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:51:00 [---] [cpu_sched_debug] preliminary job list: 14:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:51:00 [---] [cpu_sched_debug] final job list: 14:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:51:00 [---] [cpu_sched_debug] enforce_run_list: end 14:51:00 [---] choose_project(): 53460.000000 14:51:00 [WUProp@Home] doing RPC to get work 14:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382139.09 cpu 2129192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129192.90 14:51:00 [---] Got 0 tasks 14:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:52:00 [---] [rr_sim] doing sim: CPU sched 14:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979058919.64G/439.78G) 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226252.67 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 222625.27 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222625.266609 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003627.40 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 200362.74 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422988.006556 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803264.66 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 180326.47 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603314.472509 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622938.19 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 162293.82 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765608.291867 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460644.37 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 146064.44 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911672.729289 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314579.94 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 131457.99 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043130.722969 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183121.94 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 118312.19 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161442.917280 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064809.75 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 106480.97 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267923.892161 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958328.77 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 95832.88 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363756.769554 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862495.90 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 86249.59 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1450006.359207 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776246.31 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 77624.63 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527630.989895 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698621.68 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 69862.17 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597493.157514 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628759.51 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 62875.95 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660369.108371 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565883.56 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 56588.36 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716957.464143 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509295.20 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 50929.52 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767886.984337 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458365.68 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 45836.57 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813723.552512 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412529.11 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 41252.91 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854976.463869 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371276.20 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 37127.62 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892104.084091 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334148.58 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 33414.86 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925518.942291 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300733.72 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 30073.37 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955592.314670 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270660.35 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 27066.04 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982658.349812 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243594.32 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 24359.43 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007017.781439 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219234.88 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 21923.49 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028941.269904 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197311.40 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 19731.14 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048672.409522 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177580.26 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 17758.03 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066430.435178 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159822.23 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 15982.22 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082412.658269 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143840.01 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 14384.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096796.659051 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129456.01 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 12945.60 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109742.259754 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116510.41 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 11651.04 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121393.300388 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104859.37 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 10485.94 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131879.236957 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94373.43 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 9437.34 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141316.579870 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84936.09 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 8493.61 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149810.188492 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76442.48 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 7644.25 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157454.436251 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68798.23 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 6879.82 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164334.259235 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61918.41 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 6191.84 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170526.099920 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55726.57 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 5572.66 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176098.756537 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50153.91 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 5015.39 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181114.147492 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45138.52 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 4513.85 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185627.999351 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40624.67 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 4062.47 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189690.466025 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36562.20 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3656.22 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193346.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32905.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196946.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29305.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200546.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25705.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204146.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22105.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207746.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18505.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211346.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14905.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214946.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11305.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218546.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7705.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222146.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4105.98 sec) 14:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225746.686031 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505.98 sec) 14:52:00 [GPUGRID] [rr_sim] 2226252.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979058919.64G/439.78G) 14:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852813.58 14:52:00 [---] pick_jobs_to_run() start 14:52:00 [---] pick_jobs_to_run() end 14:52:00 [---] [rr_sim] end 14:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:52:00 [---] [cpu_sched_debug] preliminary job list: 14:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:52:00 [---] [cpu_sched_debug] final job list: 14:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:52:00 [---] [cpu_sched_debug] enforce_run_list: end 14:52:00 [---] choose_project(): 53520.000000 14:52:00 [WUProp@Home] doing RPC to get work 14:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382079.09 cpu 2129132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129132.90 14:52:00 [---] Got 0 tasks 14:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:53:00 [---] [rr_sim] doing sim: CPU sched 14:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979031329.28G/439.78G) 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226189.93 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 222618.99 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222618.992920 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003570.94 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 200357.09 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422976.086549 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803213.84 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 180321.38 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603297.470814 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622892.46 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 162289.25 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765586.716653 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460603.21 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 146060.32 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911647.037908 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314542.89 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 131454.29 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043101.327038 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183088.60 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 118308.86 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161410.187255 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064779.74 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 106477.97 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267888.161449 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958301.77 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 95830.18 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363718.338225 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862471.59 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 86247.16 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449965.497323 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776224.43 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 77622.44 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527587.940511 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698601.99 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 69860.20 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597448.139380 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628741.79 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 62874.18 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660322.318363 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565867.61 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 56586.76 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716909.079447 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509280.85 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 50928.08 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767837.164422 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458352.76 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 45835.28 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813672.440901 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412517.49 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 41251.75 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854924.189731 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371265.74 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 37126.57 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1892050.763678 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334139.17 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 33413.92 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925464.680231 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300725.25 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 30072.52 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955537.205128 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270652.72 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 27065.27 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982602.477536 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243587.45 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 24358.75 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006961.222703 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219228.71 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 21922.87 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028884.093353 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197305.84 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 19730.58 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048614.676938 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177575.25 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 17757.53 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066372.202164 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159817.73 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 15981.77 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082353.974868 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143835.95 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 14383.60 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096737.570302 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129452.36 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 12945.24 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109682.806192 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116507.12 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 11650.71 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121333.518493 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104856.41 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 10485.64 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131819.159564 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94370.77 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 9437.08 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141256.236528 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84933.69 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 8493.37 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149749.605796 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76440.32 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 7644.03 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157393.638137 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68796.29 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 6879.63 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164273.267243 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61916.66 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 6191.67 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170464.933440 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55725.00 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 5572.50 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176037.433016 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50152.50 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 5015.25 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181052.682635 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45137.25 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 4513.72 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185566.407292 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40623.52 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 4062.35 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189628.759483 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36561.17 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3656.12 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193284.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32905.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196884.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29305.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200484.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25705.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204084.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22105.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207684.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18505.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211284.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14905.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214884.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11305.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218484.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7705.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222084.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4105.05 sec) 14:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225684.876455 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505.05 sec) 14:53:00 [GPUGRID] [rr_sim] 2226189.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979031329.28G/439.78G) 14:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852810.84 14:53:00 [---] pick_jobs_to_run() start 14:53:00 [---] pick_jobs_to_run() end 14:53:00 [---] [rr_sim] end 14:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:53:00 [---] [cpu_sched_debug] preliminary job list: 14:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:53:00 [---] [cpu_sched_debug] final job list: 14:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:53:00 [---] [cpu_sched_debug] enforce_run_list: end 14:53:00 [---] choose_project(): 53580.000000 14:53:00 [WUProp@Home] doing RPC to get work 14:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 382019.09 cpu 2129072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129072.90 14:53:00 [---] Got 0 tasks 14:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:54:00 [---] [rr_sim] doing sim: CPU sched 14:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (979003738.92G/439.78G) 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226127.19 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 222612.72 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222612.719232 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003514.47 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 200351.45 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422964.166541 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803163.03 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 180316.30 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603280.469119 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622846.72 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 162284.67 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765565.141439 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460562.05 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 146056.21 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911621.346528 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314505.85 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 131450.58 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043071.931107 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183055.26 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 118305.53 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161377.457229 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064749.74 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 106474.97 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267852.430738 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958274.76 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 95827.48 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363679.906896 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862447.29 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 86244.73 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449924.635439 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776202.56 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 77620.26 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527544.891127 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698582.30 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 69858.23 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597403.121247 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628724.07 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 62872.41 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660275.528354 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565851.66 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 56585.17 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716860.694751 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509266.50 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 50926.65 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767787.344508 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458339.85 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 45833.98 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813621.329289 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412505.86 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 41250.59 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854871.915593 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371255.28 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 37125.53 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891997.443266 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334129.75 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 33412.97 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925410.418171 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300716.77 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 30071.68 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955482.095586 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270645.10 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 27064.51 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982546.605260 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243580.59 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 24358.06 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006904.663966 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219222.53 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 21922.25 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028826.916802 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197300.28 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 19730.03 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048556.944354 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177570.25 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 17757.02 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066313.969150 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159813.22 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 15981.32 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082295.291468 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143831.90 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 14383.19 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096678.481553 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129448.71 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 12944.87 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109623.352630 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116503.84 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 11650.38 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121273.736599 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104853.46 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 10485.35 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131759.082171 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94368.11 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 9436.81 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141195.893186 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84931.30 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 8493.13 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149689.023100 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76438.17 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 7643.82 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157332.840022 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68794.35 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 6879.44 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164212.275252 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61914.92 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 6191.49 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170403.766959 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55723.43 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 5572.34 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175976.109495 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50151.08 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 5015.11 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180991.217778 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45135.97 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 4513.60 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185504.815232 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40622.38 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 4062.24 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189567.052941 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36560.14 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3656.01 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193223.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32904.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196823.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29304.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200423.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25704.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204023.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22104.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207623.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18504.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211223.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14904.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214823.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11304.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218423.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7704.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222023.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4104.13 sec) 14:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225623.066879 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504.13 sec) 14:54:00 [GPUGRID] [rr_sim] 2226127.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (979003738.92G/439.78G) 14:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852808.11 14:54:00 [---] pick_jobs_to_run() start 14:54:00 [---] pick_jobs_to_run() end 14:54:00 [---] [rr_sim] end 14:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:54:00 [---] [cpu_sched_debug] preliminary job list: 14:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:54:00 [---] [cpu_sched_debug] final job list: 14:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:54:00 [---] [cpu_sched_debug] enforce_run_list: end 14:54:00 [---] choose_project(): 53640.000000 14:54:00 [WUProp@Home] doing RPC to get work 14:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381959.09 cpu 2129012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2129012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2129012.90 14:54:00 [---] Got 0 tasks 14:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:55:00 [---] [rr_sim] doing sim: CPU sched 14:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978976148.57G/439.78G) 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226064.46 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 222606.45 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222606.445544 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003458.01 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 200345.80 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422952.246534 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803112.21 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 180311.22 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603263.467424 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622800.99 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 162280.10 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765543.566226 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460520.89 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 146052.09 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911595.655147 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314468.80 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 131446.88 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043042.535176 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1183021.92 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 118302.19 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161344.727203 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064719.73 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 106471.97 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267816.700026 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958247.76 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 95824.78 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363641.475568 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862422.98 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 86242.30 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449883.773555 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776180.68 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 77618.07 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527501.841743 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698562.61 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 69856.26 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597358.103113 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628706.35 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 62870.64 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660228.738346 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565835.72 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 56583.57 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716812.310055 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509252.15 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 50925.21 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767737.524594 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458326.93 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 45832.69 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813570.217678 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412494.24 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 41249.42 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854819.641454 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371244.81 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 37124.48 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891944.122853 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334120.33 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 33412.03 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925356.156112 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300708.30 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 30070.83 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955426.986044 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270637.47 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 27063.75 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982490.732984 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243573.72 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 24357.37 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006848.105229 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219216.35 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 21921.64 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028769.740250 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197294.72 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 19729.47 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048499.211769 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177565.24 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 17756.52 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066255.736136 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159808.72 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 15980.87 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082236.608067 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143827.85 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 14382.78 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096619.392804 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129445.06 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 12944.51 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109563.899068 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116500.56 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 11650.06 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121213.954705 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104850.50 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 10485.05 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131699.004778 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94365.45 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 9436.55 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141135.549844 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84928.91 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 8492.89 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149628.440404 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76436.02 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 7643.60 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157272.041908 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68792.41 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 6879.24 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164151.283261 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61913.17 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 6191.32 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170342.600479 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55721.85 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 5572.19 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175914.785975 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50149.67 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 5014.97 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180929.752921 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45134.70 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 4513.47 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185443.223173 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40621.23 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 4062.12 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189505.346400 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36559.11 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3655.91 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193161.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32903.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196761.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29303.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200361.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25703.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203961.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22103.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207561.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18503.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211161.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14903.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214761.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11303.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218361.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7703.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221961.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4103.20 sec) 14:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225561.257304 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503.20 sec) 14:55:00 [GPUGRID] [rr_sim] 2226064.46: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978976148.57G/439.78G) 14:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852805.37 14:55:00 [---] pick_jobs_to_run() start 14:55:00 [---] pick_jobs_to_run() end 14:55:00 [---] [rr_sim] end 14:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:55:00 [---] [cpu_sched_debug] preliminary job list: 14:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:55:00 [---] [cpu_sched_debug] final job list: 14:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:55:00 [---] [cpu_sched_debug] enforce_run_list: end 14:55:00 [---] choose_project(): 53700.000000 14:55:00 [WUProp@Home] doing RPC to get work 14:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381899.09 cpu 2128952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128952.90 14:55:00 [---] Got 0 tasks 14:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:56:00 [---] [rr_sim] doing sim: CPU sched 14:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978948558.21G/439.78G) 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2226001.72 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 222600.17 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222600.171856 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003401.55 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 200340.15 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422940.326526 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803061.39 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 180306.14 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603246.465729 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622755.25 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 162275.53 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765521.991012 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460479.73 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 146047.97 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911569.963766 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314431.75 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 131443.18 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1043013.139246 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182988.58 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 118298.86 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161311.997177 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064689.72 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 106468.97 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267780.969315 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958220.75 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 95822.07 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363603.044239 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862398.67 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 86239.87 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449842.911671 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776158.81 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 77615.88 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527458.792360 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698542.93 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 69854.29 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597313.084979 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628688.63 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 62868.86 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660181.948337 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565819.77 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 56581.98 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716763.925359 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509237.79 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 50923.78 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767687.704679 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458314.01 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 45831.40 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813519.106067 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412482.61 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 41248.26 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854767.367316 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371234.35 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 37123.44 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891890.802440 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334110.92 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 33411.09 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925301.894052 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300699.82 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 30069.98 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955371.876502 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270629.84 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 27062.98 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982434.860708 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243566.86 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 24356.69 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006791.546493 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219210.17 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 21921.02 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028712.563699 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197289.15 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 19728.92 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048441.479185 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177560.24 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 17756.02 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066197.503122 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159804.22 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 15980.42 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082177.924666 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143823.79 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 14382.38 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096560.304055 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129441.41 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 12944.14 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109504.445505 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116497.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 11649.73 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121154.172811 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104847.55 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 10484.75 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131638.927385 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94362.79 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 9436.28 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141075.206502 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84926.51 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 8492.65 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149567.857708 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76433.86 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 7643.39 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157211.243793 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68790.47 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 6879.05 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164090.291269 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61911.43 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 6191.14 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170281.433998 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55720.28 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 5572.03 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175853.462454 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50148.26 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 5014.83 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180868.288064 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45133.43 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 4513.34 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185381.631114 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40620.09 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 4062.01 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189443.639858 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36558.08 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3655.81 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193099.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32902.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196699.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29302.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200299.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25702.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203899.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22102.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207499.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18502.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211099.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14902.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214699.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11302.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218299.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7702.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221899.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4102.27 sec) 14:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225499.447728 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502.27 sec) 14:56:00 [GPUGRID] [rr_sim] 2226001.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978948558.21G/439.78G) 14:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852802.63 14:56:00 [---] pick_jobs_to_run() start 14:56:00 [---] pick_jobs_to_run() end 14:56:00 [---] [rr_sim] end 14:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:56:00 [---] [cpu_sched_debug] preliminary job list: 14:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:56:00 [---] [cpu_sched_debug] final job list: 14:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:56:00 [---] [cpu_sched_debug] enforce_run_list: end 14:56:00 [---] choose_project(): 53760.000000 14:56:00 [WUProp@Home] doing RPC to get work 14:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381839.09 cpu 2128892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128892.90 14:56:00 [---] Got 0 tasks 14:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:57:00 [---] [rr_sim] doing sim: CPU sched 14:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978920967.85G/439.78G) 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225938.98 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 222593.90 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222593.898168 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003345.08 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 200334.51 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422928.406518 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1803010.58 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 180301.06 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603229.464034 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622709.52 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 162270.95 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765500.415798 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460438.57 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 146043.86 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911544.272386 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314394.71 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 131439.47 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042983.743315 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182955.24 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 118295.52 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161279.267151 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064659.71 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 106465.97 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267745.238603 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958193.74 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 95819.37 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363564.612911 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862374.37 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 86237.44 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449802.049787 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776136.93 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 77613.69 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527415.742976 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698523.24 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 69852.32 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597268.066846 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628670.91 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 62867.09 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660135.158329 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565803.82 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 56580.38 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716715.540663 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509223.44 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 50922.34 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767637.884765 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458301.10 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 45830.11 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813467.994456 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412470.99 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 41247.10 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854715.093178 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371223.89 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 37122.39 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891837.482027 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334101.50 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 33410.15 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925247.631992 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300691.35 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 30069.13 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955316.766961 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270622.21 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 27062.22 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982378.988432 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243559.99 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 24356.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006734.987756 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219203.99 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 21920.40 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028655.387148 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197283.59 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 19728.36 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048383.746601 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177555.24 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 17755.52 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066139.270108 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159799.71 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 15979.97 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082119.241265 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143819.74 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 14381.97 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096501.215306 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129437.77 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 12943.78 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109444.991943 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116493.99 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 11649.40 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121094.390916 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104844.59 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 10484.46 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131578.849992 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94360.13 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 9436.01 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141014.863160 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84924.12 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 8492.41 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149507.275012 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76431.71 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 7643.17 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157150.445678 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68788.54 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 6878.85 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164029.299278 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61909.68 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 6190.97 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170220.267518 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55718.71 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 5571.87 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175792.138933 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50146.84 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 5014.68 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180806.823208 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45132.16 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 4513.22 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185320.039054 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40618.94 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 4061.89 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189381.933317 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36557.05 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3655.70 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193037.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32901.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196637.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29301.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200237.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25701.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203837.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22101.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207437.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18501.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211037.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14901.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214637.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11301.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218237.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7701.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221837.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4101.34 sec) 14:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225437.638152 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501.34 sec) 14:57:00 [GPUGRID] [rr_sim] 2225938.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978920967.85G/439.78G) 14:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852799.89 14:57:00 [---] pick_jobs_to_run() start 14:57:00 [---] pick_jobs_to_run() end 14:57:00 [---] [rr_sim] end 14:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:57:00 [---] [cpu_sched_debug] preliminary job list: 14:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:57:00 [---] [cpu_sched_debug] final job list: 14:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:57:00 [---] [cpu_sched_debug] enforce_run_list: end 14:57:00 [---] choose_project(): 53820.000000 14:57:00 [WUProp@Home] doing RPC to get work 14:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381779.09 cpu 2128832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128832.90 14:57:00 [---] Got 0 tasks 14:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:58:00 [---] [rr_sim] doing sim: CPU sched 14:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978893377.50G/439.78G) 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225876.24 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 222587.62 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222587.624479 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003288.62 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 200328.86 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422916.486511 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802959.76 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 180295.98 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603212.462339 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622663.78 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 162266.38 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765478.840584 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460397.40 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 146039.74 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911518.581005 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314357.66 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 131435.77 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042954.347384 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182921.90 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 118292.19 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161246.537125 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064629.71 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 106462.97 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267709.507892 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958166.74 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 95816.67 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363526.181582 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862350.06 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 86235.01 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449761.187903 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776115.06 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 77611.51 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527372.693592 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698503.55 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 69850.36 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597223.048712 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628653.20 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 62865.32 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660088.368320 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565787.88 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 56578.79 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716667.155968 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509209.09 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 50920.91 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767588.064850 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458288.18 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 45828.82 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813416.882844 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412459.36 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 41245.94 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854662.819039 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371213.43 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 37121.34 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891784.161615 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334092.08 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 33409.21 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925193.369933 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300682.87 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 30068.29 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955261.657419 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270614.59 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 27061.46 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982323.116156 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243553.13 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 24355.31 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006678.429020 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219197.82 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 21919.78 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028598.210597 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197278.03 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 19727.80 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048326.014017 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177550.23 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 17755.02 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066081.037094 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159795.21 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 15979.52 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082060.557864 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143815.69 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 14381.57 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096442.126557 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129434.12 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 12943.41 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109385.538381 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116490.71 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 11649.07 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121034.609022 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104841.64 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 10484.16 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131518.772599 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94357.47 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 9435.75 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140954.519819 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84921.72 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 8492.17 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149446.692316 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76429.55 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 7642.96 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157089.647564 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68786.60 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 6878.66 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163968.307287 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61907.94 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 6190.79 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170159.101037 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55717.14 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 5571.71 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175730.815413 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50145.43 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 5014.54 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180745.358351 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45130.89 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 4513.09 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185258.446995 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40617.80 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 4061.78 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189320.226775 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36556.02 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3655.60 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192975.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32900.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196575.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29300.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200175.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25700.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203775.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22100.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207375.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18500.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210975.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14900.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214575.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11300.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218175.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7700.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221775.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4100.42 sec) 14:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225375.828577 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 500.42 sec) 14:58:00 [GPUGRID] [rr_sim] 2225876.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978893377.50G/439.78G) 14:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852797.16 14:58:00 [---] pick_jobs_to_run() start 14:58:00 [---] pick_jobs_to_run() end 14:58:00 [---] [rr_sim] end 14:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:58:00 [---] [cpu_sched_debug] preliminary job list: 14:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:58:00 [---] [cpu_sched_debug] final job list: 14:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:58:00 [---] [cpu_sched_debug] enforce_run_list: end 14:58:00 [---] choose_project(): 53880.000000 14:58:00 [WUProp@Home] doing RPC to get work 14:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381719.09 cpu 2128772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128772.90 14:58:00 [---] Got 0 tasks 14:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 14:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 14:59:00 [---] [rr_sim] doing sim: CPU sched 14:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978865787.14G/439.78G) 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225813.51 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 222581.35 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222581.350791 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003232.16 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 200323.22 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422904.566503 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802908.94 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 180290.89 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603195.460644 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622618.05 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 162261.80 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765457.265371 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460356.24 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 146035.62 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911492.889625 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314320.62 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 131432.06 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042924.951453 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182888.56 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 118288.86 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161213.807099 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064599.70 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 106459.97 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267673.777180 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958139.73 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 95813.97 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363487.750253 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862325.76 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 86232.58 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449720.326019 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776093.18 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 77609.32 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527329.644208 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698483.86 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 69848.39 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597178.030579 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628635.48 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 62863.55 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1660041.578312 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565771.93 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 56577.19 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716618.771272 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509194.74 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 50919.47 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767538.244936 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458275.26 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 45827.53 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813365.771233 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412447.74 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 41244.77 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854610.544901 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371202.96 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 37120.30 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891730.841202 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334082.67 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 33408.27 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925139.107873 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300674.40 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 30067.44 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955206.547877 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270606.96 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 27060.70 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982267.243880 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243546.26 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 24354.63 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006621.870283 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219191.64 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 21919.16 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028541.034046 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197272.47 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 19727.25 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048268.281432 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177545.23 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 17754.52 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066022.804080 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159790.70 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 15979.07 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082001.874463 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143811.63 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 14381.16 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096383.037808 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129430.47 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 12943.05 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109326.084818 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116487.42 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 11648.74 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120974.827128 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104838.68 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 10483.87 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131458.695206 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94354.81 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 9435.48 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140894.176477 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84919.33 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 8491.93 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149386.109620 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76427.40 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 7642.74 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157028.849449 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68784.66 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 6878.47 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163907.315295 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61906.19 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 6190.62 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170097.934557 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55715.57 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 5571.56 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175669.491892 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50144.02 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 5014.40 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180683.893494 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45129.61 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 4512.96 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185196.854936 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40616.65 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 4061.67 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189258.520233 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36554.99 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3655.50 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192914.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32899.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196514.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29299.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200114.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25699.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203714.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22099.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207314.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18499.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210914.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14899.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214514.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11299.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218114.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7699.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221714.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4099.49 sec) 14:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225314.019001 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 499.49 sec) 14:59:00 [GPUGRID] [rr_sim] 2225813.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978865787.14G/439.78G) 14:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852794.42 14:59:00 [---] pick_jobs_to_run() start 14:59:00 [---] pick_jobs_to_run() end 14:59:00 [---] [rr_sim] end 14:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 14:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 14:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 14:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 14:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 14:59:00 [---] [cpu_sched_debug] preliminary job list: 14:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:59:00 [---] [cpu_sched_debug] final job list: 14:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 14:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 14:59:00 [---] [cpu_sched_debug] enforce_run_list: end 14:59:00 [---] choose_project(): 53940.000000 14:59:00 [WUProp@Home] doing RPC to get work 14:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381659.09 cpu 2128712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128712.90 14:59:00 [---] Got 0 tasks 15:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:00:00 [---] [rr_sim] doing sim: CPU sched 15:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978838196.79G/439.78G) 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225750.77 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 222575.08 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222575.077103 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003175.69 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 200317.57 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422892.646495 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802858.12 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 180285.81 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603178.458949 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622572.31 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 162257.23 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765435.690157 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460315.08 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 146031.51 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911467.198244 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314283.57 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 131428.36 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042895.555523 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182855.22 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 118285.52 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161181.077073 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064569.69 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 106456.97 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267638.046469 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958112.72 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 95811.27 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363449.318925 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862301.45 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 86230.15 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449679.464135 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776071.31 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 77607.13 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527286.594825 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698464.18 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 69846.42 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597133.012445 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628617.76 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 62861.78 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659994.788303 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565755.98 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 56575.60 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716570.386576 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509180.38 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 50918.04 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767488.425021 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458262.35 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 45826.23 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813314.659622 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412436.11 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 41243.61 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854558.270763 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371192.50 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 37119.25 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891677.520789 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334073.25 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 33407.33 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925084.845813 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300665.93 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 30066.59 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955151.438335 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270599.33 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 27059.93 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982211.371604 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243539.40 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 24353.94 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006565.311547 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219185.46 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 21918.55 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028483.857495 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197266.91 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 19726.69 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048210.548848 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177540.22 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 17754.02 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065964.571066 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159786.20 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 15978.62 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081943.191063 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143807.58 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 14380.76 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096323.949059 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129426.82 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 12942.68 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109266.631256 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116484.14 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 11648.41 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120915.045233 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104835.73 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 10483.57 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131398.617813 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94352.15 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 9435.22 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140833.833135 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84916.94 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 8491.69 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149325.526924 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76425.24 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 7642.52 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156968.051334 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68782.72 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 6878.27 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163846.323304 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61904.45 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 6190.44 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170036.768076 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55714.00 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 5571.40 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175608.168372 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50142.60 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 5014.26 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180622.428637 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45128.34 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 4512.83 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185135.262877 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40615.51 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 4061.55 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189196.813692 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36553.96 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3655.40 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192852.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32898.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196452.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29298.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200052.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25698.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203652.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22098.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207252.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18498.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210852.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14898.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214452.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11298.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218052.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7698.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221652.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4098.56 sec) 15:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225252.209425 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 498.56 sec) 15:00:00 [GPUGRID] [rr_sim] 2225750.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978838196.79G/439.78G) 15:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852791.68 15:00:00 [---] pick_jobs_to_run() start 15:00:00 [---] pick_jobs_to_run() end 15:00:00 [---] [rr_sim] end 15:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:00:00 [---] [cpu_sched_debug] preliminary job list: 15:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:00:00 [---] [cpu_sched_debug] final job list: 15:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:00:00 [---] [cpu_sched_debug] enforce_run_list: end 15:00:00 [---] choose_project(): 54000.000000 15:00:00 [WUProp@Home] doing RPC to get work 15:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381599.09 cpu 2128652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128652.90 15:00:00 [---] Got 0 tasks 15:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:01:00 [---] [rr_sim] doing sim: CPU sched 15:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978810606.43G/439.78G) 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225688.03 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 222568.80 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222568.803415 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003119.23 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 200311.92 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422880.726488 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802807.31 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 180280.73 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603161.457254 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622526.58 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 162252.66 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765414.114943 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460273.92 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 146027.39 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911441.506863 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314246.53 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 131424.65 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042866.159592 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182821.87 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 118282.19 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161148.347047 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064539.69 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 106453.97 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267602.315757 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958085.72 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 95808.57 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363410.887596 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862277.15 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 86227.71 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449638.602251 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776049.43 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 77604.94 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527243.545441 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698444.49 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 69844.45 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597087.994311 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628600.04 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 62860.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659947.998295 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565740.04 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 56574.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716522.001880 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509166.03 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 50916.60 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767438.605107 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458249.43 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 45824.94 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813263.548011 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412424.49 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 41242.45 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854505.996624 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371182.04 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 37118.20 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891624.200377 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334063.83 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 33406.38 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925030.583754 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300657.45 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 30065.75 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955096.328793 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270591.71 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 27059.17 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982155.499328 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243532.53 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 24353.25 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006508.752810 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219179.28 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 21917.93 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028426.680944 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197261.35 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 19726.14 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048152.816264 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177535.22 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 17753.52 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065906.338052 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159781.70 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 15978.17 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081884.507662 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143803.53 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 14380.35 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096264.860310 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129423.17 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 12942.32 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109207.177694 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116480.86 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 11648.09 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120855.263339 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104832.77 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 10483.28 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131338.540420 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94349.49 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 9434.95 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140773.489793 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84914.54 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 8491.45 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149264.944228 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76423.09 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 7642.31 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156907.253220 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68780.78 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 6878.08 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163785.331313 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61902.70 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 6190.27 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169975.601596 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55712.43 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 5571.24 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175546.844851 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50141.19 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 5014.12 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180560.963781 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45127.07 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 4512.71 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185073.670817 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40614.36 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 4061.44 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189135.107150 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36552.93 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3655.29 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192790.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32897.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196390.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29297.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199990.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25697.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203590.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22097.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207190.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18497.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210790.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14897.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214390.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11297.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217990.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7697.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221590.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4097.63 sec) 15:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225190.399850 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 497.63 sec) 15:01:00 [GPUGRID] [rr_sim] 2225688.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978810606.43G/439.78G) 15:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852788.95 15:01:00 [---] pick_jobs_to_run() start 15:01:00 [---] pick_jobs_to_run() end 15:01:00 [---] [rr_sim] end 15:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:01:00 [---] [cpu_sched_debug] preliminary job list: 15:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:01:00 [---] [cpu_sched_debug] final job list: 15:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:01:00 [---] [cpu_sched_debug] enforce_run_list: end 15:01:00 [---] choose_project(): 54060.000000 15:01:00 [WUProp@Home] doing RPC to get work 15:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381539.09 cpu 2128592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128592.90 15:01:00 [---] Got 0 tasks 15:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:02:00 [---] [rr_sim] doing sim: CPU sched 15:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978783016.07G/439.78G) 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225625.30 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 222562.53 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222562.529726 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003062.77 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 200306.28 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422868.806480 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802756.49 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 180275.65 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603144.455559 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622480.84 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 162248.08 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765392.539729 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460232.76 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 146023.28 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911415.815483 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314209.48 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 131420.95 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042836.763661 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182788.53 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 118278.85 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161115.617021 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064509.68 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 106450.97 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267566.585046 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958058.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 95805.87 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363372.456268 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862252.84 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 86225.28 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449597.740367 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776027.56 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 77602.76 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527200.496057 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698424.80 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 69842.48 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1597042.976178 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628582.32 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 62858.23 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659901.208286 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565724.09 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 56572.41 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716473.617184 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509151.68 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 50915.17 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767388.785192 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458236.51 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 45823.65 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813212.436399 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412412.86 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 41241.29 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854453.722486 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371171.57 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 37117.16 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891570.879964 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334054.42 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 33405.44 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924976.321694 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300648.98 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 30064.90 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955041.219251 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270584.08 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 27058.41 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982099.627052 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243525.67 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 24352.57 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006452.194074 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219173.10 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 21917.31 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028369.504393 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197255.79 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 19725.58 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048095.083680 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177530.21 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 17753.02 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065848.105038 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159777.19 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 15977.72 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081825.824261 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143799.47 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 14379.95 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096205.771561 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129419.53 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 12941.95 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109147.724132 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116477.57 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 11647.76 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120795.481445 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104829.82 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 10482.98 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131278.463027 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94346.83 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 9434.68 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140713.146451 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84912.15 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 8491.22 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149204.361532 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76420.94 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 7642.09 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156846.455105 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68778.84 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 6877.88 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163724.339321 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61900.96 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 6190.10 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169914.435116 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55710.86 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 5571.09 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175485.521330 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50139.78 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 5013.98 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180499.498924 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45125.80 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 4512.58 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185012.078758 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40613.22 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 4061.32 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189073.400609 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36551.90 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3655.19 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192728.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32896.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196328.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29296.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199928.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25696.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203528.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22096.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207128.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18496.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210728.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14896.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214328.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11296.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217928.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7696.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221528.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4096.71 sec) 15:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225128.590274 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 496.71 sec) 15:02:00 [GPUGRID] [rr_sim] 2225625.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978783016.07G/439.78G) 15:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852786.21 15:02:00 [---] pick_jobs_to_run() start 15:02:00 [---] pick_jobs_to_run() end 15:02:00 [---] [rr_sim] end 15:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:02:00 [---] [cpu_sched_debug] preliminary job list: 15:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:02:00 [---] [cpu_sched_debug] final job list: 15:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:02:00 [---] [cpu_sched_debug] enforce_run_list: end 15:02:00 [---] choose_project(): 54120.000000 15:02:00 [WUProp@Home] doing RPC to get work 15:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381479.09 cpu 2128532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128532.90 15:02:00 [---] Got 0 tasks 15:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:03:00 [---] [rr_sim] doing sim: CPU sched 15:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978755425.72G/439.78G) 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225562.56 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 222556.26 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222556.256038 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2003006.30 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 200300.63 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422856.886473 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802705.67 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 180270.57 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603127.453864 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622435.11 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 162243.51 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765370.964515 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460191.60 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 146019.16 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911390.124102 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314172.44 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 131417.24 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042807.367730 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182755.19 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 118275.52 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161082.886995 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064479.67 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 106447.97 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267530.854334 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958031.71 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 95803.17 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363334.024939 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862228.54 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 86222.85 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449556.878483 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 776005.68 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 77600.57 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527157.446673 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698405.11 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 69840.51 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596997.958044 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628564.60 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 62856.46 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659854.418278 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565708.14 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 56570.81 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716425.232488 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509137.33 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 50913.73 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767338.965278 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458223.60 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 45822.36 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813161.324788 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412401.24 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 41240.12 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854401.448348 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371161.11 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 37116.11 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891517.559551 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334045.00 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 33404.50 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924922.059634 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300640.50 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 30064.05 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954986.109709 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270576.45 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 27057.65 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982043.754776 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243518.81 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 24351.88 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006395.635337 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219166.93 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 21916.69 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028312.327842 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197250.23 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 19725.02 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048037.351096 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177525.21 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 17752.52 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065789.872024 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159772.69 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 15977.27 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081767.140860 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143795.42 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 14379.54 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096146.682812 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129415.88 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 12941.59 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109088.270569 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116474.29 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 11647.43 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120735.699551 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104826.86 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 10482.69 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131218.385634 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94344.17 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 9434.42 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140652.803109 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84909.76 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 8490.98 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149143.778836 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76418.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 7641.88 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156785.656991 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68776.90 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 6877.69 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163663.347330 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61899.21 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 6189.92 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169853.268635 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55709.29 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 5570.93 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175424.197810 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50138.36 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 5013.84 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180438.034067 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45124.53 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 4512.45 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184950.486699 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40612.07 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 4061.21 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189011.694067 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36550.87 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3655.09 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192666.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32895.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196266.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29295.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199866.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25695.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203466.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22095.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207066.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18495.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210666.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14895.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214266.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11295.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217866.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7695.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221466.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4095.78 sec) 15:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225066.780698 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 495.78 sec) 15:03:00 [GPUGRID] [rr_sim] 2225562.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978755425.72G/439.78G) 15:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852783.47 15:03:00 [---] pick_jobs_to_run() start 15:03:00 [---] pick_jobs_to_run() end 15:03:00 [---] [rr_sim] end 15:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:03:00 [---] [cpu_sched_debug] preliminary job list: 15:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:03:00 [---] [cpu_sched_debug] final job list: 15:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:03:00 [---] [cpu_sched_debug] enforce_run_list: end 15:03:00 [---] choose_project(): 54180.000000 15:03:00 [WUProp@Home] doing RPC to get work 15:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381419.09 cpu 2128472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128472.90 15:03:00 [---] Got 0 tasks 15:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:04:00 [---] [rr_sim] doing sim: CPU sched 15:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978727835.36G/439.78G) 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225499.82 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 222549.98 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222549.982350 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002949.84 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 200294.98 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422844.966465 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802654.86 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 180265.49 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603110.452169 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622389.37 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 162238.94 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765349.389302 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460150.43 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 146015.04 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911364.432722 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314135.39 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 131413.54 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042777.971799 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182721.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 118272.19 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161050.156969 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064449.67 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 106444.97 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267495.123623 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 958004.70 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 95800.47 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363295.593610 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862204.23 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 86220.42 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449516.016599 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775983.81 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 77598.38 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527114.397289 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698385.43 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 69838.54 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596952.939910 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628546.88 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 62854.69 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659807.628269 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565692.20 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 56569.22 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716376.847793 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509122.98 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 50912.30 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767289.145363 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458210.68 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 45821.07 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813110.213177 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412389.61 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 41238.96 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854349.174209 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371150.65 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 37115.06 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891464.239138 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334035.58 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 33403.56 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924867.797575 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300632.03 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 30063.20 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954931.000167 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270568.82 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 27056.88 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981987.882500 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243511.94 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 24351.19 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006339.076600 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219160.75 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 21916.07 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028255.151290 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197244.67 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 19724.47 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047979.618511 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177520.20 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 17752.02 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065731.639010 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159768.18 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 15976.82 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081708.457459 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143791.37 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 14379.14 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096087.594063 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129412.23 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 12941.22 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109028.817007 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116471.01 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 11647.10 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120675.917656 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104823.91 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 10482.39 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131158.308241 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94341.52 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 9434.15 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140592.459767 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84907.36 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 8490.74 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149083.196140 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76416.63 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 7641.66 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156724.858876 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68774.96 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 6877.50 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163602.355338 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61897.47 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 6189.75 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169792.102155 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55707.72 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 5570.77 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175362.874289 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50136.95 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 5013.69 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180376.569210 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45123.25 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 4512.33 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184888.894639 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40610.93 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 4061.09 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188949.987525 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36549.84 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3654.98 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192604.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32894.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196204.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29294.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199804.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25694.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203404.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22094.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207004.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18494.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210604.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14894.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214204.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11294.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217804.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7694.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221404.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4094.85 sec) 15:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2225004.971123 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 494.85 sec) 15:04:00 [GPUGRID] [rr_sim] 2225499.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978727835.36G/439.78G) 15:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852780.74 15:04:00 [---] pick_jobs_to_run() start 15:04:00 [---] pick_jobs_to_run() end 15:04:00 [---] [rr_sim] end 15:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:04:00 [---] [cpu_sched_debug] preliminary job list: 15:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:04:00 [---] [cpu_sched_debug] final job list: 15:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:04:00 [---] [cpu_sched_debug] enforce_run_list: end 15:04:00 [---] choose_project(): 54240.000000 15:04:00 [WUProp@Home] doing RPC to get work 15:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381359.09 cpu 2128412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128412.90 15:04:00 [---] Got 0 tasks 15:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:05:00 [---] [rr_sim] doing sim: CPU sched 15:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978700245.00G/439.78G) 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225437.09 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 222543.71 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222543.708662 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002893.38 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 200289.34 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422833.046457 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802604.04 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 180260.40 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603093.450473 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622343.64 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 162234.36 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765327.814088 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460109.27 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 146010.93 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911338.741341 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314098.35 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 131409.83 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042748.575869 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182688.51 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 118268.85 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1161017.426944 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064419.66 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 106441.97 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267459.392911 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957977.69 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 95797.77 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363257.162282 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862179.92 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 86217.99 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449475.154715 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775961.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 77596.19 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527071.347906 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698365.74 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 69836.57 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596907.921777 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628529.16 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 62852.92 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659760.838261 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565676.25 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 56567.62 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716328.463097 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509108.62 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 50910.86 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767239.325449 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458197.76 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 45819.78 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813059.101566 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412377.99 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 41237.80 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854296.900071 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371140.19 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 37114.02 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891410.918726 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334026.17 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 33402.62 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924813.535515 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300623.55 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 30062.36 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954875.890625 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270561.20 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 27056.12 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981932.010224 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243505.08 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 24350.51 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006282.517864 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219154.57 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 21915.46 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028197.974739 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197239.11 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 19723.91 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047921.885927 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177515.20 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 17751.52 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065673.405996 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159763.68 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 15976.37 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081649.774058 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143787.31 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 14378.73 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096028.505314 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129408.58 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 12940.86 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108969.363445 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116467.72 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 11646.77 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120616.135762 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104820.95 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 10482.10 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131098.230848 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94338.86 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 9433.89 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140532.116425 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84904.97 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 8490.50 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149022.613444 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76414.47 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 7641.45 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156664.060761 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68773.03 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 6877.30 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163541.363347 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61895.72 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 6189.57 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169730.935674 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55706.15 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 5570.62 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175301.550769 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50135.54 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 5013.55 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180315.104354 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45121.98 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 4512.20 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184827.302580 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40609.78 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 4060.98 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188888.280984 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36548.81 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3654.88 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192543.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32893.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196143.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29293.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199743.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25693.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203343.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22093.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206943.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18493.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210543.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14893.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214143.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11293.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217743.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7693.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221343.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4093.93 sec) 15:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224943.161547 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 493.93 sec) 15:05:00 [GPUGRID] [rr_sim] 2225437.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978700245.00G/439.78G) 15:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852778.00 15:05:00 [---] pick_jobs_to_run() start 15:05:00 [---] pick_jobs_to_run() end 15:05:00 [---] [rr_sim] end 15:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:05:00 [---] [cpu_sched_debug] preliminary job list: 15:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:05:00 [---] [cpu_sched_debug] final job list: 15:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:05:00 [---] [cpu_sched_debug] enforce_run_list: end 15:05:00 [---] choose_project(): 54300.000000 15:05:00 [WUProp@Home] doing RPC to get work 15:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381299.09 cpu 2128352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128352.90 15:05:00 [---] Got 0 tasks 15:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:06:00 [---] [rr_sim] doing sim: CPU sched 15:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978672654.65G/439.78G) 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225374.35 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 222537.43 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222537.434974 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002836.91 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 200283.69 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422821.126450 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802553.22 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 180255.32 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603076.448778 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622297.90 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 162229.79 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765306.238874 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460068.11 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 146006.81 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911313.049960 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314061.30 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 131406.13 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042719.179938 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182655.17 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 118265.52 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160984.696918 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064389.65 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 106438.97 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267423.662199 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957950.69 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 95795.07 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363218.730953 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862155.62 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 86215.56 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449434.292831 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775940.06 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 77594.01 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1527028.298522 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698346.05 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 69834.61 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596862.903643 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628511.45 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 62851.14 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659714.048253 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565660.30 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 56566.03 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716280.078401 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509094.27 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 50909.43 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767189.505534 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458184.84 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 45818.48 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1813007.989954 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412366.36 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 41236.64 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854244.625933 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371129.72 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 37112.97 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891357.598313 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334016.75 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 33401.68 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924759.273455 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300615.08 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 30061.51 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954820.781083 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270553.57 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 27055.36 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981876.137949 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243498.21 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 24349.82 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006225.959127 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219148.39 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 21914.84 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028140.798188 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197233.55 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 19723.36 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047864.153343 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177510.20 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 17751.02 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065615.172982 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159759.18 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 15975.92 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081591.090658 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143783.26 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 14378.33 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095969.416565 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129404.93 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 12940.49 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108909.909882 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116464.44 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 11646.44 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120556.353868 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104818.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 10481.80 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131038.153455 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94336.20 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 9433.62 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140471.773083 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84902.58 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 8490.26 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148962.030748 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76412.32 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 7641.23 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156603.262647 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68771.09 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 6877.11 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163480.371356 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61893.98 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 6189.40 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169669.769194 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55704.58 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 5570.46 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175240.227248 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50134.12 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 5013.41 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180253.639497 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45120.71 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 4512.07 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184765.710521 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40608.64 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 4060.86 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188826.574442 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36547.78 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3654.78 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192481.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32893.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196081.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29293.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199681.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25693.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203281.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22093.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206881.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18493.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210481.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14893.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214081.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11293.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217681.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7693.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221281.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4093.00 sec) 15:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224881.351972 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 493.00 sec) 15:06:00 [GPUGRID] [rr_sim] 2225374.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978672654.65G/439.78G) 15:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852775.26 15:06:00 [---] pick_jobs_to_run() start 15:06:00 [---] pick_jobs_to_run() end 15:06:00 [---] [rr_sim] end 15:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:06:00 [---] [cpu_sched_debug] preliminary job list: 15:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:06:00 [---] [cpu_sched_debug] final job list: 15:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:06:00 [---] [cpu_sched_debug] enforce_run_list: end 15:06:00 [---] choose_project(): 54360.000000 15:06:00 [WUProp@Home] doing RPC to get work 15:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381239.09 cpu 2128292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128292.90 15:06:00 [---] Got 0 tasks 15:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:07:00 [---] [rr_sim] doing sim: CPU sched 15:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978645064.29G/439.78G) 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225311.61 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 222531.16 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222531.161285 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002780.45 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 200278.05 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422809.206442 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802502.41 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 180250.24 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603059.447083 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622252.17 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 162225.22 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765284.663660 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1460026.95 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 146002.69 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911287.358580 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1314024.25 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 131402.43 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042689.784007 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182621.83 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 118262.18 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160951.966892 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064359.65 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 106435.96 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267387.931488 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957923.68 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 95792.37 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363180.299625 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862131.31 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 86213.13 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449393.430947 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775918.18 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 77591.82 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526985.249138 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698326.36 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 69832.64 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596817.885510 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628493.73 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 62849.37 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659667.258244 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565644.35 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 56564.44 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716231.693705 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509079.92 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 50907.99 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767139.685620 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458171.93 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 45817.19 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812956.878343 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412354.73 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 41235.47 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854192.351794 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371119.26 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 37111.93 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891304.277900 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334007.33 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 33400.73 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924705.011396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300606.60 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 30060.66 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954765.671541 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270545.94 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 27054.59 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981820.265673 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243491.35 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 24349.13 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006169.400391 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219142.21 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 21914.22 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028083.621637 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197227.99 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 19722.80 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047806.420759 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177505.19 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 17750.52 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065556.939968 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159754.67 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 15975.47 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081532.407257 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143779.21 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 14377.92 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095910.327816 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129401.29 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 12940.13 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108850.456320 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116461.16 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 11646.12 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120496.571974 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104815.04 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 10481.50 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130978.076062 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94333.54 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 9433.35 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140411.429741 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84900.18 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 8490.02 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148901.448052 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76410.16 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 7641.02 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156542.464532 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68769.15 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 6876.91 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163419.379364 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61892.23 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 6189.22 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169608.602713 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55703.01 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 5570.30 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175178.903727 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50132.71 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 5013.27 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180192.174640 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45119.44 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 4511.94 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184704.118461 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40607.49 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 4060.75 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188764.867901 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36546.74 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3654.67 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192419.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32892.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196019.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29292.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199619.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25692.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203219.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22092.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206819.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18492.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210419.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14892.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214019.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11292.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217619.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7692.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221219.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4092.07 sec) 15:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224819.542396 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 492.07 sec) 15:07:00 [GPUGRID] [rr_sim] 2225311.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978645064.29G/439.78G) 15:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852772.53 15:07:00 [---] pick_jobs_to_run() start 15:07:00 [---] pick_jobs_to_run() end 15:07:00 [---] [rr_sim] end 15:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:07:00 [---] [cpu_sched_debug] preliminary job list: 15:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:07:00 [---] [cpu_sched_debug] final job list: 15:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:07:00 [---] [cpu_sched_debug] enforce_run_list: end 15:07:00 [---] choose_project(): 54420.000000 15:07:00 [WUProp@Home] doing RPC to get work 15:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381179.09 cpu 2128232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128232.90 15:07:00 [---] Got 0 tasks 15:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:08:00 [---] [rr_sim] doing sim: CPU sched 15:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978617473.94G/439.78G) 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225248.88 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 222524.89 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222524.887597 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002723.99 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 200272.40 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422797.286435 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802451.59 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 180245.16 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603042.445388 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622206.43 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 162220.64 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765263.088447 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459985.79 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 145998.58 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911261.667199 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313987.21 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 131398.72 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042660.388076 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182588.49 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 118258.85 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160919.236866 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064329.64 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 106432.96 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267352.200776 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957896.68 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 95789.67 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363141.868296 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862107.01 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 86210.70 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449352.569063 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775896.31 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 77589.63 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526942.199754 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698306.68 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 69830.67 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596772.867376 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628476.01 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 62847.60 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659620.468236 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565628.41 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 56562.84 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716183.309009 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509065.57 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 50906.56 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767089.865705 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458159.01 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 45815.90 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812905.766732 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412343.11 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 41234.31 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854140.077656 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371108.80 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 37110.88 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891250.957488 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333997.92 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 33399.79 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924650.749336 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300598.13 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 30059.81 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954710.561999 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270538.31 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 27053.83 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981764.393397 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243484.48 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 24348.45 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006112.841654 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219136.03 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 21913.60 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028026.445086 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197222.43 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 19722.24 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047748.688174 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177500.19 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 17750.02 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065498.706954 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159750.17 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 15975.02 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081473.723856 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143775.15 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 14377.52 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095851.239067 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129397.64 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 12939.76 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108791.002758 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116457.87 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 11645.79 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120436.790079 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104812.09 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 10481.21 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130917.998668 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94330.88 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 9433.09 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140351.086399 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84897.79 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 8489.78 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148840.865356 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76408.01 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 7640.80 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156481.666418 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68767.21 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 6876.72 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163358.387373 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61890.49 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 6189.05 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169547.436233 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55701.44 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 5570.14 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175117.580207 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50131.30 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 5013.13 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180130.709783 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45118.17 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 4511.82 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184642.526402 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40606.35 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 4060.63 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188703.161359 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36545.71 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3654.57 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192357.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32891.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195957.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29291.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199557.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25691.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203157.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22091.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206757.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18491.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210357.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14891.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213957.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11291.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217557.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7691.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221157.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4091.14 sec) 15:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224757.732820 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 491.14 sec) 15:08:00 [GPUGRID] [rr_sim] 2225248.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978617473.94G/439.78G) 15:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852769.79 15:08:00 [---] pick_jobs_to_run() start 15:08:00 [---] pick_jobs_to_run() end 15:08:00 [---] [rr_sim] end 15:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:08:00 [---] [cpu_sched_debug] preliminary job list: 15:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:08:00 [---] [cpu_sched_debug] final job list: 15:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:08:00 [---] [cpu_sched_debug] enforce_run_list: end 15:08:00 [---] choose_project(): 54480.000000 15:08:00 [WUProp@Home] doing RPC to get work 15:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 381119.09 cpu 2128172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128172.90 15:08:00 [---] Got 0 tasks 15:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:09:00 [---] [rr_sim] doing sim: CPU sched 15:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978589883.58G/439.78G) 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225186.14 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 222518.61 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222518.613909 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002667.53 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 200266.75 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422785.366427 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802400.77 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 180240.08 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603025.443693 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622160.70 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 162216.07 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765241.513233 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459944.63 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 145994.46 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911235.975818 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313950.16 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 131395.02 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042630.992146 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182555.15 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 118255.51 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160886.506840 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064299.63 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 106429.96 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267316.470065 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957869.67 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 95786.97 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363103.436967 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862082.70 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 86208.27 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449311.707180 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775874.43 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 77587.44 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526899.150371 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698286.99 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 69828.70 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596727.849242 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628458.29 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 62845.83 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659573.678227 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565612.46 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 56561.25 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716134.924313 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509051.21 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 50905.12 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1767040.045791 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458146.09 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 45814.61 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812854.655121 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412331.48 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 41233.15 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854087.803518 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371098.34 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 37109.83 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891197.637075 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333988.50 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 33398.85 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924596.487276 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300589.65 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 30058.97 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954655.452458 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270530.69 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 27053.07 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981708.521121 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243477.62 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 24347.76 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006056.282918 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219129.86 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 21912.99 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027969.268535 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197216.87 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 19721.69 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047690.955590 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177495.18 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 17749.52 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065440.473940 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159745.67 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 15974.57 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081415.040455 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143771.10 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 14377.11 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095792.150319 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129393.99 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 12939.40 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108731.549196 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116454.59 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 11645.46 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120377.008185 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104809.13 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 10480.91 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130857.921275 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94328.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 9432.82 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140290.743057 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84895.40 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 8489.54 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148780.282660 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76405.86 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 7640.59 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156420.868303 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68765.27 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 6876.53 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163297.395382 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61888.74 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 6188.87 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169486.269752 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55699.87 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 5569.99 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175056.256686 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50129.88 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 5012.99 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180069.244926 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45116.89 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 4511.69 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184580.934343 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40605.20 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 4060.52 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188641.454817 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36544.68 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3654.47 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192295.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32890.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195895.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29290.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199495.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25690.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203095.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22090.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206695.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18490.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210295.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14890.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213895.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11290.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217495.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7690.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221095.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4090.22 sec) 15:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224695.923245 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 490.22 sec) 15:09:00 [GPUGRID] [rr_sim] 2225186.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978589883.58G/439.78G) 15:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852767.05 15:09:00 [---] pick_jobs_to_run() start 15:09:00 [---] pick_jobs_to_run() end 15:09:00 [---] [rr_sim] end 15:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:09:00 [---] [cpu_sched_debug] preliminary job list: 15:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:09:00 [---] [cpu_sched_debug] final job list: 15:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:09:00 [---] [cpu_sched_debug] enforce_run_list: end 15:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:10:00 [---] [rr_sim] doing sim: CPU sched 15:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978562293.22G/439.78G) 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225123.40 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 222512.34 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222512.340221 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002611.06 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 200261.11 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422773.446419 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802349.96 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 180235.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 603008.441998 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622114.96 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 162211.50 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765219.938019 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459903.46 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 145990.35 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911210.284438 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313913.12 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 131391.31 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042601.596215 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182521.81 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 118252.18 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160853.776814 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064269.63 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 106426.96 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267280.739353 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957842.66 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 95784.27 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363065.005639 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862058.40 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 86205.84 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449270.845296 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775852.56 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 77585.26 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526856.100987 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698267.30 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 69826.73 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596682.831109 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628440.57 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 62844.06 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659526.888219 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565596.51 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 56559.65 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716086.539617 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509036.86 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 50903.69 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766990.225876 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458133.18 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 45813.32 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812803.543510 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412319.86 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 41231.99 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1854035.529379 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371087.87 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 37108.79 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891144.316662 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333979.09 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 33397.91 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924542.225217 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300581.18 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 30058.12 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954600.342916 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270523.06 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 27052.31 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981652.648845 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243470.75 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 24347.08 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005999.724181 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219123.68 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 21912.37 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027912.091984 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197211.31 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 19721.13 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047633.223006 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177490.18 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 17749.02 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065382.240926 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159741.16 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 15974.12 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081356.357054 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143767.05 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 14376.70 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095733.061570 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129390.34 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 12939.03 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108672.095633 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116451.31 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 11645.13 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120317.226291 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104806.18 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 10480.62 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130797.843882 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94325.56 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 9432.56 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140230.399715 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84893.00 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 8489.30 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148719.699964 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76403.70 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 7640.37 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156360.070188 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68763.33 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 6876.33 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163236.403390 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61887.00 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 6188.70 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169425.103272 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55698.30 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 5569.83 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174994.933165 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50128.47 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 5012.85 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180007.780070 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45115.62 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 4511.56 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184519.342283 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40604.06 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 4060.41 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188579.748276 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36543.65 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3654.37 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192234.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32889.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195834.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29289.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199434.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25689.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203034.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22089.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206634.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18489.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210234.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14889.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213834.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11289.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217434.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7689.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221034.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4089.29 sec) 15:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224634.113669 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 489.29 sec) 15:10:00 [GPUGRID] [rr_sim] 2225123.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978562293.22G/439.78G) 15:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852764.31 15:10:00 [---] pick_jobs_to_run() start 15:10:00 [---] pick_jobs_to_run() end 15:10:00 [---] [rr_sim] end 15:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:10:00 [---] [cpu_sched_debug] preliminary job list: 15:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:10:00 [---] [cpu_sched_debug] final job list: 15:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:10:00 [---] [cpu_sched_debug] enforce_run_list: end 15:10:00 [---] choose_project(): 54600.000000 15:10:00 [WUProp@Home] doing RPC to get work 15:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380999.09 cpu 2128052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2128052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2128052.90 15:10:00 [---] Got 0 tasks 15:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:11:00 [---] [rr_sim] doing sim: CPU sched 15:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978534702.87G/439.78G) 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2225060.67 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 222506.07 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222506.066533 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002554.60 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 200255.46 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422761.526412 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802299.14 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 180229.91 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602991.440303 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622069.23 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 162206.92 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765198.362805 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459862.30 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 145986.23 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911184.593057 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313876.07 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 131387.61 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042572.200284 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182488.47 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 118248.85 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160821.046788 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064239.62 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 106423.96 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267245.008642 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957815.66 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 95781.57 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1363026.574310 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862034.09 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 86203.41 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449229.983412 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775830.68 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 77583.07 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526813.051603 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698247.61 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 69824.76 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596637.812975 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628422.85 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 62842.29 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659480.098210 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565580.57 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 56558.06 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1716038.154922 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509022.51 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 50902.25 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766940.405962 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458120.26 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 45812.03 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812752.431898 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412308.23 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 41230.82 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853983.255241 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371077.41 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 37107.74 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891090.996249 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333969.67 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 33396.97 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924487.963157 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300572.70 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 30057.27 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954545.233374 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270515.43 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 27051.54 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981596.776569 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243463.89 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 24346.39 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005943.165444 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219117.50 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 21911.75 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027854.915433 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197205.75 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 19720.57 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047575.490422 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177485.17 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 17748.52 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065324.007912 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159736.66 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 15973.67 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081297.673653 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143762.99 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 14376.30 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095673.972821 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129386.69 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 12938.67 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108612.642071 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116448.02 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 11644.80 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120257.444396 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104803.22 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 10480.32 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130737.766489 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94322.90 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 9432.29 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140170.056373 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84890.61 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 8489.06 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148659.117268 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76401.55 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 7640.15 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156299.272074 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68761.39 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 6876.14 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163175.411399 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61885.25 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 6188.53 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169363.936792 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55696.73 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 5569.67 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174933.609645 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50127.06 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 5012.71 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179946.315213 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45114.35 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 4511.44 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184457.750224 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40602.92 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 4060.29 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188518.041734 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36542.62 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3654.26 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192172.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32888.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195772.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29288.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199372.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25688.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202972.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22088.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206572.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18488.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210172.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14888.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213772.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11288.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217372.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7688.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220972.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4088.36 sec) 15:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224572.304093 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 488.36 sec) 15:11:00 [GPUGRID] [rr_sim] 2225060.67: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978534702.87G/439.78G) 15:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852761.58 15:11:00 [---] pick_jobs_to_run() start 15:11:00 [---] pick_jobs_to_run() end 15:11:00 [---] [rr_sim] end 15:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:11:00 [---] [cpu_sched_debug] preliminary job list: 15:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:11:00 [---] [cpu_sched_debug] final job list: 15:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:11:00 [---] [cpu_sched_debug] enforce_run_list: end 15:11:00 [---] choose_project(): 54660.000000 15:11:00 [WUProp@Home] doing RPC to get work 15:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380939.09 cpu 2127992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127992.90 15:11:00 [---] Got 0 tasks 15:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:13:00 [---] [rr_sim] doing sim: CPU sched 15:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978507112.51G/439.78G) 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224997.93 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 222499.79 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222499.792844 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002498.14 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 200249.81 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422749.606404 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802248.32 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 180224.83 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602974.438608 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1622023.49 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 162202.35 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765176.787592 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459821.14 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 145982.11 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911158.901677 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313839.03 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 131383.90 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042542.804353 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182455.12 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 118245.51 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160788.316762 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064209.61 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 106420.96 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267209.277930 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957788.65 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 95778.87 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362988.142982 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 862009.79 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 86200.98 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449189.121528 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775808.81 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 77580.88 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526770.002219 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698227.93 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 69822.79 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596592.794842 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628405.13 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 62840.51 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659433.308202 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565564.62 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 56556.46 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715989.770226 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 509008.16 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 50900.82 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766890.586048 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458107.34 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 45810.73 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812701.320287 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412296.61 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 41229.66 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853930.981103 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371066.95 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 37106.69 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1891037.675837 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333960.25 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 33396.03 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924433.701097 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300564.23 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 30056.42 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954490.123832 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270507.80 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 27050.78 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981540.904293 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243457.02 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 24345.70 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005886.606708 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219111.32 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 21911.13 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027797.738881 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197200.19 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 19720.02 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047517.757838 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177480.17 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 17748.02 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065265.774898 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159732.15 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 15973.22 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081238.990253 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143758.94 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 14375.89 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095614.884072 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129383.04 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 12938.30 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108553.188509 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116444.74 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 11644.47 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120197.662502 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104800.27 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 10480.03 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130677.689096 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94320.24 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 9432.02 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140109.713031 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84888.22 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 8488.82 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148598.534572 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76399.39 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 7639.94 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156238.473959 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68759.45 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 6875.95 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163114.419408 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61883.51 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 6188.35 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169302.770311 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55695.16 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 5569.52 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174872.286124 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50125.64 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 5012.56 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179884.850356 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45113.08 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 4511.31 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184396.158165 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40601.77 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 4060.18 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188456.335193 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36541.59 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3654.16 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192110.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32887.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195710.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29287.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199310.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25687.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202910.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22087.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206510.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18487.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210110.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14887.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213710.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11287.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217310.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7687.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220910.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4087.43 sec) 15:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224510.494518 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 487.43 sec) 15:13:00 [GPUGRID] [rr_sim] 2224997.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978507112.51G/439.78G) 15:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852818.84 15:13:00 [---] pick_jobs_to_run() start 15:13:00 [---] pick_jobs_to_run() end 15:13:00 [---] [rr_sim] end 15:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:13:00 [---] [cpu_sched_debug] preliminary job list: 15:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:13:00 [---] [cpu_sched_debug] final job list: 15:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:13:00 [---] [cpu_sched_debug] enforce_run_list: end 15:13:00 [---] choose_project(): 54780.000000 15:13:00 [WUProp@Home] doing RPC to get work 15:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380819.09 cpu 2127932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127932.90 15:13:00 [---] Got 0 tasks 15:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:14:00 [---] [rr_sim] doing sim: CPU sched 15:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978479522.16G/439.78G) 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224935.19 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 222493.52 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222493.519156 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002441.67 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 200244.17 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422737.686397 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802197.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 180219.75 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602957.436913 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621977.75 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 162197.78 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765155.212378 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459779.98 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 145978.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911133.210296 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313801.98 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 131380.20 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042513.408422 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182421.78 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 118242.18 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160755.586736 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064179.60 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 106417.96 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267173.547219 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957761.64 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 95776.16 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362949.711653 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861985.48 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 86198.55 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449148.259644 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775786.93 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 77578.69 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526726.952835 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698208.24 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 69820.82 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596547.776708 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628387.41 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 62838.74 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659386.518193 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565548.67 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 56554.87 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715941.385530 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508993.81 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 50899.38 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766840.766133 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458094.43 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 45809.44 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812650.208676 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412284.98 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 41228.50 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853878.706964 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371056.48 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 37105.65 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890984.355424 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333950.84 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 33395.08 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924379.439038 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300555.75 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 30055.58 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954435.014290 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270500.18 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 27050.02 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981485.032017 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243450.16 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 24345.02 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005830.047971 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219105.14 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 21910.51 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027740.562330 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197194.63 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 19719.46 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047460.025253 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177475.17 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 17747.52 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065207.541884 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159727.65 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 15972.76 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081180.306852 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143754.88 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 14375.49 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095555.795323 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129379.40 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 12937.94 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108493.734946 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116441.46 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 11644.15 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120137.880608 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104797.31 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 10479.73 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130617.611703 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94317.58 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 9431.76 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140049.369689 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84885.82 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 8488.58 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148537.951876 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76397.24 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 7639.72 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156177.675845 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68757.52 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 6875.75 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163053.427416 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61881.76 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 6188.18 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169241.603831 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55693.59 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 5569.36 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174810.962604 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50124.23 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 5012.42 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179823.385499 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45111.81 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 4511.18 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184334.566105 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40600.63 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 4060.06 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188394.628651 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36540.56 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3654.06 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192048.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32886.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195648.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29286.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199248.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25686.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202848.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22086.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206448.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18486.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210048.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14886.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213648.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11286.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217248.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7686.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220848.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4086.51 sec) 15:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224448.684942 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 486.51 sec) 15:14:00 [GPUGRID] [rr_sim] 2224935.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978479522.16G/439.78G) 15:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852816.10 15:14:00 [---] pick_jobs_to_run() start 15:14:00 [---] pick_jobs_to_run() end 15:14:00 [---] [rr_sim] end 15:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:14:00 [---] [cpu_sched_debug] preliminary job list: 15:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:14:00 [---] [cpu_sched_debug] final job list: 15:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:14:00 [---] [cpu_sched_debug] enforce_run_list: end 15:14:00 [---] choose_project(): 54840.000000 15:14:00 [WUProp@Home] doing RPC to get work 15:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380759.09 cpu 2127872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127872.90 15:14:00 [---] Got 0 tasks 15:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:15:00 [---] [rr_sim] doing sim: CPU sched 15:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978451931.80G/439.78G) 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224872.45 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 222487.25 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222487.245468 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002385.21 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 200238.52 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422725.766389 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802146.69 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 180214.67 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602940.435218 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621932.02 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 162193.20 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765133.637164 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459738.82 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 145973.88 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911107.518915 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313764.94 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 131376.49 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042484.012492 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182388.44 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 118238.84 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160722.856710 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064149.60 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 106414.96 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267137.816507 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957734.64 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 95773.46 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362911.280324 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861961.17 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 86196.12 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449107.397760 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775765.06 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 77576.51 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526683.903452 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698188.55 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 69818.86 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596502.758574 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628369.70 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 62836.97 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659339.728185 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565532.73 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 56553.27 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715893.000834 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508979.45 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 50897.95 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766790.946219 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458081.51 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 45808.15 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812599.097065 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412273.36 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 41227.34 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853826.432826 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371046.02 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 37104.60 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890931.035011 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333941.42 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 33394.14 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924325.176978 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300547.28 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 30054.73 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954379.904748 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270492.55 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 27049.25 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981429.159741 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243443.29 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 24344.33 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005773.489235 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219098.97 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 21909.90 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027683.385779 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197189.07 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 19718.91 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047402.292669 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177470.16 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 17747.02 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065149.308870 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159723.15 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 15972.31 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081121.623451 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143750.83 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 14375.08 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095496.706574 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129375.75 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 12937.57 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108434.281384 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116438.17 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 11643.82 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120078.098714 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104794.36 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 10479.44 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130557.534310 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94314.92 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 9431.49 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139989.026347 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84883.43 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 8488.34 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148477.369180 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76395.09 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 7639.51 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156116.877730 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68755.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 6875.56 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162992.435425 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61880.02 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 6188.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169180.437350 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55692.02 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 5569.20 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174749.639083 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50122.82 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 5012.28 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179761.920643 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45110.53 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 4511.05 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184272.974046 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40599.48 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 4059.95 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188332.922109 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36539.53 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3653.95 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191986.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32885.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195586.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29285.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199186.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25685.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202786.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22085.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206386.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18485.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209986.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14885.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213586.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11285.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217186.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7685.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220786.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4085.58 sec) 15:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224386.875366 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 485.58 sec) 15:15:00 [GPUGRID] [rr_sim] 2224872.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978451931.80G/439.78G) 15:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852813.37 15:15:00 [---] pick_jobs_to_run() start 15:15:00 [---] pick_jobs_to_run() end 15:15:00 [---] [rr_sim] end 15:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:15:00 [---] [cpu_sched_debug] preliminary job list: 15:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:15:00 [---] [cpu_sched_debug] final job list: 15:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:15:00 [---] [cpu_sched_debug] enforce_run_list: end 15:15:00 [---] choose_project(): 54900.000000 15:15:00 [WUProp@Home] doing RPC to get work 15:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380699.09 cpu 2127812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127812.90 15:15:00 [---] Got 0 tasks 15:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:16:00 [---] [rr_sim] doing sim: CPU sched 15:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978424341.44G/439.78G) 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224809.72 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 222480.97 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222480.971780 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002328.75 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 200232.87 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422713.846381 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802095.87 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 180209.59 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602923.433523 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621886.28 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 162188.63 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765112.061950 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459697.66 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 145969.77 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911081.827535 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313727.89 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 131372.79 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042454.616561 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182355.10 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 118235.51 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160690.126684 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064119.59 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 106411.96 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267102.085796 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957707.63 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 95770.76 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362872.848996 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861936.87 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 86193.69 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449066.535876 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775743.18 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 77574.32 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526640.854068 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698168.86 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 69816.89 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596457.740441 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628351.98 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 62835.20 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659292.938176 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565516.78 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 56551.68 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715844.616138 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508965.10 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 50896.51 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766741.126304 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458068.59 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 45806.86 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812547.985453 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412261.73 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 41226.17 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853774.158688 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371035.56 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 37103.56 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890877.714599 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333932.00 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 33393.20 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924270.914918 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300538.80 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 30053.88 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954324.795206 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270484.92 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 27048.49 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981373.287465 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243436.43 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 24343.64 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005716.930498 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219092.79 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 21909.28 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027626.209228 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197183.51 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 19718.35 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047344.560085 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177465.16 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 17746.52 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065091.075856 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159718.64 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 15971.86 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081062.940050 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143746.78 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 14374.68 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095437.617825 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129372.10 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 12937.21 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108374.827822 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116434.89 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 11643.49 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120018.316819 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104791.40 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 10479.14 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130497.456917 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94312.26 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 9431.23 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139928.683005 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84881.03 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 8488.10 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148416.786484 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76392.93 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 7639.29 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156056.079615 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68753.64 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 6875.36 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162931.443433 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61878.27 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 6187.83 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169119.270870 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55690.45 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 5569.04 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174688.315562 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50121.40 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 5012.14 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179700.455786 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45109.26 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 4510.93 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184211.381987 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40598.34 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 4059.83 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188271.215568 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36538.50 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3653.85 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191925.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32884.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195525.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29284.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199125.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25684.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202725.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22084.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206325.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18484.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209925.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14884.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213525.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11284.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217125.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7684.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220725.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4084.65 sec) 15:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224325.065791 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 484.65 sec) 15:16:00 [GPUGRID] [rr_sim] 2224809.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978424341.44G/439.78G) 15:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852810.63 15:16:00 [---] pick_jobs_to_run() start 15:16:00 [---] pick_jobs_to_run() end 15:16:00 [---] [rr_sim] end 15:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:16:00 [---] [cpu_sched_debug] preliminary job list: 15:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:16:00 [---] [cpu_sched_debug] final job list: 15:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:16:00 [---] [cpu_sched_debug] enforce_run_list: end 15:16:00 [---] choose_project(): 54960.000000 15:16:00 [WUProp@Home] doing RPC to get work 15:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380639.09 cpu 2127752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127752.90 15:16:00 [---] Got 0 tasks 15:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:17:00 [---] [rr_sim] doing sim: CPU sched 15:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978396751.09G/439.78G) 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224746.98 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 222474.70 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222474.698091 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002272.28 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 200227.23 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422701.926374 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1802045.05 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 180204.51 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602906.431828 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621840.55 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 162184.05 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765090.486736 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459656.49 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 145965.65 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911056.136154 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313690.84 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 131369.08 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042425.220630 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182321.76 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 118232.18 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160657.396659 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064089.58 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 106408.96 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267066.355084 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957680.63 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 95768.06 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362834.417667 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861912.56 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 86191.26 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1449025.673992 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775721.31 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 77572.13 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526597.804684 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698149.18 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 69814.92 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596412.722307 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628334.26 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 62833.43 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659246.148168 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565500.83 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 56550.08 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715796.231442 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508950.75 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 50895.07 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766691.306390 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458055.67 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 45805.57 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812496.873842 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412250.11 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 41225.01 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853721.884549 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371025.10 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 37102.51 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890824.394186 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333922.59 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 33392.26 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924216.652859 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300530.33 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 30053.03 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954269.685664 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270477.30 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 27047.73 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981317.415189 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243429.57 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 24342.96 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005660.371762 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219086.61 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 21908.66 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027569.032677 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197177.95 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 19717.79 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047286.827501 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177460.15 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 17746.02 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065032.842842 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159714.14 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 15971.41 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081004.256649 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143742.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 14374.27 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095378.529076 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129368.45 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 12936.85 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108315.374260 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116431.61 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 11643.16 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119958.534925 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104788.45 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 10478.84 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130437.379524 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94309.60 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 9430.96 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139868.339663 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84878.64 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 8487.86 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148356.203788 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76390.78 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 7639.08 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155995.281501 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68751.70 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 6875.17 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162870.451442 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61876.53 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 6187.65 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169058.104389 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55688.88 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 5568.89 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174626.992042 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50119.99 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 5012.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179638.990929 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45107.99 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 4510.80 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184149.789928 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40597.19 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 4059.72 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188209.509026 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36537.47 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3653.75 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191863.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32883.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195463.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29283.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199063.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25683.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202663.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22083.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206263.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18483.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209863.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14883.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213463.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11283.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217063.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7683.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220663.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4083.72 sec) 15:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224263.256215 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 483.72 sec) 15:17:00 [GPUGRID] [rr_sim] 2224746.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978396751.09G/439.78G) 15:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852807.89 15:17:00 [---] pick_jobs_to_run() start 15:17:00 [---] pick_jobs_to_run() end 15:17:00 [---] [rr_sim] end 15:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:17:00 [---] [cpu_sched_debug] preliminary job list: 15:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:17:00 [---] [cpu_sched_debug] final job list: 15:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:17:00 [---] [cpu_sched_debug] enforce_run_list: end 15:17:00 [---] choose_project(): 55020.000000 15:17:00 [WUProp@Home] doing RPC to get work 15:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380579.09 cpu 2127692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127692.90 15:17:00 [---] Got 0 tasks 15:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:18:00 [---] [rr_sim] doing sim: CPU sched 15:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978369160.73G/439.78G) 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224684.24 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 222468.42 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222468.424403 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002215.82 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 200221.58 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422690.006366 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801994.24 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 180199.42 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602889.430133 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621794.81 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 162179.48 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765068.911523 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459615.33 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 145961.53 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911030.444774 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313653.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 131365.38 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042395.824699 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182288.42 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 118228.84 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160624.666633 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064059.58 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 106405.96 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1267030.624373 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957653.62 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 95765.36 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362795.986339 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861888.26 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 86188.83 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448984.812108 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775699.43 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 77569.94 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526554.755300 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698129.49 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 69812.95 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596367.704173 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628316.54 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 62831.65 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659199.358159 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565484.89 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 56548.49 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715747.846747 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508936.40 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 50893.64 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766641.486475 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458042.76 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 45804.28 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812445.762231 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412238.48 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 41223.85 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853669.610411 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371014.63 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 37101.46 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890771.073773 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333913.17 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 33391.32 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924162.390799 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300521.85 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 30052.19 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954214.576122 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270469.67 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 27046.97 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981261.542913 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243422.70 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 24342.27 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005603.813025 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219080.43 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 21908.04 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027511.856126 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197172.39 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 19717.24 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047229.094916 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177455.15 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 17745.51 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064974.609828 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159709.63 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 15970.96 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080945.573248 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143738.67 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 14373.87 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095319.440327 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129364.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 12936.48 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108255.920697 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116428.32 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 11642.83 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119898.753031 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104785.49 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 10478.55 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130377.302131 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94306.94 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 9430.69 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139807.996321 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84876.25 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 8487.62 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148295.621092 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76388.62 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 7638.86 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155934.483386 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68749.76 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 6874.98 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162809.459451 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61874.78 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 6187.48 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168996.937909 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55687.31 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 5568.73 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174565.668521 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50118.58 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 5011.86 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179577.526072 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45106.72 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 4510.67 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184088.197868 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40596.05 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 4059.60 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188147.802485 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36536.44 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3653.64 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191801.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32882.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195401.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29282.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199001.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25682.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202601.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22082.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206201.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18482.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209801.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14882.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213401.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11282.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217001.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7682.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220601.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4082.80 sec) 15:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224201.446639 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 482.80 sec) 15:18:00 [GPUGRID] [rr_sim] 2224684.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978369160.73G/439.78G) 15:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852805.16 15:18:00 [---] pick_jobs_to_run() start 15:18:00 [---] pick_jobs_to_run() end 15:18:00 [---] [rr_sim] end 15:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:18:00 [---] [cpu_sched_debug] preliminary job list: 15:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:18:00 [---] [cpu_sched_debug] final job list: 15:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:18:00 [---] [cpu_sched_debug] enforce_run_list: end 15:18:00 [---] choose_project(): 55080.000000 15:18:00 [WUProp@Home] doing RPC to get work 15:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380519.09 cpu 2127632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127632.90 15:18:00 [---] Got 0 tasks 15:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:19:00 [---] [rr_sim] doing sim: CPU sched 15:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978341570.37G/439.78G) 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224621.51 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 222462.15 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222462.150715 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002159.36 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 200215.94 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422678.086358 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801943.42 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 180194.34 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602872.428438 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621749.08 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 162174.91 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765047.336309 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459574.17 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 145957.42 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 911004.753393 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313616.75 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 131361.68 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042366.428769 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182255.08 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 118225.51 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160591.936607 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1064029.57 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 106402.96 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266994.893661 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957626.61 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 95762.66 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362757.555010 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861863.95 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 86186.40 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448943.950224 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775677.56 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 77567.76 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526511.705917 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698109.80 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 69810.98 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596322.686040 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628298.82 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 62829.88 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659152.568151 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565468.94 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 56546.89 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715699.462051 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508922.05 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 50892.20 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766591.666561 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458029.84 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 45802.98 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812394.650620 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412226.86 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 41222.69 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853617.336273 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371004.17 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 37100.42 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890717.753360 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333903.75 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 33390.38 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924108.128739 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300513.38 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 30051.34 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954159.466580 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270462.04 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 27046.20 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981205.670637 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243415.84 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 24341.58 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005547.254289 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219074.25 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 21907.43 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027454.679575 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197166.83 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 19716.68 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047171.362332 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177450.14 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 17745.01 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064916.376814 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159705.13 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 15970.51 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080886.889848 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143734.62 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 14373.46 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095260.351578 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129361.16 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 12936.12 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108196.467135 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116425.04 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 11642.50 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119838.971136 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104782.54 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 10478.25 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130317.224738 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94304.28 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 9430.43 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139747.652979 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84873.85 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 8487.39 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148235.038396 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76386.47 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 7638.65 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155873.685272 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68747.82 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 6874.78 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162748.467459 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61873.04 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 6187.30 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168935.771428 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55685.74 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 5568.57 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174504.345001 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50117.16 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 5011.72 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179516.061215 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45105.45 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 4510.54 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184026.605809 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40594.90 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 4059.49 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188086.095943 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36535.41 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3653.54 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191739.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32881.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195339.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29281.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198939.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25681.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202539.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22081.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206139.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18481.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209739.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14881.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213339.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11281.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216939.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7681.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220539.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4081.87 sec) 15:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224139.637064 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 481.87 sec) 15:19:00 [GPUGRID] [rr_sim] 2224621.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978341570.37G/439.78G) 15:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852802.42 15:19:00 [---] pick_jobs_to_run() start 15:19:00 [---] pick_jobs_to_run() end 15:19:00 [---] [rr_sim] end 15:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:19:00 [---] [cpu_sched_debug] preliminary job list: 15:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:19:00 [---] [cpu_sched_debug] final job list: 15:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:19:00 [---] [cpu_sched_debug] enforce_run_list: end 15:19:00 [---] choose_project(): 55140.000000 15:19:00 [WUProp@Home] doing RPC to get work 15:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380459.09 cpu 2127572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127572.90 15:19:00 [---] Got 0 tasks 15:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:20:00 [---] [rr_sim] doing sim: CPU sched 15:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978313980.02G/439.78G) 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224558.77 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 222455.88 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222455.877027 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002102.89 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 200210.29 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422666.166351 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801892.60 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 180189.26 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602855.426743 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621703.34 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 162170.33 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765025.761095 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459533.01 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 145953.30 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910979.062012 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313579.71 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 131357.97 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042337.032838 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182221.74 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 118222.17 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160559.206581 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063999.56 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 106399.96 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266959.162950 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957599.61 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 95759.96 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362719.123681 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861839.65 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 86183.96 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448903.088340 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775655.68 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 77565.57 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526468.656533 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698090.11 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 69809.01 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596277.667906 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628281.10 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 62828.11 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659105.778142 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565452.99 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 56545.30 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715651.077355 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508907.69 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 50890.77 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766541.846646 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458016.92 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 45801.69 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812343.539008 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412215.23 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 41221.52 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853565.062134 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370993.71 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 37099.37 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890664.432948 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333894.34 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 33389.43 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924053.866680 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300504.90 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 30050.49 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954104.357038 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270454.41 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 27045.44 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981149.798361 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243408.97 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 24340.90 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005490.695552 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219068.07 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 21906.81 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027397.503024 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197161.27 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 19716.13 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047113.629748 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177445.14 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 17744.51 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064858.143800 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159700.63 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 15970.06 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080828.206447 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143730.56 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 14373.06 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095201.262829 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129357.51 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 12935.75 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108137.013573 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116421.76 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 11642.18 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119779.189242 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104779.58 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 10477.96 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130257.147345 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94301.62 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 9430.16 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139687.309637 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84871.46 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 8487.15 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148174.455700 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76384.31 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 7638.43 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155812.887157 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68745.88 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 6874.59 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162687.475468 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61871.29 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 6187.13 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168874.604948 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55684.17 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 5568.42 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174443.021480 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50115.75 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 5011.57 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179454.596359 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45104.17 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 4510.42 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183965.013750 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40593.76 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 4059.38 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188024.389401 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36534.38 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3653.44 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191677.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32880.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195277.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29280.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198877.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25680.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202477.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22080.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206077.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18480.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209677.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14880.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213277.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11280.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216877.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7680.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220477.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4080.94 sec) 15:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224077.827488 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 480.94 sec) 15:20:00 [GPUGRID] [rr_sim] 2224558.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978313980.02G/439.78G) 15:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852799.68 15:20:00 [---] pick_jobs_to_run() start 15:20:00 [---] pick_jobs_to_run() end 15:20:00 [---] [rr_sim] end 15:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:20:00 [---] [cpu_sched_debug] preliminary job list: 15:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:20:00 [---] [cpu_sched_debug] final job list: 15:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:20:00 [---] [cpu_sched_debug] enforce_run_list: end 15:20:00 [---] choose_project(): 55200.000000 15:20:00 [WUProp@Home] doing RPC to get work 15:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380399.09 cpu 2127512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127512.90 15:20:00 [---] Got 0 tasks 15:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:21:00 [---] [rr_sim] doing sim: CPU sched 15:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978286389.66G/439.78G) 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224496.03 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 222449.60 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222449.603339 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2002046.43 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 200204.64 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422654.246343 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801841.79 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 180184.18 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602838.425047 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621657.61 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 162165.76 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 765004.185881 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459491.85 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 145949.18 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910953.370632 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313542.66 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 131354.27 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042307.636907 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182188.40 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 118218.84 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160526.476555 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063969.56 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 106396.96 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266923.432238 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957572.60 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 95757.26 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362680.692353 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861815.34 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 86181.53 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448862.226456 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775633.81 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 77563.38 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526425.607149 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698070.43 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 69807.04 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596232.649773 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628263.38 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 62826.34 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659058.988134 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565437.05 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 56543.70 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715602.692659 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508893.34 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 50889.33 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766492.026732 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458004.01 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 45800.40 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812292.427397 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412203.61 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 41220.36 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853512.787996 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370983.25 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 37098.32 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890611.112535 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333884.92 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 33388.49 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923999.604620 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300496.43 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 30049.64 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954049.247497 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270446.79 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 27044.68 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981093.926085 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243402.11 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 24340.21 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005434.136815 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219061.90 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 21906.19 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027340.326472 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197155.71 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 19715.57 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047055.897164 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177440.14 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 17744.01 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064799.910786 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159696.12 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 15969.61 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080769.523046 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143726.51 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 14372.65 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095142.174080 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129353.86 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 12935.39 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108077.560010 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116418.47 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 11641.85 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119719.407348 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104776.63 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 10477.66 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130197.069952 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94298.96 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 9429.90 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139626.966295 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84869.07 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 8486.91 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148113.873004 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76382.16 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 7638.22 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155752.089042 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68743.94 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 6874.39 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162626.483477 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61869.55 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 6186.95 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168813.438468 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55682.59 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 5568.26 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174381.697959 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50114.34 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 5011.43 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179393.131502 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45102.90 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 4510.29 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183903.421690 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40592.61 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 4059.26 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187962.682860 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36533.35 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3653.34 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191616.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32880.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195216.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29280.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198816.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25680.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202416.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22080.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206016.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18480.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209616.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14880.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213216.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11280.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216816.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7680.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220416.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4080.02 sec) 15:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2224016.017912 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 480.02 sec) 15:21:00 [GPUGRID] [rr_sim] 2224496.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978286389.66G/439.78G) 15:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852796.95 15:21:00 [---] pick_jobs_to_run() start 15:21:00 [---] pick_jobs_to_run() end 15:21:00 [---] [rr_sim] end 15:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:21:00 [---] [cpu_sched_debug] preliminary job list: 15:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:21:00 [---] [cpu_sched_debug] final job list: 15:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:21:00 [---] [cpu_sched_debug] enforce_run_list: end 15:21:00 [---] choose_project(): 55260.000000 15:21:00 [WUProp@Home] doing RPC to get work 15:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380339.09 cpu 2127452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127452.90 15:21:00 [---] Got 0 tasks 15:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:22:00 [---] [rr_sim] doing sim: CPU sched 15:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978258799.31G/439.78G) 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224433.30 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 222443.33 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222443.329650 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001989.97 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 200199.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422642.326336 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801790.97 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 180179.10 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602821.423352 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621611.87 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 162161.19 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764982.610668 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459450.69 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 145945.07 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910927.679251 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313505.62 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 131350.56 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042278.240976 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182155.06 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 118215.51 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160493.746529 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063939.55 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 106393.95 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266887.701526 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957545.59 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 95754.56 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362642.261024 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861791.04 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 86179.10 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448821.364572 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775611.93 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 77561.19 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526382.557765 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698050.74 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 69805.07 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596187.631639 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628245.66 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 62824.57 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659012.198125 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565421.10 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 56542.11 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715554.307963 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508878.99 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 50887.90 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766442.206817 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457991.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 45799.11 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812241.315786 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412191.98 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 41219.20 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853460.513858 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370972.78 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 37097.28 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890557.792122 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333875.50 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 33387.55 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923945.342560 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300487.95 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 30048.80 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953994.137955 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270439.16 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 27043.92 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981038.053810 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243395.24 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 24339.52 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005377.578079 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219055.72 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 21905.57 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027283.149921 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197150.15 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 19715.01 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046998.164580 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177435.13 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 17743.51 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064741.677772 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159691.62 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 15969.16 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080710.839645 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143722.46 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 14372.25 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095083.085331 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129350.21 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 12935.02 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108018.106448 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116415.19 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 11641.52 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119659.625454 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104773.67 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 10477.37 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130136.992559 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94296.30 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 9429.63 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139566.622953 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84866.67 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 8486.67 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148053.290308 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76380.01 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 7638.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155691.290928 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68742.01 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 6874.20 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162565.491485 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61867.81 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 6186.78 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168752.271987 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55681.02 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 5568.10 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174320.374439 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50112.92 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 5011.29 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179331.666645 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45101.63 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 4510.16 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183841.829631 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40591.47 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 4059.15 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187900.976318 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36532.32 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3653.23 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191554.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32879.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195154.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29279.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198754.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25679.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202354.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22079.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205954.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18479.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209554.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14879.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213154.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11279.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216754.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7679.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220354.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4079.09 sec) 15:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223954.208337 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 479.09 sec) 15:22:00 [GPUGRID] [rr_sim] 2224433.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978258799.31G/439.78G) 15:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852794.21 15:22:00 [---] pick_jobs_to_run() start 15:22:00 [---] pick_jobs_to_run() end 15:22:00 [---] [rr_sim] end 15:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:22:00 [---] [cpu_sched_debug] preliminary job list: 15:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:22:00 [---] [cpu_sched_debug] final job list: 15:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:22:00 [---] [cpu_sched_debug] enforce_run_list: end 15:22:00 [---] choose_project(): 55320.000000 15:22:00 [WUProp@Home] doing RPC to get work 15:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380279.09 cpu 2127392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127392.90 15:22:00 [---] Got 0 tasks 15:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:23:00 [---] [rr_sim] doing sim: CPU sched 15:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978258799.31G/439.78G) 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224433.30 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 222443.33 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222443.329650 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001989.97 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 200199.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422642.326336 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801790.97 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 180179.10 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602821.423352 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621611.87 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 162161.19 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764982.610668 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459450.69 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 145945.07 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910927.679251 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313505.62 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 131350.56 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042278.240976 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182155.06 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 118215.51 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160493.746529 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063939.55 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 106393.95 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266887.701526 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957545.59 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 95754.56 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362642.261024 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861791.04 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 86179.10 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448821.364572 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775611.93 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 77561.19 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526382.557765 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698050.74 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 69805.07 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596187.631639 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628245.66 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 62824.57 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1659012.198125 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565421.10 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 56542.11 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715554.307963 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508878.99 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 50887.90 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766442.206817 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457991.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 45799.11 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812241.315786 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412191.98 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 41219.20 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853460.513858 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370972.78 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 37097.28 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890557.792122 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333875.50 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 33387.55 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923945.342560 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300487.95 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 30048.80 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953994.137955 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270439.16 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 27043.92 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981038.053810 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243395.24 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 24339.52 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005377.578079 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219055.72 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 21905.57 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027283.149921 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197150.15 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 19715.01 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046998.164580 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177435.13 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 17743.51 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064741.677772 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159691.62 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 15969.16 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080710.839645 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143722.46 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 14372.25 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095083.085331 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129350.21 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 12935.02 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108018.106448 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116415.19 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 11641.52 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119659.625454 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104773.67 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 10477.37 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130136.992559 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94296.30 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 9429.63 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139566.622953 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84866.67 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 8486.67 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148053.290308 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76380.01 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 7638.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155691.290928 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68742.01 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 6874.20 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162565.491485 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61867.81 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 6186.78 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168752.271987 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55681.02 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 5568.10 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174320.374439 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50112.92 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 5011.29 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179331.666645 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45101.63 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 4510.16 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183841.829631 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40591.47 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 4059.15 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187900.976318 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36532.32 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3653.23 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191554.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32879.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195154.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29279.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198754.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25679.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202354.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22079.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205954.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18479.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209554.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14879.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213154.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11279.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216754.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7679.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220354.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4079.09 sec) 15:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223954.208337 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 479.09 sec) 15:23:00 [GPUGRID] [rr_sim] 2224433.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978258799.31G/439.78G) 15:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852854.21 15:23:00 [---] pick_jobs_to_run() start 15:23:00 [---] pick_jobs_to_run() end 15:23:00 [---] [rr_sim] end 15:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:23:00 [---] [cpu_sched_debug] preliminary job list: 15:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:23:00 [---] [cpu_sched_debug] final job list: 15:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:23:00 [---] [cpu_sched_debug] enforce_run_list: end 15:23:00 [---] choose_project(): 55380.000000 15:23:00 [WUProp@Home] doing RPC to get work 15:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380219.09 cpu 2127392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127392.90 15:23:00 [---] Got 0 tasks 15:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:24:00 [---] [rr_sim] doing sim: CPU sched 15:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978231208.95G/439.78G) 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224370.56 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 222437.06 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222437.055962 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001933.50 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 200193.35 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422630.406328 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801740.15 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 180174.02 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602804.421657 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621566.14 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 162156.61 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764961.035454 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459409.52 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 145940.95 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910901.987870 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313468.57 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 131346.86 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042248.845046 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182121.71 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 118212.17 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160461.016503 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063909.54 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 106390.95 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266851.970815 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957518.59 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 95751.86 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362603.829696 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861766.73 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 86176.67 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448780.502688 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775590.06 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 77559.01 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526339.508381 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698031.05 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 69803.11 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596142.613505 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628227.95 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 62822.79 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658965.408117 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565405.15 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 56540.52 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715505.923267 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508864.64 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 50886.46 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766392.386903 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457978.17 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 45797.82 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812190.204175 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412180.36 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 41218.04 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853408.239719 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370962.32 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 37096.23 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890504.471709 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333866.09 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 33386.61 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923891.080501 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300479.48 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 30047.95 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953939.028413 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270431.53 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 27043.15 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980982.181534 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243388.38 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 24338.84 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005321.019342 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219049.54 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 21904.95 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027225.973370 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197144.59 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 19714.46 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046940.431995 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177430.13 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 17743.01 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064683.444758 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159687.11 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 15968.71 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080652.156244 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143718.40 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 14371.84 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095023.996582 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129346.56 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 12934.66 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107958.652886 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116411.91 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 11641.19 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119599.843559 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104770.72 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 10477.07 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130076.915166 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94293.64 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 9429.36 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139506.279611 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84864.28 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 8486.43 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147992.707612 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76377.85 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 7637.79 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155630.492813 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68740.07 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 6874.01 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162504.499494 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61866.06 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 6186.61 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168691.105507 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55679.45 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 5567.95 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174259.050918 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50111.51 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 5011.15 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179270.201788 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45100.36 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 4510.04 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183780.237572 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40590.32 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 4059.03 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187839.269777 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36531.29 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3653.13 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191492.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32878.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195092.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29278.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198692.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25678.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202292.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22078.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205892.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18478.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209492.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14878.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213092.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11278.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216692.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7678.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220292.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4078.16 sec) 15:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223892.398761 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 478.16 sec) 15:24:00 [GPUGRID] [rr_sim] 2224370.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978231208.95G/439.78G) 15:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852851.47 15:24:00 [---] pick_jobs_to_run() start 15:24:00 [---] pick_jobs_to_run() end 15:24:00 [---] [rr_sim] end 15:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:24:00 [---] [cpu_sched_debug] preliminary job list: 15:24:00 [---] [cpu_sched_debug] final job list: 15:24:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 15:24:00 [---] [work_fetch] Request work fetch: CPUs idle 15:24:00 [---] [cpu_sched_debug] enforce_run_list: end 15:24:00 [---] choose_project(): 55440.000000 15:24:00 [WUProp@Home] doing RPC to get work 15:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380159.09 cpu 2127332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127332.90 15:24:00 [---] Got 0 tasks 15:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:25:00 [---] [rr_sim] doing sim: CPU sched 15:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978231208.95G/439.78G) 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224370.56 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 222437.06 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222437.055962 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001933.50 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 200193.35 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422630.406328 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801740.15 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 180174.02 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602804.421657 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621566.14 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 162156.61 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764961.035454 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459409.52 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 145940.95 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910901.987870 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313468.57 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 131346.86 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042248.845046 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182121.71 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 118212.17 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160461.016503 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063909.54 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 106390.95 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266851.970815 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957518.59 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 95751.86 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362603.829696 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861766.73 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 86176.67 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448780.502688 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775590.06 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 77559.01 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526339.508381 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698031.05 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 69803.11 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596142.613505 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628227.95 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 62822.79 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658965.408117 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565405.15 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 56540.52 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715505.923267 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508864.64 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 50886.46 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766392.386903 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457978.17 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 45797.82 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812190.204175 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412180.36 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 41218.04 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853408.239719 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370962.32 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 37096.23 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890504.471709 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333866.09 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 33386.61 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923891.080501 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300479.48 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 30047.95 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953939.028413 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270431.53 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 27043.15 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980982.181534 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243388.38 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 24338.84 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005321.019342 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219049.54 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 21904.95 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027225.973370 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197144.59 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 19714.46 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046940.431995 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177430.13 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 17743.01 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064683.444758 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159687.11 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 15968.71 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080652.156244 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143718.40 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 14371.84 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095023.996582 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129346.56 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 12934.66 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107958.652886 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116411.91 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 11641.19 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119599.843559 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104770.72 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 10477.07 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130076.915166 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94293.64 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 9429.36 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139506.279611 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84864.28 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 8486.43 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147992.707612 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76377.85 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 7637.79 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155630.492813 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68740.07 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 6874.01 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162504.499494 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61866.06 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 6186.61 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168691.105507 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55679.45 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 5567.95 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174259.050918 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50111.51 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 5011.15 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179270.201788 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45100.36 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 4510.04 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183780.237572 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40590.32 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 4059.03 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187839.269777 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36531.29 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3653.13 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191492.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32878.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195092.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29278.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198692.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25678.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202292.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22078.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205892.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18478.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209492.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14878.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213092.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11278.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216692.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7678.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220292.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4078.16 sec) 15:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223892.398761 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 478.16 sec) 15:25:00 [GPUGRID] [rr_sim] 2224370.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978231208.95G/439.78G) 15:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852911.47 15:25:00 [---] pick_jobs_to_run() start 15:25:00 [---] pick_jobs_to_run() end 15:25:00 [---] [rr_sim] end 15:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:25:00 [---] [cpu_sched_debug] preliminary job list: 15:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:25:00 [---] [cpu_sched_debug] final job list: 15:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 15:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:25:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 978231208.95G 15:25:00 [---] [cpu_sched_debug] enforce_run_list: end 15:25:00 [---] choose_project(): 55500.000000 15:25:00 [WUProp@Home] doing RPC to get work 15:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380099.09 cpu 2127332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127332.90 15:25:00 [---] Got 0 tasks 15:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:26:00 [---] [rr_sim] doing sim: CPU sched 15:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978203618.59G/439.78G) 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224307.82 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 222430.78 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222430.782274 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001877.04 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 200187.70 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422618.486320 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801689.34 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 180168.93 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602787.419962 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621520.40 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 162152.04 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764939.460240 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459368.36 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 145936.84 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910876.296490 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313431.53 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 131343.15 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042219.449115 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182088.37 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 118208.84 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160428.286477 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063879.54 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 106387.95 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266816.240103 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957491.58 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 95749.16 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362565.398367 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861742.42 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 86174.24 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448739.640804 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775568.18 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 77556.82 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526296.458998 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 698011.36 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 69801.14 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596097.595372 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628210.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 62821.02 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658918.618109 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565389.20 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 56538.92 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715457.538572 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508850.28 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 50885.03 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766342.566988 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457965.26 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 45796.53 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812139.092563 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412168.73 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 41216.87 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853355.965581 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370951.86 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 37095.19 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890451.151297 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333856.67 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 33385.67 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923836.818441 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300471.00 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 30047.10 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953883.918871 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270423.90 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 27042.39 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980926.309258 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243381.51 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 24338.15 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005264.460606 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219043.36 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 21904.34 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027168.796819 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197139.03 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 19713.90 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046882.699411 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177425.12 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 17742.51 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064625.211744 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159682.61 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 15968.26 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080593.472843 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143714.35 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 14371.43 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094964.907833 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129342.91 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 12934.29 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107899.199324 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116408.62 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 11640.86 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119540.061665 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104767.76 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 10476.78 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130016.837773 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94290.98 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 9429.10 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139445.936269 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84861.89 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 8486.19 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147932.124916 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76375.70 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 7637.57 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155569.694698 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68738.13 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 6873.81 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162443.507503 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61864.32 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 6186.43 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168629.939026 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55677.88 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 5567.79 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174197.727397 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50110.10 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 5011.01 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179208.736932 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45099.09 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 4509.91 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183718.645512 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40589.18 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 4058.92 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187777.563235 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36530.26 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3653.03 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191430.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32877.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195030.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29277.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198630.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25677.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202230.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22077.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205830.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18477.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209430.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14877.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213030.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11277.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216630.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7677.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220230.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4077.23 sec) 15:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223830.589185 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 477.23 sec) 15:26:00 [GPUGRID] [rr_sim] 2224307.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978203618.59G/439.78G) 15:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852908.74 15:26:00 [---] pick_jobs_to_run() start 15:26:00 [---] pick_jobs_to_run() end 15:26:00 [---] [rr_sim] end 15:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:26:00 [---] [cpu_sched_debug] preliminary job list: 15:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 15:26:00 [---] [cpu_sched_debug] final job list: 15:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:26:00 [---] [cpu_sched_debug] enforce_run_list: end 15:26:00 [---] choose_project(): 55560.000000 15:26:00 [WUProp@Home] doing RPC to get work 15:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 380039.09 cpu 2127272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127272.90 15:26:00 [---] Got 0 tasks 15:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:27:00 [---] [rr_sim] doing sim: CPU sched 15:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978176028.24G/439.78G) 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224245.09 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 222424.51 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222424.508586 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001820.58 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 200182.06 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422606.566313 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801638.52 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 180163.85 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602770.418267 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621474.67 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 162147.47 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764917.885026 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459327.20 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 145932.72 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910850.605109 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313394.48 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 131339.45 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042190.053184 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182055.03 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 118205.50 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160395.556451 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063849.53 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 106384.95 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266780.509392 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957464.58 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 95746.46 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362526.967038 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861718.12 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 86171.81 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448698.778920 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775546.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 77554.63 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526253.409614 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697991.68 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 69799.17 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596052.577238 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628192.51 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 62819.25 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658871.828100 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565373.26 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 56537.33 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715409.153876 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508835.93 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 50883.59 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766292.747074 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457952.34 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 45795.23 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812087.980952 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412157.10 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 41215.71 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853303.691443 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370941.39 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 37094.14 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890397.830884 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333847.25 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 33384.73 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923782.556381 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300462.53 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 30046.25 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953828.809329 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270416.28 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 27041.63 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980870.436982 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243374.65 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 24337.46 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005207.901869 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219037.18 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 21903.72 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027111.620268 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197133.47 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 19713.35 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046824.966827 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177420.12 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 17742.01 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064566.978730 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159678.11 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 15967.81 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080534.789443 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143710.30 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 14371.03 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094905.819084 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129339.27 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 12933.93 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107839.745761 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116405.34 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 11640.53 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119480.279771 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104764.81 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 10476.48 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129956.760379 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94288.33 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 9428.83 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139385.592927 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84859.49 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 8485.95 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147871.542220 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76373.54 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 7637.35 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155508.896584 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68736.19 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 6873.62 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162382.515511 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61862.57 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 6186.26 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168568.772546 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55676.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 5567.63 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174136.403877 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50108.68 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 5010.87 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179147.272075 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45097.81 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 4509.78 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183657.053453 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40588.03 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 4058.80 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187715.856693 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36529.23 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3652.92 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191368.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32876.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194968.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29276.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198568.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25676.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202168.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22076.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205768.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18476.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209368.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14876.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212968.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11276.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216568.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7676.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220168.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4076.31 sec) 15:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223768.779610 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 476.31 sec) 15:27:00 [GPUGRID] [rr_sim] 2224245.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978176028.24G/439.78G) 15:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852906.00 15:27:00 [---] pick_jobs_to_run() start 15:27:00 [---] pick_jobs_to_run() end 15:27:00 [---] [rr_sim] end 15:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:27:00 [---] [cpu_sched_debug] preliminary job list: 15:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:27:00 [---] [cpu_sched_debug] final job list: 15:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:27:00 [---] [cpu_sched_debug] enforce_run_list: end 15:27:00 [---] choose_project(): 55620.000000 15:27:00 [WUProp@Home] doing RPC to get work 15:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379979.09 cpu 2127212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127212.90 15:27:00 [---] Got 0 tasks 15:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:28:00 [---] [rr_sim] doing sim: CPU sched 15:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978148437.88G/439.78G) 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224182.35 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 222418.23 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222418.234897 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001764.11 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 200176.41 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422594.646305 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801587.70 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 180158.77 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602753.416572 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621428.93 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 162142.89 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764896.309812 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459286.04 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 145928.60 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910824.913729 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313357.44 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 131335.74 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042160.657253 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1182021.69 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 118202.17 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160362.826425 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063819.52 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 106381.95 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266744.778680 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957437.57 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 95743.76 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362488.535710 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861693.81 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 86169.38 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448657.917036 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775524.43 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 77552.44 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526210.360230 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697971.99 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 69797.20 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1596007.559105 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628174.79 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 62817.48 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658825.038092 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565357.31 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 56535.73 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715360.769180 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508821.58 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 50882.16 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766242.927159 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457939.42 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 45793.94 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1812036.869341 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412145.48 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 41214.55 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853251.417304 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370930.93 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 37093.09 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890344.510471 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333837.84 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 33383.78 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923728.294322 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300454.05 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 30045.41 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953773.699787 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270408.65 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 27040.86 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980814.564706 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243367.78 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 24336.78 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005151.343133 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219031.01 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 21903.10 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027054.443717 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197127.91 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 19712.79 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046767.234243 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177415.11 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 17741.51 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064508.745716 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159673.60 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 15967.36 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080476.106042 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143706.24 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 14370.62 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094846.730335 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129335.62 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 12933.56 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107780.292199 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116402.06 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 11640.21 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119420.497877 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104761.85 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 10476.19 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129896.682986 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94285.67 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 9428.57 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139325.249585 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84857.10 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 8485.71 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147810.959524 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76371.39 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 7637.14 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155448.098469 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68734.25 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 6873.43 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162321.523520 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61860.83 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 6186.08 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168507.606065 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55674.74 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 5567.47 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174075.080356 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50107.27 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 5010.73 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179085.807218 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45096.54 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 4509.65 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183595.461394 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40586.89 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 4058.69 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187654.150152 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36528.20 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3652.82 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191306.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32875.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194906.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29275.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198506.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25675.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202106.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22075.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205706.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18475.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209306.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14875.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212906.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11275.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216506.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7675.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220106.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4075.38 sec) 15:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223706.970034 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 475.38 sec) 15:28:00 [GPUGRID] [rr_sim] 2224182.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978148437.88G/439.78G) 15:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852903.26 15:28:00 [---] pick_jobs_to_run() start 15:28:00 [---] pick_jobs_to_run() end 15:28:00 [---] [rr_sim] end 15:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:28:00 [---] [cpu_sched_debug] preliminary job list: 15:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:28:00 [---] [cpu_sched_debug] final job list: 15:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:28:00 [---] [cpu_sched_debug] enforce_run_list: end 15:28:00 [---] choose_project(): 55680.000000 15:28:00 [WUProp@Home] doing RPC to get work 15:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379919.09 cpu 2127152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127152.90 15:28:00 [---] Got 0 tasks 15:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:29:00 [---] [rr_sim] doing sim: CPU sched 15:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978120847.52G/439.78G) 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224119.61 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 222411.96 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222411.961209 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001707.65 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 200170.77 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422582.726298 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801536.89 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 180153.69 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602736.414877 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621383.20 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 162138.32 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764874.734599 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459244.88 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 145924.49 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910799.222348 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313320.39 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 131332.04 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042131.261322 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181988.35 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 118198.84 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160330.096399 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063789.52 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 106378.95 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266709.047969 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957410.56 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 95741.06 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362450.104381 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861669.51 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 86166.95 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448617.055152 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775502.56 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 77550.26 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526167.310846 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697952.30 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 69795.23 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595962.540971 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628157.07 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 62815.71 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658778.248083 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565341.36 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 56534.14 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715312.384484 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508807.23 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 50880.72 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766193.107245 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457926.50 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 45792.65 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811985.757730 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412133.85 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 41213.39 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853199.143166 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370920.47 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 37092.05 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890291.190059 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333828.42 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 33382.84 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923674.032262 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300445.58 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 30044.56 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953718.590245 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270401.02 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 27040.10 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980758.692430 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243360.92 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 24336.09 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005094.784396 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219024.83 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 21902.48 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026997.267166 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197122.34 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 19712.23 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046709.501658 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177410.11 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 17741.01 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064450.512702 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159669.10 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 15966.91 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080417.422641 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143702.19 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 14370.22 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094787.641586 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129331.97 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 12933.20 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107720.838637 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116398.77 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 11639.88 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119360.715982 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104758.90 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 10475.89 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129836.605593 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94283.01 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 9428.30 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139264.906243 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84854.71 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 8485.47 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147750.376828 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76369.24 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 7636.92 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155387.300355 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68732.31 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 6873.23 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162260.531528 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61859.08 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 6185.91 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168446.439585 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55673.17 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 5567.32 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174013.756836 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50105.86 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 5010.59 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179024.342361 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45095.27 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 4509.53 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183533.869334 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40585.74 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 4058.57 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187592.443610 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36527.17 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3652.72 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191245.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32874.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194845.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29274.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198445.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25674.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202045.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22074.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205645.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18474.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209245.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14874.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212845.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11274.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216445.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7674.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220045.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4074.45 sec) 15:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223645.160458 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 474.45 sec) 15:29:00 [GPUGRID] [rr_sim] 2224119.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978120847.52G/439.78G) 15:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852900.52 15:29:00 [---] pick_jobs_to_run() start 15:29:00 [---] pick_jobs_to_run() end 15:29:00 [---] [rr_sim] end 15:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:29:00 [---] [cpu_sched_debug] preliminary job list: 15:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:29:00 [---] [cpu_sched_debug] final job list: 15:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:29:00 [---] [cpu_sched_debug] enforce_run_list: end 15:29:00 [---] choose_project(): 55740.000000 15:29:00 [WUProp@Home] doing RPC to get work 15:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379859.09 cpu 2127092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127092.90 15:29:00 [---] Got 0 tasks 15:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:30:00 [---] [rr_sim] doing sim: CPU sched 15:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978093257.17G/439.78G) 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2224056.88 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 222405.69 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222405.687521 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001651.19 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 200165.12 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422570.806290 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801486.07 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 180148.61 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602719.413182 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621337.46 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 162133.75 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764853.159385 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459203.72 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 145920.37 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910773.530967 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313283.34 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 131328.33 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042101.865392 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181955.01 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 118195.50 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160297.366374 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063759.51 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 106375.95 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266673.317257 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957383.56 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 95738.36 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362411.673053 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861645.20 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 86164.52 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448576.193268 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775480.68 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 77548.07 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526124.261463 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697932.61 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 69793.26 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595917.522837 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628139.35 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 62813.94 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658731.458075 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565325.42 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 56532.54 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715263.999788 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508792.88 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 50879.29 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766143.287330 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457913.59 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 45791.36 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811934.646118 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412122.23 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 41212.22 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853146.869028 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370910.01 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 37091.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890237.869646 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333819.01 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 33381.90 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923619.770202 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300437.11 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 30043.71 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953663.480703 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270393.39 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 27039.34 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980702.820154 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243354.06 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 24335.41 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005038.225660 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219018.65 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 21901.86 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026940.090615 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197116.78 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 19711.68 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046651.769074 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177405.11 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 17740.51 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064392.279688 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159664.60 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 15966.46 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080358.739240 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143698.14 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 14369.81 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094728.552837 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129328.32 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 12932.83 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107661.385074 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116395.49 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 11639.55 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119300.934088 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104755.94 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 10475.59 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129776.528200 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94280.35 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 9428.03 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139204.562901 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84852.31 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 8485.23 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147689.794132 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76367.08 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 7636.71 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155326.502240 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68730.37 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 6873.04 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162199.539537 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61857.34 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 6185.73 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168385.273104 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55671.60 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 5567.16 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173952.433315 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50104.44 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 5010.44 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178962.877505 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45094.00 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 4509.40 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183472.277275 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40584.60 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 4058.46 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187530.737069 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36526.14 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3652.61 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191183.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32873.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194783.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29273.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198383.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25673.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201983.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22073.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205583.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18473.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209183.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14873.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212783.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11273.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216383.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7673.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219983.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4073.52 sec) 15:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223583.350883 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 473.52 sec) 15:30:00 [GPUGRID] [rr_sim] 2224056.88: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978093257.17G/439.78G) 15:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852897.79 15:30:00 [---] pick_jobs_to_run() start 15:30:00 [---] pick_jobs_to_run() end 15:30:00 [---] [rr_sim] end 15:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:30:00 [---] [cpu_sched_debug] preliminary job list: 15:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:30:00 [---] [cpu_sched_debug] final job list: 15:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:30:00 [---] [cpu_sched_debug] enforce_run_list: end 15:30:00 [---] choose_project(): 55800.000000 15:30:00 [WUProp@Home] doing RPC to get work 15:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379799.09 cpu 2127032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2127032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2127032.90 15:30:00 [---] Got 0 tasks 15:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:31:00 [---] [rr_sim] doing sim: CPU sched 15:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978065666.81G/439.78G) 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223994.14 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 222399.41 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222399.413833 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001594.72 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 200159.47 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422558.886282 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801435.25 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 180143.53 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602702.411487 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621291.73 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 162129.17 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764831.584171 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459162.55 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 145916.26 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910747.839587 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313246.30 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 131324.63 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042072.469461 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181921.67 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 118192.17 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160264.636348 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063729.50 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 106372.95 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266637.586546 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957356.55 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 95735.66 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362373.241724 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861620.90 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 86162.09 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448535.331384 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775458.81 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 77545.88 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526081.212079 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697912.93 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 69791.29 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595872.504704 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628121.63 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 62812.16 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658684.668066 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565309.47 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 56530.95 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715215.615092 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508778.52 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 50877.85 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766093.467416 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457900.67 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 45790.07 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811883.534507 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412110.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 41211.06 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853094.594889 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370899.54 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 37089.95 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890184.549233 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333809.59 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 33380.96 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923565.508143 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300428.63 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 30042.86 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953608.371161 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270385.77 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 27038.58 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980646.947878 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243347.19 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 24334.72 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004981.666923 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219012.47 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 21901.25 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026882.914064 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197111.22 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 19711.12 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046594.036490 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177400.10 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 17740.01 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064334.046674 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159660.09 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 15966.01 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080300.055839 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143694.08 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 14369.41 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094669.464088 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129324.67 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 12932.47 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107601.931512 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116392.21 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 11639.22 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119241.152194 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104752.99 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 10475.30 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129716.450807 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94277.69 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 9427.77 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139144.219559 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84849.92 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 8484.99 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147629.211436 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76364.93 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 7636.49 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155265.704125 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68728.43 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 6872.84 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162138.547546 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61855.59 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 6185.56 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168324.106624 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55670.03 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 5567.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173891.109794 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50103.03 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 5010.30 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178901.412648 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45092.73 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 4509.27 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183410.685216 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40583.45 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 4058.35 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187469.030527 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36525.11 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3652.51 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191121.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32872.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194721.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29272.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198321.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25672.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201921.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22072.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205521.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18472.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209121.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14872.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212721.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11272.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216321.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7672.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219921.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4072.60 sec) 15:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223521.541307 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 472.60 sec) 15:31:00 [GPUGRID] [rr_sim] 2223994.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978065666.81G/439.78G) 15:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852895.05 15:31:00 [---] pick_jobs_to_run() start 15:31:00 [---] pick_jobs_to_run() end 15:31:00 [---] [rr_sim] end 15:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:31:00 [---] [cpu_sched_debug] preliminary job list: 15:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:31:00 [---] [cpu_sched_debug] final job list: 15:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:31:00 [---] [cpu_sched_debug] enforce_run_list: end 15:31:00 [---] choose_project(): 55860.000000 15:31:00 [WUProp@Home] doing RPC to get work 15:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379739.09 cpu 2126972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126972.90 15:31:00 [---] Got 0 tasks 15:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:32:00 [---] [rr_sim] doing sim: CPU sched 15:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978038076.46G/439.78G) 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223931.40 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 222393.14 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222393.140145 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001538.26 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 200153.83 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422546.966275 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801384.44 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 180138.44 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602685.409792 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621245.99 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 162124.60 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764810.008957 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459121.39 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 145912.14 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910722.148206 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313209.25 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 131320.93 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042043.073530 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181888.33 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 118188.83 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160231.906322 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063699.50 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 106369.95 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266601.855834 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957329.55 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 95732.95 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362334.810395 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861596.59 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 86159.66 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448494.469500 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775436.93 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 77543.69 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1526038.162695 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697893.24 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 69789.32 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595827.486570 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628103.91 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 62810.39 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658637.878058 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565293.52 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 56529.35 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715167.230397 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508764.17 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 50876.42 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1766043.647501 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457887.75 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 45788.78 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811832.422896 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412098.98 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 41209.90 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1853042.320751 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370889.08 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 37088.91 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890131.228820 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333800.17 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 33380.02 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923511.246083 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300420.16 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 30042.02 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953553.261619 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270378.14 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 27037.81 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980591.075602 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243340.33 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 24334.03 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004925.108186 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219006.29 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 21900.63 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026825.737512 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197105.66 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 19710.57 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046536.303906 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177395.10 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 17739.51 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064275.813660 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159655.59 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 15965.56 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080241.372438 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143690.03 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 14369.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094610.375339 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129321.03 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 12932.10 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107542.477950 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116388.92 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 11638.89 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119181.370299 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104750.03 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 10475.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129656.373414 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94275.03 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 9427.50 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139083.876217 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84847.53 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 8484.75 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147568.628740 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76362.77 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 7636.28 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155204.906011 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68726.50 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 6872.65 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162077.555554 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61853.85 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 6185.38 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168262.940143 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55668.46 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 5566.85 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173829.786274 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50101.62 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 5010.16 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178839.947791 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45091.45 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 4509.15 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183349.093156 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40582.31 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 4058.23 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187407.323985 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36524.08 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3652.41 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191059.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32871.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194659.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29271.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198259.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25671.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201859.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22071.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205459.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18471.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209059.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14871.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212659.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11271.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216259.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7671.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219859.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4071.67 sec) 15:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223459.731732 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 471.67 sec) 15:32:00 [GPUGRID] [rr_sim] 2223931.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978038076.46G/439.78G) 15:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852892.31 15:32:00 [---] pick_jobs_to_run() start 15:32:00 [---] pick_jobs_to_run() end 15:32:00 [---] [rr_sim] end 15:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:32:00 [---] [cpu_sched_debug] preliminary job list: 15:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:32:00 [---] [cpu_sched_debug] final job list: 15:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:32:00 [---] [cpu_sched_debug] enforce_run_list: end 15:32:00 [---] choose_project(): 55920.000000 15:32:00 [WUProp@Home] doing RPC to get work 15:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379679.09 cpu 2126912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126912.90 15:32:00 [---] Got 0 tasks 15:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:33:00 [---] [rr_sim] doing sim: CPU sched 15:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (978010486.10G/439.78G) 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223868.66 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 222386.87 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222386.866456 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001481.80 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 200148.18 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422535.046267 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801333.62 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 180133.36 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602668.408097 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621200.26 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 162120.03 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764788.433744 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459080.23 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 145908.02 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910696.456826 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313172.21 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 131317.22 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1042013.677599 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181854.99 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 118185.50 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160199.176296 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063669.49 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 106366.95 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266566.125123 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957302.54 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 95730.25 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362296.379067 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861572.29 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 86157.23 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448453.607616 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775415.06 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 77541.51 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525995.113311 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697873.55 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 69787.36 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595782.468436 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628086.20 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 62808.62 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658591.088049 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565277.58 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 56527.76 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715118.845701 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508749.82 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 50874.98 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765993.827587 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457874.84 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 45787.48 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811781.311285 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412087.35 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 41208.74 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852990.046613 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370878.62 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 37087.86 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890077.908408 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333790.76 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 33379.08 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923456.984023 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300411.68 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 30041.17 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953498.152077 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270370.51 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 27037.05 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980535.203326 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243333.46 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 24333.35 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004868.549450 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219000.12 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 21900.01 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026768.560961 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197100.10 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 19710.01 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046478.571322 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177390.09 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 17739.01 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064217.580646 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159651.08 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 15965.11 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080182.689038 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143685.98 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 14368.60 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094551.286590 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129317.38 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 12931.74 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107483.024388 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116385.64 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 11638.56 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119121.588405 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104747.08 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 10474.71 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129596.296021 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94272.37 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 9427.24 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139023.532875 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84845.13 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 8484.51 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147508.046044 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76360.62 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 7636.06 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155144.107896 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68724.56 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 6872.46 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162016.563563 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61852.10 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 6185.21 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168201.773663 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55666.89 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 5566.69 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173768.462753 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50100.20 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 5010.02 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178778.482934 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45090.18 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 4509.02 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183287.501097 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40581.16 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 4058.12 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187345.617444 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36523.05 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3652.30 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190997.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32870.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194597.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29270.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198197.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25670.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201797.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22070.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205397.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18470.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208997.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14870.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212597.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11270.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216197.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7670.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219797.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4070.74 sec) 15:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223397.922156 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 470.74 sec) 15:33:00 [GPUGRID] [rr_sim] 2223868.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (978010486.10G/439.78G) 15:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852889.58 15:33:00 [---] pick_jobs_to_run() start 15:33:00 [---] pick_jobs_to_run() end 15:33:00 [---] [rr_sim] end 15:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:33:00 [---] [cpu_sched_debug] preliminary job list: 15:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:33:00 [---] [cpu_sched_debug] final job list: 15:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:33:00 [---] [cpu_sched_debug] enforce_run_list: end 15:33:00 [---] choose_project(): 55980.000000 15:33:00 [WUProp@Home] doing RPC to get work 15:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379619.09 cpu 2126852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126852.90 15:33:00 [---] Got 0 tasks 15:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:34:00 [---] [rr_sim] doing sim: CPU sched 15:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977982895.74G/439.78G) 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223805.93 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 222380.59 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222380.592768 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001425.33 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 200142.53 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422523.126260 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801282.80 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 180128.28 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602651.406402 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621154.52 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 162115.45 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764766.858530 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1459039.07 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 145903.91 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910670.765445 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313135.16 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 131313.52 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041984.281669 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181821.65 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 118182.16 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160166.446270 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063639.48 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 106363.95 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266530.394411 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957275.53 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 95727.55 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362257.947738 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861547.98 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 86154.80 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448412.745733 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775393.18 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 77539.32 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525952.063927 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697853.86 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 69785.39 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595737.450303 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628068.48 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 62806.85 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658544.298041 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565261.63 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 56526.16 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715070.461005 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508735.47 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 50873.55 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765944.007673 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457861.92 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 45786.19 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811730.199673 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412075.73 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 41207.57 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852937.772474 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370868.16 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 37086.82 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1890024.587995 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333781.34 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 33378.13 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923402.721964 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300403.21 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 30040.32 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953443.042536 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270362.89 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 27036.29 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980479.331050 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243326.60 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 24332.66 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004811.990713 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218993.94 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 21899.39 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026711.384410 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197094.54 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 19709.45 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046420.838737 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177385.09 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 17738.51 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064159.347632 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159646.58 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 15964.66 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080124.005637 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143681.92 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 14368.19 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094492.197841 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129313.73 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 12931.37 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107423.570825 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116382.36 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 11638.24 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119061.806511 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104744.12 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 10474.41 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129536.218628 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94269.71 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 9426.97 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138963.189533 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84842.74 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 8484.27 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147447.463348 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76358.46 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 7635.85 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155083.309782 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68722.62 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 6872.26 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161955.571572 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61850.36 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 6185.04 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168140.607183 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55665.32 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 5566.53 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173707.139233 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50098.79 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 5009.88 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178717.018077 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45088.91 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 4508.89 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183225.909038 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40580.02 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 4058.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187283.910902 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36522.02 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3652.20 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190936.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32869.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194536.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29269.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198136.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25669.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201736.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22069.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205336.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18469.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208936.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14869.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212536.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11269.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216136.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7669.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219736.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4069.82 sec) 15:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223336.112580 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 469.82 sec) 15:34:00 [GPUGRID] [rr_sim] 2223805.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977982895.74G/439.78G) 15:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852886.84 15:34:00 [---] pick_jobs_to_run() start 15:34:00 [---] pick_jobs_to_run() end 15:34:00 [---] [rr_sim] end 15:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:34:00 [---] [cpu_sched_debug] preliminary job list: 15:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:34:00 [---] [cpu_sched_debug] final job list: 15:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:34:00 [---] [cpu_sched_debug] enforce_run_list: end 15:34:00 [---] choose_project(): 56040.000000 15:34:00 [WUProp@Home] doing RPC to get work 15:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379559.09 cpu 2126792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126792.90 15:34:00 [---] Got 0 tasks 15:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:35:00 [---] [rr_sim] doing sim: CPU sched 15:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977955305.39G/439.78G) 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223743.19 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 222374.32 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222374.319080 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001368.87 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 200136.89 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422511.206252 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801231.98 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 180123.20 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602634.404707 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621108.79 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 162110.88 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764745.283316 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458997.91 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 145899.79 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910645.074064 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313098.12 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 131309.81 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041954.885738 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181788.31 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 118178.83 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160133.716244 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063609.47 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 106360.95 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266494.663700 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957248.53 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 95724.85 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362219.516410 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861523.67 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 86152.37 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448371.883849 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775371.31 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 77537.13 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525909.014544 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697834.18 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 69783.42 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595692.432169 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628050.76 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 62805.08 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658497.508032 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565245.68 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 56524.57 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1715022.076309 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508721.11 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 50872.11 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765894.187758 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457849.00 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 45784.90 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811679.088062 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412064.10 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 41206.41 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852885.498336 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370857.69 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 37085.77 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889971.267582 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333771.92 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 33377.19 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923348.459904 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300394.73 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 30039.47 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953387.932994 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270355.26 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 27035.53 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980423.458774 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243319.73 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 24331.97 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004755.431977 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218987.76 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 21898.78 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026654.207859 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197088.98 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 19708.90 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046363.106153 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177380.08 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 17738.01 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064101.114618 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159642.08 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 15964.21 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080065.322236 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143677.87 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 14367.79 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094433.109092 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129310.08 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 12931.01 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107364.117263 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116379.07 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 11637.91 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119002.024617 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104741.17 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 10474.12 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129476.141235 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94267.05 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 9426.70 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138902.846191 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84840.34 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 8484.03 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147386.880652 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76356.31 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 7635.63 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155022.511667 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68720.68 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 6872.07 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161894.579580 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61848.61 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 6184.86 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168079.440702 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55663.75 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 5566.38 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173645.815712 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50097.38 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 5009.74 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178655.553221 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45087.64 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 4508.76 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183164.316979 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40578.87 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 4057.89 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187222.204361 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36520.99 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3652.10 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190874.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32868.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194474.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29268.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198074.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25668.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201674.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22068.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205274.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18468.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208874.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14868.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212474.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11268.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216074.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7668.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219674.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4068.89 sec) 15:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223274.303005 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 468.89 sec) 15:35:00 [GPUGRID] [rr_sim] 2223743.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977955305.39G/439.78G) 15:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852884.10 15:35:00 [---] pick_jobs_to_run() start 15:35:00 [---] pick_jobs_to_run() end 15:35:00 [---] [rr_sim] end 15:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:35:00 [---] [cpu_sched_debug] preliminary job list: 15:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:35:00 [---] [cpu_sched_debug] final job list: 15:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:35:00 [---] [cpu_sched_debug] enforce_run_list: end 15:35:00 [---] choose_project(): 56100.000000 15:35:00 [WUProp@Home] doing RPC to get work 15:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379499.09 cpu 2126732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126732.90 15:35:00 [---] Got 0 tasks 15:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:36:00 [---] [rr_sim] doing sim: CPU sched 15:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977927715.03G/439.78G) 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223680.45 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 222368.05 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222368.045392 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001312.41 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 200131.24 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422499.286244 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801181.17 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 180118.12 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602617.403012 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621063.05 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 162106.31 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764723.708102 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458956.75 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 145895.67 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910619.382684 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313061.07 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 131306.11 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041925.489807 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181754.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 118175.50 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160100.986218 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063579.47 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 106357.95 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266458.932988 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957221.52 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 95722.15 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362181.085081 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861499.37 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 86149.94 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448331.021965 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775349.43 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 77534.94 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525865.965160 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697814.49 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 69781.45 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595647.414036 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628033.04 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 62803.30 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658450.718024 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565229.74 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 56522.97 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714973.691613 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508706.76 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 50870.68 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765844.367844 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457836.09 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 45783.61 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811627.976451 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412052.48 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 41205.25 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852833.224198 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370847.23 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 37084.72 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889917.947170 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333762.51 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 33376.25 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923294.197844 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300386.26 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 30038.63 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953332.823452 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270347.63 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 27034.76 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980367.586498 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243312.87 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 24331.29 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004698.873240 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218981.58 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 21898.16 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026597.031308 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197083.42 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 19708.34 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046305.373569 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177375.08 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 17737.51 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064042.881604 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159637.57 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 15963.76 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080006.638835 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143673.82 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 14367.38 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094374.020343 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129306.43 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 12930.64 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107304.663701 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116375.79 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 11637.58 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118942.242722 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104738.21 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 10473.82 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129416.063842 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94264.39 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 9426.44 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138842.502849 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84837.95 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 8483.80 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147326.297956 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76354.16 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 7635.42 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154961.713552 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68718.74 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 6871.87 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161833.587589 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61846.87 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 6184.69 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168018.274222 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55662.18 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 5566.22 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173584.492191 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50095.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 5009.60 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178594.088364 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45086.37 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 4508.64 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183102.724919 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40577.73 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 4057.77 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187160.497819 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36519.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3652.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190812.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32867.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194412.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29267.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198012.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25667.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201612.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22067.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205212.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18467.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208812.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14867.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212412.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11267.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216012.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7667.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219612.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4067.96 sec) 15:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223212.493429 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467.96 sec) 15:36:00 [GPUGRID] [rr_sim] 2223680.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977927715.03G/439.78G) 15:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852881.37 15:36:00 [---] pick_jobs_to_run() start 15:36:00 [---] pick_jobs_to_run() end 15:36:00 [---] [rr_sim] end 15:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:36:00 [---] [cpu_sched_debug] preliminary job list: 15:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:36:00 [---] [cpu_sched_debug] final job list: 15:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:36:00 [---] [cpu_sched_debug] enforce_run_list: end 15:36:00 [---] choose_project(): 56160.000000 15:36:00 [WUProp@Home] doing RPC to get work 15:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379439.09 cpu 2126672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126672.90 15:36:00 [---] Got 0 tasks 15:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:37:00 [---] [rr_sim] doing sim: CPU sched 15:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977900124.67G/439.78G) 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223617.72 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 222361.77 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222361.771704 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001255.95 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 200125.59 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422487.366237 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801130.35 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 180113.04 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602600.401317 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1621017.32 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 162101.73 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764702.132888 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458915.58 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 145891.56 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910593.691303 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1313024.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 131302.40 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041896.093876 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181721.62 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 118172.16 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160068.256192 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063549.46 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 106354.95 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266423.202277 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957194.51 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 95719.45 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362142.653752 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861475.06 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 86147.51 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448290.160081 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775327.56 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 77532.76 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525822.915776 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697794.80 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 69779.48 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595602.395902 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 628015.32 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 62801.53 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658403.928015 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565213.79 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 56521.38 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714925.306917 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508692.41 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 50869.24 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765794.547929 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457823.17 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 45782.32 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811576.864840 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412040.85 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 41204.09 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852780.950059 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370836.77 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 37083.68 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889864.626757 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333753.09 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 33375.31 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923239.935785 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300377.78 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 30037.78 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953277.713910 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270340.00 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 27034.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980311.714222 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243306.00 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 24330.60 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004642.314504 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218975.40 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 21897.54 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026539.854757 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197077.86 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 19707.79 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046247.640985 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177370.08 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 17737.01 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063984.648590 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159633.07 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 15963.31 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079947.955434 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143669.76 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 14366.98 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094314.931594 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129302.79 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 12930.28 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107245.210138 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116372.51 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 11637.25 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118882.460828 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104735.26 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 10473.53 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129355.986449 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94261.73 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 9426.17 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138782.159507 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84835.56 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 8483.56 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147265.715260 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76352.00 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 7635.20 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154900.915438 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68716.80 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 6871.68 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161772.595597 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61845.12 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 6184.51 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167957.107741 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55660.61 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 5566.06 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173523.168671 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50094.55 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 5009.45 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178532.623507 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45085.09 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 4508.51 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183041.132860 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40576.58 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 4057.66 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187098.791277 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36518.93 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3651.89 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190750.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32867.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194350.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29267.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197950.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25667.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201550.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22067.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205150.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18467.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208750.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14867.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212350.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11267.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215950.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7667.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219550.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4067.03 sec) 15:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223150.683853 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 467.03 sec) 15:37:00 [GPUGRID] [rr_sim] 2223617.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977900124.67G/439.78G) 15:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852878.63 15:37:00 [---] pick_jobs_to_run() start 15:37:00 [---] pick_jobs_to_run() end 15:37:00 [---] [rr_sim] end 15:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:37:00 [---] [cpu_sched_debug] preliminary job list: 15:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:37:00 [---] [cpu_sched_debug] final job list: 15:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:37:00 [---] [cpu_sched_debug] enforce_run_list: end 15:37:00 [---] choose_project(): 56220.000000 15:37:00 [WUProp@Home] doing RPC to get work 15:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379379.09 cpu 2126612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126612.90 15:37:00 [---] Got 0 tasks 15:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:38:00 [---] [rr_sim] doing sim: CPU sched 15:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977872534.32G/439.78G) 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223554.98 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 222355.50 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222355.498015 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001199.48 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 200119.95 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422475.446229 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801079.53 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 180107.95 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602583.399621 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620971.58 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 162097.16 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764680.557675 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458874.42 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 145887.44 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910567.999922 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312986.98 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 131298.70 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041866.697946 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181688.28 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 118168.83 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160035.526166 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063519.45 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 106351.95 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266387.471565 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957167.51 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 95716.75 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362104.222424 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861450.76 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 86145.08 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448249.298197 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775305.68 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 77530.57 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525779.866392 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697775.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 69777.51 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595557.377768 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627997.60 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 62799.76 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658357.138007 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565197.84 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 56519.78 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714876.922222 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508678.06 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 50867.81 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765744.728015 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457810.25 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 45781.03 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811525.753229 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412029.23 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 41202.92 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852728.675921 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370826.30 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 37082.63 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889811.306344 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333743.67 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 33374.37 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923185.673725 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300369.31 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 30036.93 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953222.604368 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270332.38 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 27033.24 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980255.841946 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243299.14 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 24329.91 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004585.755767 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218969.22 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 21896.92 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026482.678206 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197072.30 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 19707.23 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046189.908400 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177365.07 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 17736.51 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063926.415576 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159628.56 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 15962.86 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079889.272033 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143665.71 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 14366.57 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094255.842845 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129299.14 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 12929.91 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107185.756576 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116369.22 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 11636.92 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118822.678934 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104732.30 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 10473.23 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129295.909056 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94259.07 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 9425.91 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138721.816165 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84833.16 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 8483.32 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147205.132564 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76349.85 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 7634.98 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154840.117323 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68714.86 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 6871.49 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161711.603606 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61843.38 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 6184.34 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167895.941261 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55659.04 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 5565.90 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173461.845150 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50093.14 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 5009.31 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178471.158650 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45083.82 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 4508.38 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182979.540801 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40575.44 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 4057.54 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187037.084736 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36517.90 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3651.79 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190688.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32866.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194288.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29266.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197888.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25666.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201488.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22066.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205088.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18466.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208688.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14866.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212288.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11266.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215888.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7666.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219488.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4066.11 sec) 15:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223088.874278 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 466.11 sec) 15:38:00 [GPUGRID] [rr_sim] 2223554.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977872534.32G/439.78G) 15:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852875.89 15:38:00 [---] pick_jobs_to_run() start 15:38:00 [---] pick_jobs_to_run() end 15:38:00 [---] [rr_sim] end 15:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:38:00 [---] [cpu_sched_debug] preliminary job list: 15:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:38:00 [---] [cpu_sched_debug] final job list: 15:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:38:00 [---] [cpu_sched_debug] enforce_run_list: end 15:38:00 [---] choose_project(): 56280.000000 15:38:00 [WUProp@Home] doing RPC to get work 15:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379319.09 cpu 2126552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126552.90 15:38:00 [---] Got 0 tasks 15:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:39:00 [---] [rr_sim] doing sim: CPU sched 15:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977844943.96G/439.78G) 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223492.24 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 222349.22 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222349.224327 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001143.02 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 200114.30 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422463.526221 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1801028.72 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 180102.87 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602566.397926 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620925.85 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 162092.58 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764658.982461 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458833.26 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 145883.33 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910542.308542 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312949.93 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 131294.99 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041837.302015 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181654.94 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 118165.49 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1160002.796140 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063489.45 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 106348.94 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266351.740853 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957140.50 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 95714.05 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362065.791095 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861426.45 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 86142.65 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448208.436313 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775283.81 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 77528.38 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525736.817009 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697755.43 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 69775.54 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595512.359635 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627979.88 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 62797.99 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658310.347998 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565181.90 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 56518.19 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714828.537526 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508663.71 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 50866.37 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765694.908100 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457797.34 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 45779.73 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811474.641617 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412017.60 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 41201.76 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852676.401783 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370815.84 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 37081.58 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889757.985931 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333734.26 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 33373.43 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923131.411665 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300360.83 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 30036.08 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953167.494826 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270324.75 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 27032.47 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980199.969670 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243292.27 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 24329.23 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004529.197030 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218963.05 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 21896.30 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026425.501655 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197066.74 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 19706.67 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046132.175816 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177360.07 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 17736.01 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063868.182562 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159624.06 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 15962.41 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079830.588633 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143661.65 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 14366.17 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094196.754096 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129295.49 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 12929.55 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107126.303014 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116365.94 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 11636.59 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118762.897040 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104729.35 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 10472.93 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129235.831663 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94256.41 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 9425.64 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138661.472824 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84830.77 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 8483.08 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147144.549868 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76347.69 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 7634.77 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154779.319209 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68712.92 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 6871.29 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161650.611615 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61841.63 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 6184.16 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167834.774780 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55657.47 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 5565.75 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173400.521629 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50091.72 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 5009.17 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178409.693794 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45082.55 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 4508.25 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182917.948741 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40574.29 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 4057.43 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186975.378194 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36516.87 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3651.69 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190627.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32865.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194227.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29265.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197827.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25665.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201427.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22065.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205027.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18465.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208627.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14865.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212227.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11265.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215827.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7665.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219427.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4065.18 sec) 15:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2223027.064702 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 465.18 sec) 15:39:00 [GPUGRID] [rr_sim] 2223492.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977844943.96G/439.78G) 15:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852873.16 15:39:00 [---] pick_jobs_to_run() start 15:39:00 [---] pick_jobs_to_run() end 15:39:00 [---] [rr_sim] end 15:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:39:00 [---] [cpu_sched_debug] preliminary job list: 15:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:39:00 [---] [cpu_sched_debug] final job list: 15:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:39:00 [---] [cpu_sched_debug] enforce_run_list: end 15:39:00 [---] choose_project(): 56340.000000 15:39:00 [WUProp@Home] doing RPC to get work 15:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379259.09 cpu 2126492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126492.90 15:39:00 [---] Got 0 tasks 15:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:40:00 [---] [rr_sim] doing sim: CPU sched 15:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977817353.61G/439.78G) 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223429.51 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 222342.95 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222342.950639 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001086.56 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 200108.66 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422451.606214 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800977.90 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 180097.79 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602549.396231 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620880.11 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 162088.01 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764637.407247 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458792.10 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 145879.21 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910516.617161 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312912.89 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 131291.29 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041807.906084 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181621.60 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 118162.16 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159970.066114 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063459.44 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 106345.94 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266316.010142 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957113.50 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 95711.35 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1362027.359767 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861402.15 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 86140.21 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448167.574429 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775261.93 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 77526.19 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525693.767625 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697735.74 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 69773.57 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595467.341501 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627962.16 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 62796.22 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658263.557990 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565165.95 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 56516.59 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714780.152830 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508649.35 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 50864.94 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765645.088186 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457784.42 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 45778.44 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811423.530006 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412005.98 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 41200.60 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852624.127644 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370805.38 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 37080.54 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889704.665519 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333724.84 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 33372.48 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923077.149606 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300352.36 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 30035.24 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953112.385284 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270317.12 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 27031.71 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980144.097395 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243285.41 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 24328.54 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004472.638294 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218956.87 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 21895.69 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026368.325103 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197061.18 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 19706.12 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046074.443232 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177355.06 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 17735.51 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063809.949548 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159619.56 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 15961.96 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079771.905232 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143657.60 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 14365.76 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094137.665347 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129291.84 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 12929.18 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107066.849452 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116362.66 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 11636.27 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118703.115145 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104726.39 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 10472.64 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129175.754270 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94253.75 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 9425.38 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138601.129482 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84828.38 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 8482.84 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147083.967172 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76345.54 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 7634.55 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154718.521094 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68710.99 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 6871.10 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161589.619623 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61839.89 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 6183.99 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167773.608300 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55655.90 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 5565.59 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173339.198109 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50090.31 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 5009.03 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178348.228937 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45081.28 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 4508.13 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182856.356682 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40573.15 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 4057.31 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186913.671653 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36515.83 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3651.58 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190565.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32864.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194165.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29264.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197765.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25664.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201365.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22064.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204965.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18464.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208565.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14864.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212165.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11264.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215765.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7664.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219365.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4064.25 sec) 15:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222965.255126 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 464.25 sec) 15:40:00 [GPUGRID] [rr_sim] 2223429.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977817353.61G/439.78G) 15:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852870.42 15:40:00 [---] pick_jobs_to_run() start 15:40:00 [---] pick_jobs_to_run() end 15:40:00 [---] [rr_sim] end 15:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:40:00 [---] [cpu_sched_debug] preliminary job list: 15:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:40:00 [---] [cpu_sched_debug] final job list: 15:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:40:00 [---] [cpu_sched_debug] enforce_run_list: end 15:40:00 [---] choose_project(): 56400.000000 15:40:00 [WUProp@Home] doing RPC to get work 15:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379199.09 cpu 2126432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126432.90 15:40:00 [---] Got 0 tasks 15:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:41:00 [---] [rr_sim] doing sim: CPU sched 15:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977789763.25G/439.78G) 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223366.77 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 222336.68 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222336.676951 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2001030.09 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 200103.01 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422439.686206 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800927.08 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 180092.71 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602532.394536 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620834.37 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 162083.44 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764615.832033 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458750.94 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 145875.09 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910490.925781 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312875.84 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 131287.58 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041778.510153 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181588.26 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 118158.83 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159937.336089 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063429.43 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 106342.94 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266280.279430 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957086.49 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 95708.65 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361988.928438 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861377.84 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 86137.78 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448126.712545 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775240.06 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 77524.01 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525650.718241 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697716.05 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 69771.61 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595422.323368 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627944.45 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 62794.44 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658216.767981 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565150.00 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 56515.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714731.768134 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508635.00 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 50863.50 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765595.268271 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457771.50 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 45777.15 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811372.418395 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411994.35 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 41199.44 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852571.853506 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370794.92 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 37079.49 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889651.345106 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333715.42 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 33371.54 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1923022.887546 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300343.88 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 30034.39 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953057.275742 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270309.49 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 27030.95 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980088.225119 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243278.54 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 24327.85 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004416.079557 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218950.69 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 21895.07 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026311.148552 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197055.62 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 19705.56 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046016.710648 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177350.06 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 17735.01 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063751.716534 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159615.05 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 15961.51 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079713.221831 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143653.55 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 14365.35 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094078.576598 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129288.19 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 12928.82 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107007.395889 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116359.37 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 11635.94 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118643.333251 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104723.44 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 10472.34 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129115.676877 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94251.09 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 9425.11 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138540.786140 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84825.98 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 8482.60 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147023.384476 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76343.39 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 7634.34 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154657.722979 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68709.05 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 6870.90 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161528.627632 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61838.14 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 6183.81 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167712.441819 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55654.33 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 5565.43 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173277.874588 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50088.89 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 5008.89 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178286.764080 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45080.01 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 4508.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182794.764623 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40572.00 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 4057.20 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186851.965111 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36514.80 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3651.48 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190503.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32863.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194103.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29263.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197703.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25663.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201303.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22063.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204903.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18463.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208503.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14863.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212103.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11263.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215703.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7663.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219303.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4063.32 sec) 15:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222903.445551 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 463.32 sec) 15:41:00 [GPUGRID] [rr_sim] 2223366.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977789763.25G/439.78G) 15:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852867.68 15:41:00 [---] pick_jobs_to_run() start 15:41:00 [---] pick_jobs_to_run() end 15:41:00 [---] [rr_sim] end 15:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:41:00 [---] [cpu_sched_debug] preliminary job list: 15:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:41:00 [---] [cpu_sched_debug] final job list: 15:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:41:00 [---] [cpu_sched_debug] enforce_run_list: end 15:41:00 [---] choose_project(): 56460.000000 15:41:00 [WUProp@Home] doing RPC to get work 15:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379139.09 cpu 2126372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126372.90 15:41:00 [---] Got 0 tasks 15:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:42:00 [---] [rr_sim] doing sim: CPU sched 15:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977762172.89G/439.78G) 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223304.03 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 222330.40 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222330.403262 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000973.63 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 200097.36 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422427.766199 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800876.27 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 180087.63 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602515.392841 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620788.64 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 162078.86 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764594.256820 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458709.78 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 145870.98 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910465.234400 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312838.80 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 131283.88 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041749.114222 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181554.92 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 118155.49 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159904.606063 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063399.43 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 106339.94 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266244.548719 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957059.48 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 95705.95 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361950.497109 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861353.54 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 86135.35 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448085.850661 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775218.18 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 77521.82 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525607.668857 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697696.36 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 69769.64 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595377.305234 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627926.73 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 62792.67 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658169.977973 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565134.05 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 56513.41 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714683.383438 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508620.65 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 50862.06 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765545.448357 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457758.58 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 45775.86 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811321.306784 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411982.73 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 41198.27 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852519.579368 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370784.45 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 37078.45 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889598.024693 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333706.01 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 33370.60 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922968.625486 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300335.41 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 30033.54 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953002.166200 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270301.87 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 27030.19 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980032.352843 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243271.68 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 24327.17 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004359.520821 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218944.51 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 21894.45 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026253.972001 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197050.06 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 19705.01 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045958.978064 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177345.05 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 17734.51 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063693.483520 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159610.55 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 15961.05 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079654.538430 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143649.49 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 14364.95 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094019.487850 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129284.54 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 12928.45 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106947.942327 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116356.09 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 11635.61 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118583.551357 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104720.48 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 10472.05 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129055.599484 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94248.43 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 9424.84 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138480.442798 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84823.59 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 8482.36 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146962.801780 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76341.23 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 7634.12 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154596.924865 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68707.11 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 6870.71 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161467.635641 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61836.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 6183.64 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167651.275339 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55652.76 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 5565.28 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173216.551068 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50087.48 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 5008.75 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178225.299223 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45078.73 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 4507.87 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182733.172563 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40570.86 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 4057.09 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186790.258569 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36513.77 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3651.38 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190441.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32862.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194041.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29262.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197641.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25662.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201241.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22062.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204841.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18462.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208441.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14862.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212041.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11262.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215641.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7662.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219241.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4062.40 sec) 15:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222841.635975 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 462.40 sec) 15:42:00 [GPUGRID] [rr_sim] 2223304.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977762172.89G/439.78G) 15:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852864.95 15:42:00 [---] pick_jobs_to_run() start 15:42:00 [---] pick_jobs_to_run() end 15:42:00 [---] [rr_sim] end 15:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:42:00 [---] [cpu_sched_debug] preliminary job list: 15:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:42:00 [---] [cpu_sched_debug] final job list: 15:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:42:00 [---] [cpu_sched_debug] enforce_run_list: end 15:42:00 [---] choose_project(): 56520.000000 15:42:00 [WUProp@Home] doing RPC to get work 15:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379079.09 cpu 2126312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126312.90 15:42:00 [---] Got 0 tasks 15:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:43:00 [---] [rr_sim] doing sim: CPU sched 15:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977734582.54G/439.78G) 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223241.30 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 222324.13 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222324.129574 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000917.17 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 200091.72 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422415.846191 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800825.45 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 180082.54 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602498.391146 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620742.90 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 162074.29 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764572.681606 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458668.61 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 145866.86 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910439.543019 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312801.75 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 131280.18 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041719.718292 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181521.58 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 118152.16 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159871.876037 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063369.42 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 106336.94 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266208.818007 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957032.48 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 95703.25 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361912.065781 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861329.23 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 86132.92 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448044.988777 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775196.31 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 77519.63 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525564.619473 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697676.68 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 69767.67 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595332.287100 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627909.01 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 62790.90 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658123.187965 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565118.11 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 56511.81 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714634.998742 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508606.30 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 50860.63 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765495.628442 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457745.67 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 45774.57 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811270.195172 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411971.10 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 41197.11 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852467.305229 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370773.99 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 37077.40 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889544.704281 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333696.59 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 33369.66 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922914.363427 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300326.93 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 30032.69 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952947.056658 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270294.24 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 27029.42 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979976.480567 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243264.82 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 24326.48 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004302.962084 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218938.33 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 21893.83 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026196.795450 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197044.50 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 19704.45 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045901.245479 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177340.05 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 17734.01 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063635.250506 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159606.05 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 15960.60 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079595.855029 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143645.44 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 14364.54 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093960.399101 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129280.90 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 12928.09 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106888.488765 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116352.81 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 11635.28 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118523.769462 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104717.53 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 10471.75 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128995.522090 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94245.77 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 9424.58 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138420.099456 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84821.20 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 8482.12 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146902.219084 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76339.08 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 7633.91 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154536.126750 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68705.17 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 6870.52 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161406.643649 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61834.65 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 6183.47 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167590.108859 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55651.19 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 5565.12 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173155.227547 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50086.07 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 5008.61 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178163.834366 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45077.46 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 4507.75 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182671.580504 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40569.72 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 4056.97 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186728.552028 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36512.74 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3651.27 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190379.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32861.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193979.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29261.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197579.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25661.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201179.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22061.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204779.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18461.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208379.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14861.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211979.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11261.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215579.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7661.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219179.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4061.47 sec) 15:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222779.826399 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 461.47 sec) 15:43:00 [GPUGRID] [rr_sim] 2223241.30: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977734582.54G/439.78G) 15:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852862.21 15:43:00 [---] pick_jobs_to_run() start 15:43:00 [---] pick_jobs_to_run() end 15:43:00 [---] [rr_sim] end 15:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:43:00 [---] [cpu_sched_debug] preliminary job list: 15:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:43:00 [---] [cpu_sched_debug] final job list: 15:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:43:00 [---] [cpu_sched_debug] enforce_run_list: end 15:43:00 [---] choose_project(): 56580.000000 15:43:00 [WUProp@Home] doing RPC to get work 15:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 379019.09 cpu 2126252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126252.90 15:43:00 [---] Got 0 tasks 15:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:44:00 [---] [rr_sim] doing sim: CPU sched 15:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977706992.18G/439.78G) 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223178.56 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 222317.86 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222317.855886 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000860.70 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 200086.07 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422403.926183 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800774.63 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 180077.46 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602481.389451 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620697.17 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 162069.72 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764551.106392 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458627.45 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 145862.75 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910413.851639 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312764.71 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 131276.47 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041690.322361 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181488.24 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 118148.82 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159839.146011 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063339.41 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 106333.94 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266173.087296 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 957005.47 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 95700.55 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361873.634452 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861304.92 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 86130.49 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1448004.126893 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775174.43 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 77517.44 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525521.570090 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697656.99 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 69765.70 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595287.268967 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627891.29 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 62789.13 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658076.397956 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565102.16 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 56510.22 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714586.614046 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508591.94 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 50859.19 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765445.808528 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457732.75 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 45773.28 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811219.083561 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411959.48 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 41195.95 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852415.031091 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370763.53 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 37076.35 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889491.383868 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333687.17 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 33368.72 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922860.101367 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300318.46 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 30031.85 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952891.947116 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270286.61 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 27028.66 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979920.608291 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243257.95 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 24325.80 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004246.403348 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218932.16 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 21893.22 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026139.618899 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197038.94 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 19703.89 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045843.512895 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177335.05 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 17733.50 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063577.017492 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159601.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 15960.15 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079537.171628 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143641.39 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 14364.14 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093901.310352 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129277.25 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 12927.72 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106829.035202 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116349.52 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 11634.95 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118463.987568 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104714.57 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 10471.46 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128935.444697 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94243.11 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 9424.31 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138359.756114 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84818.80 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 8481.88 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146841.636388 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76336.92 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 7633.69 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154475.328635 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68703.23 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 6870.32 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161345.651658 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61832.91 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 6183.29 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167528.942378 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55649.62 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 5564.96 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173093.904026 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50084.65 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 5008.47 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178102.369510 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45076.19 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 4507.62 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182609.988445 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40568.57 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 4056.86 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186666.845486 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36511.71 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3651.17 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190318.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32860.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193918.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29260.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197518.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25660.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201118.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22060.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204718.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18460.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208318.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14860.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211918.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11260.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215518.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7660.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219118.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4060.54 sec) 15:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222718.016824 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 460.54 sec) 15:44:00 [GPUGRID] [rr_sim] 2223178.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977706992.18G/439.78G) 15:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852859.47 15:44:00 [---] pick_jobs_to_run() start 15:44:00 [---] pick_jobs_to_run() end 15:44:00 [---] [rr_sim] end 15:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:44:00 [---] [cpu_sched_debug] preliminary job list: 15:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:44:00 [---] [cpu_sched_debug] final job list: 15:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:44:00 [---] [cpu_sched_debug] enforce_run_list: end 15:44:00 [---] choose_project(): 56640.000000 15:44:00 [WUProp@Home] doing RPC to get work 15:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378959.09 cpu 2126192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126192.90 15:44:00 [---] Got 0 tasks 15:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:45:00 [---] [rr_sim] doing sim: CPU sched 15:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977679401.83G/439.78G) 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223115.82 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 222311.58 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222311.582198 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000804.24 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 200080.42 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422392.006176 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800723.82 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 180072.38 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602464.387756 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620651.43 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 162065.14 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764529.531178 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458586.29 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 145858.63 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910388.160258 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312727.66 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 131272.77 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041660.926430 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181454.90 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 118145.49 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159806.415985 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063309.41 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 106330.94 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266137.356584 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956978.47 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 95697.85 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361835.203124 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861280.62 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 86128.06 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447963.265009 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775152.56 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 77515.26 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525478.520706 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697637.30 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 69763.73 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595242.250833 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627873.57 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 62787.36 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1658029.607948 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565086.21 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 56508.62 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714538.229351 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508577.59 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 50857.76 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765395.988613 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457719.83 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 45771.98 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811167.971950 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411947.85 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 41194.79 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852362.756953 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370753.07 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 37075.31 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889438.063455 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333677.76 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 33367.78 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922805.839307 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300309.98 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 30031.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952836.837574 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270278.98 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 27027.90 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979864.736015 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243251.09 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 24325.11 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004189.844611 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218925.98 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 21892.60 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026082.442348 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197033.38 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 19703.34 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045785.780311 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177330.04 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 17733.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063518.784478 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159597.04 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 15959.70 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079478.488228 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143637.33 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 14363.73 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093842.221603 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129273.60 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 12927.36 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106769.581640 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116346.24 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 11634.62 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118404.205674 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104711.62 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 10471.16 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128875.367304 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94240.45 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 9424.05 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138299.412772 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84816.41 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 8481.64 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146781.053692 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76334.77 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 7633.48 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154414.530521 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68701.29 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 6870.13 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161284.659667 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61831.16 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 6183.12 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167467.775898 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55648.05 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 5564.80 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173032.580506 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50083.24 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 5008.32 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178040.904653 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45074.92 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 4507.49 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182548.396385 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40567.43 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 4056.74 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186605.138945 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36510.68 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3651.07 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190256.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32859.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193856.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29259.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197456.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25659.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201056.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22059.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204656.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18459.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208256.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14859.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211856.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11259.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215456.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7659.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219056.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4059.61 sec) 15:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222656.207248 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 459.61 sec) 15:45:00 [GPUGRID] [rr_sim] 2223115.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977679401.83G/439.78G) 15:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852856.73 15:45:00 [---] pick_jobs_to_run() start 15:45:00 [---] pick_jobs_to_run() end 15:45:00 [---] [rr_sim] end 15:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:45:00 [---] [cpu_sched_debug] preliminary job list: 15:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:45:00 [---] [cpu_sched_debug] final job list: 15:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:45:00 [---] [cpu_sched_debug] enforce_run_list: end 15:45:00 [---] choose_project(): 56700.000000 15:45:00 [WUProp@Home] doing RPC to get work 15:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378899.09 cpu 2126132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126132.90 15:45:00 [---] Got 0 tasks 15:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:46:00 [---] [rr_sim] doing sim: CPU sched 15:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977651811.47G/439.78G) 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2223053.09 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 222305.31 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222305.308510 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000747.78 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 200074.78 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422380.086168 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800673.00 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 180067.30 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602447.386061 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620605.70 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 162060.57 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764507.955964 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458545.13 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 145854.51 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910362.468878 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312690.62 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 131269.06 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041631.530499 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181421.55 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 118142.16 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159773.685959 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063279.40 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 106327.94 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266101.625873 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956951.46 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 95695.15 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361796.771795 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861256.31 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 86125.63 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447922.403125 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775130.68 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 77513.07 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525435.471322 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697617.61 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 69761.76 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595197.232700 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627855.85 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 62785.59 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657982.817939 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565070.27 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 56507.03 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714489.844655 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508563.24 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 50856.32 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765346.168699 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457706.92 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 45770.69 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811116.860339 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411936.22 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 41193.62 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852310.482814 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370742.60 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 37074.26 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889384.743042 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333668.34 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 33366.83 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922751.577248 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300301.51 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 30030.15 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952781.728033 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270271.36 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 27027.14 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979808.863739 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243244.22 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 24324.42 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004133.285875 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218919.80 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 21891.98 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026025.265797 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197027.82 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 19702.78 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045728.047727 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177325.04 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 17732.50 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063460.551464 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159592.53 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 15959.25 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079419.804827 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143633.28 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 14363.33 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093783.132854 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129269.95 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 12927.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106710.128078 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116342.96 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 11634.30 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118344.423780 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104708.66 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 10470.87 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128815.289911 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94237.80 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 9423.78 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138239.069430 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84814.02 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 8481.40 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146720.470996 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76332.61 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 7633.26 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154353.732406 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68699.35 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 6869.94 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161223.667675 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61829.42 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 6182.94 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167406.609417 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55646.48 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 5564.65 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172971.256985 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50081.83 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 5008.18 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177979.439796 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45073.65 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 4507.36 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182486.804326 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40566.28 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 4056.63 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186543.432403 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36509.65 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3650.97 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190194.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32858.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193794.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29258.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197394.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25658.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200994.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22058.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204594.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18458.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208194.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14858.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211794.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11258.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215394.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7658.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218994.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4058.69 sec) 15:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222594.397672 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 458.69 sec) 15:46:00 [GPUGRID] [rr_sim] 2223053.09: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977651811.47G/439.78G) 15:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852854.00 15:46:00 [---] pick_jobs_to_run() start 15:46:00 [---] pick_jobs_to_run() end 15:46:00 [---] [rr_sim] end 15:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:46:00 [---] [cpu_sched_debug] preliminary job list: 15:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:46:00 [---] [cpu_sched_debug] final job list: 15:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:46:00 [---] [cpu_sched_debug] enforce_run_list: end 15:46:00 [---] choose_project(): 56760.000000 15:46:00 [WUProp@Home] doing RPC to get work 15:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378839.09 cpu 2126072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126072.90 15:46:00 [---] Got 0 tasks 15:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:47:00 [---] [rr_sim] doing sim: CPU sched 15:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977624221.11G/439.78G) 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222990.35 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 222299.03 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222299.034821 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000691.31 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 200069.13 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422368.166161 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800622.18 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 180062.22 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602430.384366 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620559.96 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 162056.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764486.380751 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458503.97 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 145850.40 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910336.777497 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312653.57 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 131265.36 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041602.134569 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181388.21 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 118138.82 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159740.955933 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063249.39 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 106324.94 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266065.895161 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956924.45 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 95692.45 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361758.340466 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861232.01 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 86123.20 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447881.541241 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775108.81 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 77510.88 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525392.421938 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697597.93 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 69759.79 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595152.214566 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627838.13 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 62783.81 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657936.027931 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565054.32 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 56505.43 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714441.459959 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508548.89 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 50854.89 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765296.348784 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457694.00 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 45769.40 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811065.748727 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411924.60 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 41192.46 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852258.208676 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370732.14 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 37073.21 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889331.422630 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333658.93 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 33365.89 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922697.315188 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300293.03 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 30029.30 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952726.618491 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270263.73 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 27026.37 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979752.991463 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243237.36 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 24323.74 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004076.727138 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218913.62 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 21891.36 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025968.089246 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197022.26 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 19702.23 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045670.315142 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177320.03 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 17732.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063402.318450 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159588.03 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 15958.80 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079361.121426 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143629.23 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 14362.92 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093724.044105 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129266.30 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 12926.63 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106650.674516 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116339.67 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 11633.97 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118284.641885 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104705.71 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 10470.57 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128755.212518 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94235.14 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 9423.51 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138178.726088 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84811.62 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 8481.16 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146659.888300 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76330.46 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 7633.05 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154292.934292 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68697.41 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 6869.74 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161162.675684 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61827.67 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 6182.77 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167345.442937 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55644.91 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 5564.49 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172909.933464 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50080.41 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 5008.04 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177917.974939 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45072.37 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 4507.24 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182425.212267 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40565.14 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 4056.51 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186481.725862 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36508.62 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3650.86 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190132.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32857.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193732.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29257.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197332.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25657.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200932.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22057.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204532.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18457.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208132.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14857.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211732.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11257.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215332.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7657.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218932.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4057.76 sec) 15:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222532.588097 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457.76 sec) 15:47:00 [GPUGRID] [rr_sim] 2222990.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977624221.11G/439.78G) 15:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852851.26 15:47:00 [---] pick_jobs_to_run() start 15:47:00 [---] pick_jobs_to_run() end 15:47:00 [---] [rr_sim] end 15:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:47:00 [---] [cpu_sched_debug] preliminary job list: 15:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:47:00 [---] [cpu_sched_debug] final job list: 15:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:47:00 [---] [cpu_sched_debug] enforce_run_list: end 15:47:00 [---] choose_project(): 56820.000000 15:47:00 [WUProp@Home] doing RPC to get work 15:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378779.09 cpu 2126012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2126012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2126012.90 15:47:00 [---] Got 0 tasks 15:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:48:00 [---] [rr_sim] doing sim: CPU sched 15:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977596630.76G/439.78G) 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222927.61 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 222292.76 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222292.761133 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000634.85 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 200063.49 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422356.246153 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800571.37 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 180057.14 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602413.382671 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620514.23 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 162051.42 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764464.805537 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458462.81 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 145846.28 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910311.086116 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312616.53 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 131261.65 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041572.738638 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181354.87 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 118135.49 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159708.225907 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063219.39 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 106321.94 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1266030.164450 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956897.45 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 95689.74 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361719.909138 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861207.70 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 86120.77 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447840.679357 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775086.93 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 77508.69 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525349.372555 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697578.24 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 69757.82 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595107.196432 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627820.41 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 62782.04 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657889.237922 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565038.37 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 56503.84 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714393.075263 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508534.54 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 50853.45 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765246.528870 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457681.08 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 45768.11 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1811014.637116 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411912.97 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 41191.30 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852205.934538 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370721.68 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 37072.17 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889278.102217 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333649.51 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 33364.95 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922643.053128 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300284.56 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 30028.46 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952671.508949 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270256.10 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 27025.61 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979697.119187 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243230.49 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 24323.05 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004020.168401 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218907.44 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 21890.74 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025910.912694 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197016.70 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 19701.67 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045612.582558 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177315.03 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 17731.50 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063344.085435 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159583.53 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 15958.35 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079302.438025 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143625.17 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 14362.52 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093664.955356 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129262.66 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 12926.27 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106591.220953 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116336.39 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 11633.64 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118224.859991 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104702.75 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 10470.28 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128695.135125 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94232.48 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 9423.25 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138118.382746 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84809.23 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 8480.92 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146599.305604 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76328.31 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 7632.83 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154232.136177 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68695.48 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 6869.55 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161101.683692 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61825.93 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 6182.59 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167284.276456 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55643.33 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 5564.33 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172848.609944 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50079.00 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 5007.90 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177856.510083 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45071.10 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 4507.11 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182363.620208 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40563.99 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 4056.40 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186420.019320 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36507.59 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3650.76 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190070.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32856.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193670.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29256.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197270.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25656.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200870.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22056.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204470.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18456.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208070.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14856.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211670.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11256.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215270.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7656.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218870.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4056.83 sec) 15:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222470.778521 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456.83 sec) 15:48:00 [GPUGRID] [rr_sim] 2222927.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977596630.76G/439.78G) 15:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852848.52 15:48:00 [---] pick_jobs_to_run() start 15:48:00 [---] pick_jobs_to_run() end 15:48:00 [---] [rr_sim] end 15:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:48:00 [---] [cpu_sched_debug] preliminary job list: 15:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:48:00 [---] [cpu_sched_debug] final job list: 15:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:48:00 [---] [cpu_sched_debug] enforce_run_list: end 15:48:00 [---] choose_project(): 56880.000000 15:48:00 [WUProp@Home] doing RPC to get work 15:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378719.09 cpu 2125952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125952.90 15:48:00 [---] Got 0 tasks 15:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:49:00 [---] [rr_sim] doing sim: CPU sched 15:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977569040.40G/439.78G) 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222864.87 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 222286.49 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222286.487445 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000578.39 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 200057.84 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422344.326145 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800520.55 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 180052.05 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602396.380976 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620468.49 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 162046.85 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764443.230323 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458421.64 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 145842.16 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910285.394736 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312579.48 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 131257.95 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041543.342707 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181321.53 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 118132.15 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159675.495881 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063189.38 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 106318.94 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265994.433738 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956870.44 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 95687.04 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361681.477809 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861183.40 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 86118.34 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447799.817473 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775065.06 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 77506.51 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525306.323171 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697558.55 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 69755.86 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595062.178299 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627802.70 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 62780.27 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657842.447914 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565022.43 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 56502.24 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714344.690567 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508520.18 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 50852.02 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765196.708955 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457668.17 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 45766.82 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810963.525505 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411901.35 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 41190.13 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852153.660399 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370711.21 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 37071.12 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889224.781804 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333640.09 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 33364.01 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922588.791069 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300276.08 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 30027.61 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952616.399407 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270248.48 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 27024.85 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979641.246911 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243223.63 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 24322.36 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003963.609665 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218901.26 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 21890.13 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025853.736143 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197011.14 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 19701.11 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045554.849974 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177310.02 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 17731.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063285.852421 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159579.02 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 15957.90 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079243.754624 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143621.12 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 14362.11 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093605.866607 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129259.01 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 12925.90 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106531.767391 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116333.11 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 11633.31 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118165.078097 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104699.80 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 10469.98 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128635.057732 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94229.82 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 9422.98 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138058.039404 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84806.84 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 8480.68 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146538.722908 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76326.15 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 7632.62 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154171.338062 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68693.54 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 6869.35 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161040.691701 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61824.18 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 6182.42 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167223.109976 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55641.76 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 5564.18 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172787.286423 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50077.59 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 5007.76 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177795.045226 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45069.83 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 4506.98 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182302.028148 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40562.85 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 4056.28 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186358.312778 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36506.56 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3650.66 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190008.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32855.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193608.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29255.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197208.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25655.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200808.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22055.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204408.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18455.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208008.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14855.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211608.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11255.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215208.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7655.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218808.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4055.91 sec) 15:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222408.968945 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455.91 sec) 15:49:00 [GPUGRID] [rr_sim] 2222864.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977569040.40G/439.78G) 15:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852845.79 15:49:00 [---] pick_jobs_to_run() start 15:49:00 [---] pick_jobs_to_run() end 15:49:00 [---] [rr_sim] end 15:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:49:00 [---] [cpu_sched_debug] preliminary job list: 15:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:49:00 [---] [cpu_sched_debug] final job list: 15:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:49:00 [---] [cpu_sched_debug] enforce_run_list: end 15:49:00 [---] choose_project(): 56940.000000 15:49:00 [WUProp@Home] doing RPC to get work 15:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378659.09 cpu 2125892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125892.90 15:49:00 [---] Got 0 tasks 15:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:50:00 [---] [rr_sim] doing sim: CPU sched 15:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977541450.04G/439.78G) 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222802.14 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 222280.21 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222280.213757 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000521.92 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 200052.19 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422332.406138 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800469.73 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 180046.97 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602379.379281 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620422.76 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 162042.28 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764421.655109 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458380.48 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 145838.05 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910259.703355 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312542.43 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 131254.24 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041513.946776 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181288.19 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 118128.82 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159642.765855 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063159.37 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 106315.94 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265958.703027 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956843.43 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 95684.34 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361643.046481 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861159.09 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 86115.91 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447758.955589 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775043.18 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 77504.32 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525263.273787 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697538.86 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 69753.89 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1595017.160165 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627784.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 62778.50 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657795.657905 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 565006.48 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 56500.65 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714296.305871 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508505.83 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 50850.58 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765146.889041 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457655.25 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 45765.52 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810912.413894 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411889.72 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 41188.97 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852101.386261 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370700.75 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 37070.08 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889171.461392 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333630.68 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 33363.07 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922534.529009 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300267.61 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 30026.76 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952561.289865 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270240.85 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 27024.08 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979585.374635 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243216.76 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 24321.68 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003907.050928 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218895.09 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 21889.51 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025796.559592 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197005.58 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 19700.56 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045497.117390 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177305.02 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 17730.50 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063227.619407 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159574.52 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 15957.45 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079185.071223 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143617.07 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 14361.71 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093546.777858 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129255.36 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 12925.54 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106472.313829 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116329.82 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 11632.98 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118105.296203 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104696.84 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 10469.68 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128574.980339 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94227.16 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 9422.72 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137997.696062 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84804.44 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 8480.44 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146478.140212 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76324.00 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 7632.40 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154110.539948 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68691.60 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 6869.16 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160979.699710 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61822.44 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 6182.24 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167161.943495 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55640.19 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 5564.02 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172725.962903 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50076.17 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 5007.62 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177733.580369 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45068.56 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 4506.86 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182240.436089 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40561.70 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 4056.17 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186296.606237 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36505.53 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3650.55 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189947.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32854.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193547.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29254.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197147.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25654.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200747.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22054.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204347.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18454.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207947.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14854.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211547.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11254.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215147.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7654.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218747.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4054.98 sec) 15:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222347.159370 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454.98 sec) 15:50:00 [GPUGRID] [rr_sim] 2222802.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977541450.04G/439.78G) 15:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852843.05 15:50:00 [---] pick_jobs_to_run() start 15:50:00 [---] pick_jobs_to_run() end 15:50:00 [---] [rr_sim] end 15:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:50:00 [---] [cpu_sched_debug] preliminary job list: 15:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:50:00 [---] [cpu_sched_debug] final job list: 15:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:50:00 [---] [cpu_sched_debug] enforce_run_list: end 15:50:00 [---] choose_project(): 57000.000000 15:50:00 [WUProp@Home] doing RPC to get work 15:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378599.09 cpu 2125832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125832.90 15:50:00 [---] Got 0 tasks 15:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:51:00 [---] [rr_sim] doing sim: CPU sched 15:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977513859.69G/439.78G) 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222739.40 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 222273.94 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222273.940068 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000465.46 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 200046.55 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422320.486130 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800418.91 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 180041.89 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602362.377586 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620377.02 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 162037.70 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764400.079896 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458339.32 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 145833.93 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910234.011974 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312505.39 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 131250.54 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041484.550846 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181254.85 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 118125.48 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159610.035829 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063129.36 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 106312.94 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265922.972315 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956816.43 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 95681.64 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361604.615152 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861134.79 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 86113.48 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447718.093705 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 775021.31 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 77502.13 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525220.224403 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697519.18 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 69751.92 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594972.142031 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627767.26 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 62776.73 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657748.867897 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564990.53 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 56499.05 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714247.921176 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508491.48 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 50849.15 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765097.069127 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457642.33 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 45764.23 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810861.302282 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411878.10 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 41187.81 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1852049.112123 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370690.29 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 37069.03 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889118.140979 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333621.26 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 33362.13 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922480.266949 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300259.13 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 30025.91 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952506.180323 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270233.22 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 27023.32 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979529.502359 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243209.90 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 24320.99 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003850.492192 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218888.91 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 21888.89 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025739.383041 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197000.02 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 19700.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045439.384805 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177300.02 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 17730.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063169.386393 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159570.01 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 15957.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079126.387823 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143613.01 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 14361.30 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093487.689109 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129251.71 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 12925.17 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106412.860266 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116326.54 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 11632.65 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118045.514308 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104693.89 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 10469.39 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128514.902946 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94224.50 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 9422.45 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137937.352720 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84802.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 8480.20 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146417.557516 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76321.84 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 7632.18 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154049.741833 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68689.66 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 6868.97 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160918.707718 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61820.69 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 6182.07 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167100.777015 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55638.62 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 5563.86 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172664.639382 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50074.76 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 5007.48 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177672.115512 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45067.29 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 4506.73 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182178.844030 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40560.56 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 4056.06 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186234.899695 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36504.50 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3650.45 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189885.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32854.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193485.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29254.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197085.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25654.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200685.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22054.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204285.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18454.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207885.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14854.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211485.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11254.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215085.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7654.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218685.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4054.05 sec) 15:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222285.349794 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454.05 sec) 15:51:00 [GPUGRID] [rr_sim] 2222739.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977513859.69G/439.78G) 15:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852840.31 15:51:00 [---] pick_jobs_to_run() start 15:51:00 [---] pick_jobs_to_run() end 15:51:00 [---] [rr_sim] end 15:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:51:00 [---] [cpu_sched_debug] preliminary job list: 15:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:51:00 [---] [cpu_sched_debug] final job list: 15:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:51:00 [---] [cpu_sched_debug] enforce_run_list: end 15:51:00 [---] choose_project(): 57060.000000 15:51:00 [WUProp@Home] doing RPC to get work 15:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378539.09 cpu 2125772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125772.90 15:51:00 [---] Got 0 tasks 15:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:52:00 [---] [rr_sim] doing sim: CPU sched 15:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977486269.33G/439.78G) 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222676.66 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 222267.67 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222267.666380 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000409.00 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 200040.90 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422308.566123 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800368.10 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 180036.81 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602345.375891 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620331.29 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 162033.13 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764378.504682 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458298.16 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 145829.82 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910208.320594 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312468.34 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 131246.83 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041455.154915 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181221.51 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 118122.15 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159577.305804 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063099.36 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 106309.94 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265887.241603 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956789.42 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 95678.94 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361566.183823 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861110.48 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 86111.05 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447677.231821 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774999.43 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 77499.94 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525177.175019 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697499.49 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 69749.95 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594927.123898 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627749.54 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 62774.95 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657702.077888 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564974.59 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 56497.46 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714199.536480 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508477.13 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 50847.71 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1765047.249212 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457629.41 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 45762.94 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810810.190671 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411866.47 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 41186.65 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851996.837984 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370679.83 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 37067.98 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889064.820566 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333611.84 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 33361.18 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922426.004890 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300250.66 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 30025.07 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952451.070781 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270225.59 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 27022.56 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979473.630083 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243203.03 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 24320.30 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003793.933455 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218882.73 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 21888.27 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025682.206490 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196994.46 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 19699.45 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045381.652221 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177295.01 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 17729.50 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063111.153379 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159565.51 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 15956.55 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079067.704422 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143608.96 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 14360.90 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093428.600360 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129248.06 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 12924.81 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106353.406704 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116323.26 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 11632.33 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117985.732414 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104690.93 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 10469.09 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128454.825553 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94221.84 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 9422.18 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137877.009378 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84799.65 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 8479.97 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146356.974820 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76319.69 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 7631.97 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153988.943719 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68687.72 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 6868.77 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160857.715727 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61818.95 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 6181.89 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167039.610535 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55637.05 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 5563.71 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172603.315861 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50073.35 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 5007.33 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177610.650656 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45066.01 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 4506.60 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182117.251970 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40559.41 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 4055.94 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186173.193154 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36503.47 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3650.35 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189823.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32853.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193423.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29253.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197023.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25653.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200623.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22053.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204223.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18453.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207823.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14853.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211423.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11253.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215023.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7653.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218623.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4053.12 sec) 15:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222223.540218 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453.12 sec) 15:52:00 [GPUGRID] [rr_sim] 2222676.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977486269.33G/439.78G) 15:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852837.58 15:52:00 [---] pick_jobs_to_run() start 15:52:00 [---] pick_jobs_to_run() end 15:52:00 [---] [rr_sim] end 15:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:52:00 [---] [cpu_sched_debug] preliminary job list: 15:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:52:00 [---] [cpu_sched_debug] final job list: 15:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:52:00 [---] [cpu_sched_debug] enforce_run_list: end 15:52:00 [---] choose_project(): 57120.000000 15:52:00 [WUProp@Home] doing RPC to get work 15:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378479.09 cpu 2125712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125712.90 15:52:00 [---] Got 0 tasks 15:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:53:00 [---] [rr_sim] doing sim: CPU sched 15:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977458678.98G/439.78G) 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222613.93 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 222261.39 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222261.392692 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000352.53 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 200035.25 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422296.646115 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800317.28 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 180031.73 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602328.374195 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620285.55 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 162028.56 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764356.929468 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458257.00 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 145825.70 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910182.629213 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312431.30 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 131243.13 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041425.758984 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181188.17 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 118118.82 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159544.575778 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063069.35 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 106306.94 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265851.510892 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956762.42 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 95676.24 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361527.752495 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861086.17 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 86108.62 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447636.369937 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774977.56 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 77497.76 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525134.125636 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697479.80 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 69747.98 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594882.105764 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627731.82 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 62773.18 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657655.287880 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564958.64 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 56495.86 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714151.151784 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508462.78 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 50846.28 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764997.429298 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457616.50 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 45761.65 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810759.079060 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411854.85 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 41185.48 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851944.563846 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370669.36 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 37066.94 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1889011.500153 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333602.43 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 33360.24 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922371.742830 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300242.18 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 30024.22 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952395.961239 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270217.97 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 27021.80 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979417.757807 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243196.17 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 24319.62 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003737.374719 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218876.55 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 21887.66 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025625.029939 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196988.90 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 19698.89 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045323.919637 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177290.01 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 17729.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063052.920365 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159561.01 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 15956.10 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079009.021021 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143604.91 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 14360.49 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093369.511611 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129244.42 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 12924.44 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106293.953142 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116319.97 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 11632.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117925.950520 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104687.98 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 10468.80 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128394.748160 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94219.18 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 9421.92 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137816.666036 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84797.26 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 8479.73 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146296.392124 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76317.53 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 7631.75 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153928.145604 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68685.78 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 6868.58 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160796.723736 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61817.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 6181.72 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166978.444054 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55635.48 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 5563.55 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172541.992341 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50071.93 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 5007.19 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177549.185799 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45064.74 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 4506.47 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182055.659911 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40558.27 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 4055.83 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186111.486612 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36502.44 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3650.24 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189761.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32852.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193361.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29252.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196961.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25652.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200561.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22052.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204161.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18452.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207761.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14852.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211361.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11252.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214961.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7652.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218561.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4052.20 sec) 15:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222161.730643 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452.20 sec) 15:53:00 [GPUGRID] [rr_sim] 2222613.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977458678.98G/439.78G) 15:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852834.84 15:53:00 [---] pick_jobs_to_run() start 15:53:00 [---] pick_jobs_to_run() end 15:53:00 [---] [rr_sim] end 15:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:53:00 [---] [cpu_sched_debug] preliminary job list: 15:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:53:00 [---] [cpu_sched_debug] final job list: 15:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:53:00 [---] [cpu_sched_debug] enforce_run_list: end 15:53:00 [---] choose_project(): 57180.000000 15:53:00 [WUProp@Home] doing RPC to get work 15:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378419.09 cpu 2125652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125652.90 15:53:00 [---] Got 0 tasks 15:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:54:00 [---] [rr_sim] doing sim: CPU sched 15:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977431088.62G/439.78G) 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222551.19 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 222255.12 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222255.119004 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000296.07 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 200029.61 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422284.726107 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800266.46 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 180026.65 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602311.372500 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620239.82 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 162023.98 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764335.354254 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458215.84 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 145821.58 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910156.937833 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312394.25 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 131239.43 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041396.363053 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181154.83 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 118115.48 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159511.845752 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063039.34 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 106303.93 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265815.780180 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956735.41 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 95673.54 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361489.321166 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861061.87 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 86106.19 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447595.508053 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774955.68 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 77495.57 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525091.076252 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697460.11 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 69746.01 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594837.087631 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627714.10 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 62771.41 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657608.497871 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564942.69 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 56494.27 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714102.767088 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508448.42 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 50844.84 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764947.609383 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457603.58 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 45760.36 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810707.967449 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411843.22 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 41184.32 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851892.289708 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370658.90 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 37065.89 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888958.179741 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333593.01 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 33359.30 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922317.480770 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300233.71 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 30023.37 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952340.851697 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270210.34 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 27021.03 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979361.885531 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243189.30 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 24318.93 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003680.815982 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218870.37 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 21887.04 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025567.853388 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196983.34 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 19698.33 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045266.187053 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177285.00 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 17728.50 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062994.687351 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159556.50 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 15955.65 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078950.337620 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143600.85 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 14360.09 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093310.422862 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129240.77 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 12924.08 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106234.499580 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116316.69 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 11631.67 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117866.168625 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104685.02 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 10468.50 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128334.670767 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94216.52 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 9421.65 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137756.322694 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84794.87 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 8479.49 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146235.809428 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76315.38 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 7631.54 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153867.347489 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68683.84 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 6868.38 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160735.731744 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61815.46 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 6181.55 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166917.277574 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55633.91 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 5563.39 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172480.668820 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50070.52 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 5007.05 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177487.720942 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45063.47 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 4506.35 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181994.067852 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40557.12 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 4055.71 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186049.780070 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36501.41 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3650.14 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189699.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32851.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193299.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29251.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196899.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25651.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200499.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22051.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204099.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18451.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207699.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14851.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211299.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11251.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214899.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7651.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218499.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4051.27 sec) 15:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222099.921067 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451.27 sec) 15:54:00 [GPUGRID] [rr_sim] 2222551.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977431088.62G/439.78G) 15:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852832.10 15:54:00 [---] pick_jobs_to_run() start 15:54:00 [---] pick_jobs_to_run() end 15:54:00 [---] [rr_sim] end 15:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:54:00 [---] [cpu_sched_debug] preliminary job list: 15:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:54:00 [---] [cpu_sched_debug] final job list: 15:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:54:00 [---] [cpu_sched_debug] enforce_run_list: end 15:54:00 [---] choose_project(): 57240.000000 15:54:00 [WUProp@Home] doing RPC to get work 15:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378359.09 cpu 2125592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125592.90 15:54:00 [---] Got 0 tasks 15:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:55:00 [---] [rr_sim] doing sim: CPU sched 15:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977403498.26G/439.78G) 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222488.45 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 222248.85 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222248.845316 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000239.61 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 200023.96 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422272.806100 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800215.65 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 180021.56 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602294.370805 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620194.08 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 162019.41 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764313.779040 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458174.67 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 145817.47 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910131.246452 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312357.21 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 131235.72 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041366.967122 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181121.49 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 118112.15 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159479.115726 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1063009.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 106300.93 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265780.049469 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956708.40 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 95670.84 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361450.889838 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861037.56 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 86103.76 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447554.646169 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774933.81 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 77493.38 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525048.026868 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697440.43 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 69744.04 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594792.069497 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627696.38 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 62769.64 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657561.707863 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564926.75 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 56492.67 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714054.382392 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508434.07 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 50843.41 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764897.789469 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457590.66 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 45759.07 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810656.855837 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411831.60 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 41183.16 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851840.015569 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370648.44 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 37064.84 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888904.859328 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333583.59 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 33358.36 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922263.218711 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300225.23 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 30022.52 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952285.742155 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270202.71 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 27020.27 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979306.013255 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243182.44 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 24318.24 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003624.257246 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218864.20 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 21886.42 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025510.676837 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196977.78 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 19697.78 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045208.454469 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177280.00 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 17728.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062936.454337 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159552.00 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 15955.20 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078891.654219 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143596.80 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 14359.68 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093251.334113 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129237.12 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 12923.71 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106175.046017 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116313.41 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 11631.34 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117806.386731 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104682.07 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 10468.21 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128274.593374 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94213.86 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 9421.39 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137695.979352 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84792.47 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 8479.25 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146175.226732 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76313.23 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 7631.32 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153806.549375 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68681.90 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 6868.19 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160674.739753 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61813.71 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 6181.37 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166856.111093 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55632.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 5563.23 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172419.345300 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50069.11 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 5006.91 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177426.256085 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45062.20 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 4506.22 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181932.475792 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40555.98 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 4055.60 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185988.073529 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36500.38 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3650.04 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189638.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32850.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193238.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29250.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196838.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25650.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200438.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22050.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204038.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18450.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207638.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14850.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211238.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11250.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214838.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7650.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218438.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4050.34 sec) 15:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2222038.111491 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 450.34 sec) 15:55:00 [GPUGRID] [rr_sim] 2222488.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977403498.26G/439.78G) 15:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852829.37 15:55:00 [---] pick_jobs_to_run() start 15:55:00 [---] pick_jobs_to_run() end 15:55:00 [---] [rr_sim] end 15:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:55:00 [---] [cpu_sched_debug] preliminary job list: 15:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:55:00 [---] [cpu_sched_debug] final job list: 15:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:55:00 [---] [cpu_sched_debug] enforce_run_list: end 15:55:00 [---] choose_project(): 57300.000000 15:55:00 [WUProp@Home] doing RPC to get work 15:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378299.09 cpu 2125532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125532.90 15:55:00 [---] Got 0 tasks 15:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:56:00 [---] [rr_sim] doing sim: CPU sched 15:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977375907.91G/439.78G) 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222425.72 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 222242.57 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222242.571627 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000183.14 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 200018.31 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422260.886092 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800164.83 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 180016.48 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602277.369110 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620148.35 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 162014.83 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764292.203827 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458133.51 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 145813.35 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910105.555071 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312320.16 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 131232.02 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041337.571192 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181088.15 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 118108.81 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159446.385700 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062979.33 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 106297.93 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265744.318757 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956681.40 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 95668.14 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361412.458509 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 861013.26 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 86101.33 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447513.784286 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774911.93 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 77491.19 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1525004.977484 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697420.74 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 69742.07 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594747.051363 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627678.66 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 62767.87 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657514.917854 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564910.80 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 56491.08 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1714005.997696 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508419.72 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 50841.97 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764847.969554 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457577.75 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 45757.77 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810605.744226 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411819.97 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 41182.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851787.741431 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370637.97 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 37063.80 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888851.538915 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333574.18 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 33357.42 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922208.956651 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300216.76 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 30021.68 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952230.632613 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270195.08 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 27019.51 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979250.140980 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243175.58 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 24317.56 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003567.698509 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218858.02 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 21885.80 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025453.500285 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196972.22 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 19697.22 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045150.721884 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177274.99 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 17727.50 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062878.221323 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159547.49 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 15954.75 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078832.970818 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143592.75 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 14359.27 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093192.245364 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129233.47 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 12923.35 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106115.592455 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116310.12 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 11631.01 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117746.604837 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104679.11 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 10467.91 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128214.515981 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94211.20 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 9421.12 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137635.636010 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84790.08 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 8479.01 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146114.644036 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76311.07 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 7631.11 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153745.751260 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68679.97 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 6868.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160613.747762 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61811.97 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 6181.20 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166794.944613 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55630.77 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 5563.08 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172358.021779 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50067.69 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 5006.77 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177364.791228 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45060.93 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 4506.09 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181870.883733 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40554.83 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 4055.48 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185926.366987 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36499.35 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3649.93 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189576.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32849.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193176.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29249.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196776.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25649.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200376.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22049.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203976.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18449.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207576.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14849.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211176.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11249.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214776.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7649.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218376.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4049.41 sec) 15:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221976.301916 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 449.41 sec) 15:56:00 [GPUGRID] [rr_sim] 2222425.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977375907.91G/439.78G) 15:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852826.63 15:56:00 [---] pick_jobs_to_run() start 15:56:00 [---] pick_jobs_to_run() end 15:56:00 [---] [rr_sim] end 15:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:56:00 [---] [cpu_sched_debug] preliminary job list: 15:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:56:00 [---] [cpu_sched_debug] final job list: 15:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:56:00 [---] [cpu_sched_debug] enforce_run_list: end 15:56:00 [---] choose_project(): 57360.000000 15:56:00 [WUProp@Home] doing RPC to get work 15:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378239.09 cpu 2125472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125472.90 15:56:00 [---] Got 0 tasks 15:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:57:00 [---] [rr_sim] doing sim: CPU sched 15:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977348317.55G/439.78G) 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222362.98 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 222236.30 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222236.297939 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000126.68 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 200012.67 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422248.966084 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800114.01 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 180011.40 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602260.367415 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620102.61 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 162010.26 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764270.628613 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458092.35 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 145809.24 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910079.863691 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312283.12 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 131228.31 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041308.175261 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181054.80 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 118105.48 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159413.655674 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062949.32 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 106294.93 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265708.588046 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956654.39 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 95665.44 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361374.027180 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860988.95 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 86098.90 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447472.922402 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774890.06 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 77489.01 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524961.928101 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697401.05 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 69740.11 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594702.033230 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627660.95 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 62766.09 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657468.127846 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564894.85 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 56489.49 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713957.613001 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508405.37 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 50840.54 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764798.149640 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457564.83 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 45756.48 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810554.632615 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411808.35 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 41180.83 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851735.467293 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370627.51 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 37062.75 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888798.218503 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333564.76 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 33356.48 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922154.694592 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300208.28 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 30020.83 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952175.523072 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270187.46 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 27018.75 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979194.268704 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243168.71 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 24316.87 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003511.139772 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218851.84 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 21885.18 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025396.323734 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196966.66 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 19696.67 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045092.989300 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177269.99 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 17727.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062819.988309 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159542.99 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 15954.30 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078774.287418 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143588.69 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 14358.87 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093133.156615 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129229.82 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 12922.98 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106056.138893 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116306.84 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 11630.68 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117686.822943 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104676.16 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 10467.62 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128154.438588 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94208.54 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 9420.85 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137575.292668 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84787.69 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 8478.77 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146054.061340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76308.92 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 7630.89 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153684.953146 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68678.03 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 6867.80 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160552.755770 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61810.22 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 6181.02 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166733.778132 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55629.20 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 5562.92 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172296.698258 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50066.28 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 5006.63 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177303.326372 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45059.65 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 4505.97 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181809.291674 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40553.69 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 4055.37 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185864.660446 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36498.32 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3649.83 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189514.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32848.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193114.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29248.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196714.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25648.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200314.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22048.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203914.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18448.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207514.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14848.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211114.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11248.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214714.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7648.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218314.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4048.49 sec) 15:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221914.492340 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 448.49 sec) 15:57:00 [GPUGRID] [rr_sim] 2222362.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977348317.55G/439.78G) 15:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852823.89 15:57:00 [---] pick_jobs_to_run() start 15:57:00 [---] pick_jobs_to_run() end 15:57:00 [---] [rr_sim] end 15:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:57:00 [---] [cpu_sched_debug] preliminary job list: 15:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:57:00 [---] [cpu_sched_debug] final job list: 15:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:57:00 [---] [cpu_sched_debug] enforce_run_list: end 15:57:00 [---] choose_project(): 57420.000000 15:57:00 [WUProp@Home] doing RPC to get work 15:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378179.09 cpu 2125412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125412.90 15:57:00 [---] Got 0 tasks 15:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:58:00 [---] [rr_sim] doing sim: CPU sched 15:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977320727.19G/439.78G) 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222300.24 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 222230.02 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222230.024251 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000070.22 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 200007.02 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422237.046077 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800063.20 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 180006.32 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602243.365720 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620056.88 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 162005.69 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764249.053399 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458051.19 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 145805.12 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910054.172310 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312246.07 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 131224.61 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041278.779330 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1181021.46 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 118102.15 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159380.925648 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062919.32 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 106291.93 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265672.857334 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956627.39 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 95662.74 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361335.595852 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860964.65 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 86096.46 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447432.060518 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774868.18 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 77486.82 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524918.878717 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697381.36 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 69738.14 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594657.015096 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627643.23 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 62764.32 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657421.337837 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564878.90 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 56487.89 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713909.228305 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508391.01 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 50839.10 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764748.329725 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457551.91 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 45755.19 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810503.521004 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411796.72 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 41179.67 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851683.193154 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370617.05 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 37061.70 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888744.898090 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333555.34 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 33355.53 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922100.432532 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300199.81 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 30019.98 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952120.413530 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270179.83 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 27017.98 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979138.396428 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243161.85 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 24316.18 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003454.581036 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218845.66 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 21884.57 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025339.147183 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196961.10 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 19696.11 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045035.256716 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177264.99 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 17726.50 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062761.755295 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159538.49 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 15953.85 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078715.604017 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143584.64 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 14358.46 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093074.067866 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129226.17 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 12922.62 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105996.685330 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116303.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 11630.36 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117627.041048 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104673.20 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 10467.32 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128094.361195 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94205.88 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 9420.59 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137514.949326 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84785.29 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 8478.53 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145993.478644 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76306.76 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 7630.68 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153624.155031 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68676.09 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 6867.61 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160491.763779 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61808.48 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 6180.85 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166672.611652 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55627.63 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 5562.76 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172235.374738 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50064.87 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 5006.49 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177241.861515 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45058.38 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 4505.84 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181747.699614 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40552.54 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 4055.25 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185802.953904 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36497.29 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3649.73 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189452.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32847.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193052.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29247.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196652.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25647.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200252.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22047.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203852.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18447.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207452.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14847.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211052.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11247.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214652.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7647.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218252.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4047.56 sec) 15:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221852.682765 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 447.56 sec) 15:58:00 [GPUGRID] [rr_sim] 2222300.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977320727.19G/439.78G) 15:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852821.16 15:58:00 [---] pick_jobs_to_run() start 15:58:00 [---] pick_jobs_to_run() end 15:58:00 [---] [rr_sim] end 15:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:58:00 [---] [cpu_sched_debug] preliminary job list: 15:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:58:00 [---] [cpu_sched_debug] final job list: 15:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:58:00 [---] [cpu_sched_debug] enforce_run_list: end 15:58:00 [---] choose_project(): 57480.000000 15:58:00 [WUProp@Home] doing RPC to get work 15:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378119.09 cpu 2125352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125352.90 15:58:00 [---] Got 0 tasks 15:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 15:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 15:59:00 [---] [rr_sim] doing sim: CPU sched 15:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977293136.84G/439.78G) 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222237.51 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 222223.75 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222223.750563 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2000013.76 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 200001.38 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422225.126069 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1800012.38 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 180001.24 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602226.364025 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1620011.14 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 162001.11 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764227.478185 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1458010.03 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 145801.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910028.480930 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312209.02 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 131220.90 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041249.383399 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180988.12 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 118098.81 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159348.195622 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062889.31 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 106288.93 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265637.126623 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956600.38 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 95660.04 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361297.164523 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860940.34 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 86094.03 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447391.198634 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774846.31 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 77484.63 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524875.829333 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697361.68 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 69736.17 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594611.996963 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627625.51 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 62762.55 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657374.547829 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564862.96 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 56486.30 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713860.843609 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508376.66 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 50837.67 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764698.509811 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457539.00 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 45753.90 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810452.409392 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411785.10 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 41178.51 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851630.919016 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370606.59 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 37060.66 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888691.577677 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333545.93 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 33354.59 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1922046.170472 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300191.34 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 30019.13 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952065.303988 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270172.20 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 27017.22 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979082.524152 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243154.98 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 24315.50 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003398.022299 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218839.48 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 21883.95 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025281.970632 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196955.53 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 19695.55 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044977.524132 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177259.98 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 17726.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062703.522281 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159533.98 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 15953.40 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078656.920616 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143580.59 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 14358.06 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093014.979117 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129222.53 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 12922.25 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105937.231768 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116300.27 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 11630.03 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117567.259154 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104670.25 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 10467.02 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128034.283801 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94203.22 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 9420.32 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137454.605984 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84782.90 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 8478.29 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145932.895948 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76304.61 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 7630.46 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153563.356916 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68674.15 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 6867.41 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160430.771787 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61806.73 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 6180.67 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166611.445171 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55626.06 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 5562.61 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172174.051217 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50063.45 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 5006.35 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177180.396658 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45057.11 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 4505.71 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181686.107555 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40551.40 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 4055.14 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185741.247362 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36496.26 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3649.63 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189390.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32846.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192990.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29246.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196590.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25646.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200190.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22046.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203790.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18446.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207390.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14846.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210990.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11246.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214590.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7646.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218190.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4046.63 sec) 15:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221790.873189 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 446.63 sec) 15:59:00 [GPUGRID] [rr_sim] 2222237.51: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977293136.84G/439.78G) 15:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852818.42 15:59:00 [---] pick_jobs_to_run() start 15:59:00 [---] pick_jobs_to_run() end 15:59:00 [---] [rr_sim] end 15:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 15:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 15:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 15:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 15:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 15:59:00 [---] [cpu_sched_debug] preliminary job list: 15:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:59:00 [---] [cpu_sched_debug] final job list: 15:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 15:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 15:59:00 [---] [cpu_sched_debug] enforce_run_list: end 15:59:00 [---] choose_project(): 57540.000000 15:59:00 [WUProp@Home] doing RPC to get work 15:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 378059.09 cpu 2125292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125292.90 15:59:00 [---] Got 0 tasks 16:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:00:00 [---] [rr_sim] doing sim: CPU sched 16:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977265546.48G/439.78G) 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222174.77 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 222217.48 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222217.476875 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999957.29 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 199995.73 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422213.206062 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799961.56 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 179996.16 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602209.362330 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619965.41 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 161996.54 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764205.902972 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457968.87 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 145796.89 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 910002.789549 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312171.98 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 131217.20 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041219.987469 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180954.78 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 118095.48 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159315.465596 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062859.30 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 106285.93 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265601.395911 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956573.37 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 95657.34 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361258.733195 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860916.04 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 86091.60 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447350.336750 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774824.43 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 77482.44 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524832.779949 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697341.99 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 69734.20 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594566.978829 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627607.79 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 62760.78 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657327.757821 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564847.01 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 56484.70 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713812.458913 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508362.31 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 50836.23 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764648.689896 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457526.08 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 45752.61 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810401.297781 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411773.47 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 41177.35 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851578.644878 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370596.12 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 37059.61 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888638.257264 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333536.51 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 33353.65 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921991.908413 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300182.86 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 30018.29 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952010.194446 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270164.57 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 27016.46 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979026.651876 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243148.12 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 24314.81 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003341.463563 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218833.31 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 21883.33 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025224.794081 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196949.97 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 19695.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044919.791547 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177254.98 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 17725.50 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062645.289267 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159529.48 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 15952.95 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078598.237215 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143576.53 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 14357.65 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092955.890368 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129218.88 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 12921.89 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105877.778206 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116296.99 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 11629.70 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117507.477260 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104667.29 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 10466.73 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127974.206408 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94200.56 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 9420.06 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137394.262642 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84780.51 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 8478.05 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145872.313252 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76302.46 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 7630.25 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153502.558802 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68672.21 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 6867.22 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160369.779796 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61804.99 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 6180.50 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166550.278691 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55624.49 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 5562.45 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172112.727696 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50062.04 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 5006.20 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177118.931801 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45055.84 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 4505.58 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181624.515496 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40550.25 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 4055.03 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185679.540821 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36495.23 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3649.52 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189329.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32845.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192929.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29245.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196529.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25645.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200129.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22045.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203729.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18445.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207329.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14845.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210929.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11245.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214529.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7645.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218129.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4045.71 sec) 16:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221729.063613 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 445.71 sec) 16:00:00 [GPUGRID] [rr_sim] 2222174.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977265546.48G/439.78G) 16:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852815.68 16:00:00 [---] pick_jobs_to_run() start 16:00:00 [---] pick_jobs_to_run() end 16:00:00 [---] [rr_sim] end 16:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:00:00 [---] [cpu_sched_debug] preliminary job list: 16:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:00:00 [---] [cpu_sched_debug] final job list: 16:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:00:00 [---] [cpu_sched_debug] enforce_run_list: end 16:00:00 [---] choose_project(): 57600.000000 16:00:00 [WUProp@Home] doing RPC to get work 16:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377999.09 cpu 2125232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125232.90 16:00:00 [---] Got 0 tasks 16:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:01:00 [---] [rr_sim] doing sim: CPU sched 16:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977237956.13G/439.78G) 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222112.03 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 222211.20 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222211.203186 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999900.83 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 199990.08 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422201.286054 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799910.75 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 179991.07 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602192.360635 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619919.67 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 161991.97 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764184.327758 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457927.70 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 145792.77 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909977.098168 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312134.93 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 131213.49 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041190.591538 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180921.44 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 118092.14 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159282.735570 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062829.30 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 106282.93 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265565.665200 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956546.37 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 95654.64 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361220.301866 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860891.73 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 86089.17 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447309.474866 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774802.56 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 77480.26 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524789.730566 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697322.30 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 69732.23 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594521.960695 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627590.07 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 62759.01 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657280.967812 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564831.06 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 56483.11 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713764.074217 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508347.96 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 50834.80 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764598.869982 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457513.16 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 45751.32 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810350.186170 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411761.85 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 41176.18 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851526.370739 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370585.66 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 37058.57 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888584.936852 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333527.10 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 33352.71 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921937.646353 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300174.39 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 30017.44 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951955.084904 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270156.95 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 27015.69 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978970.779600 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243141.25 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 24314.13 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003284.904826 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218827.13 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 21882.71 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025167.617530 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196944.41 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 19694.44 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044862.058963 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177249.97 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 17725.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062587.056253 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159524.98 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 15952.50 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078539.553814 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143572.48 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 14357.25 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092896.801619 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129215.23 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 12921.52 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105818.324644 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116293.71 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 11629.37 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117447.695366 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104664.34 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 10466.43 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127914.129015 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94197.90 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 9419.79 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137333.919300 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84778.11 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 8477.81 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145811.730556 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76300.30 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 7630.03 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153441.760687 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68670.27 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 6867.03 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160308.787805 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61803.24 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 6180.32 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166489.112211 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55622.92 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 5562.29 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172051.404176 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50060.63 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 5006.06 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177057.466945 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45054.56 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 4505.46 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181562.923436 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40549.11 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 4054.91 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185617.834279 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36494.20 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3649.42 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189267.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32844.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192867.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29244.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196467.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25644.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200067.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22044.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203667.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18444.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207267.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14844.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210867.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11244.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214467.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7644.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218067.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4044.78 sec) 16:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221667.254038 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 444.78 sec) 16:01:00 [GPUGRID] [rr_sim] 2222112.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977237956.13G/439.78G) 16:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852812.94 16:01:00 [---] pick_jobs_to_run() start 16:01:00 [---] pick_jobs_to_run() end 16:01:00 [---] [rr_sim] end 16:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:01:00 [---] [cpu_sched_debug] preliminary job list: 16:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:01:00 [---] [cpu_sched_debug] final job list: 16:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:01:00 [---] [cpu_sched_debug] enforce_run_list: end 16:01:00 [---] choose_project(): 57660.000000 16:01:00 [WUProp@Home] doing RPC to get work 16:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377939.09 cpu 2125172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125172.90 16:01:00 [---] Got 0 tasks 16:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:02:00 [---] [rr_sim] doing sim: CPU sched 16:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977210365.77G/439.78G) 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2222049.29 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 222204.93 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222204.929498 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999844.37 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 199984.44 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422189.366046 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799859.93 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 179985.99 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602175.358940 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619873.94 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 161987.39 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764162.752544 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457886.54 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 145788.65 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909951.406788 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312097.89 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 131209.79 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041161.195607 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180888.10 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 118088.81 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159250.005544 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062799.29 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 106279.93 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265529.934488 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956519.36 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 95651.94 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361181.870537 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860867.42 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 86086.74 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447268.612982 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774780.68 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 77478.07 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524746.681182 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697302.61 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 69730.26 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594476.942562 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627572.35 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 62757.24 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657234.177804 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564815.12 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 56481.51 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713715.689521 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508333.61 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 50833.36 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764549.050067 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457500.24 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 45750.02 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810299.074559 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411750.22 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 41175.02 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851474.096601 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370575.20 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 37057.52 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888531.616439 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333517.68 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 33351.77 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921883.384293 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300165.91 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 30016.59 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951899.975362 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270149.32 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 27014.93 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978914.907324 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243134.39 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 24313.44 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003228.346090 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218820.95 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 21882.09 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025110.440979 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196938.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 19693.89 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044804.326379 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177244.97 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 17724.50 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062528.823239 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159520.47 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 15952.05 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078480.870413 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143568.42 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 14356.84 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092837.712870 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129211.58 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 12921.16 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105758.871081 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116290.42 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 11629.04 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117387.913471 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104661.38 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 10466.14 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127854.051622 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94195.24 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 9419.52 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137273.575958 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84775.72 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 8477.57 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145751.147860 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76298.15 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 7629.81 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153380.962573 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68668.33 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 6866.83 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160247.795813 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61801.50 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 6180.15 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166427.945730 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55621.35 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 5562.13 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171990.080655 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50059.21 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 5005.92 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176996.002088 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45053.29 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 4505.33 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181501.331377 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40547.96 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 4054.80 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185556.127738 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36493.17 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3649.32 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189205.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32843.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192805.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29243.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196405.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25643.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200005.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22043.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203605.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18443.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207205.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14843.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210805.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11243.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214405.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7643.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218005.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4043.85 sec) 16:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221605.444462 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 443.85 sec) 16:02:00 [GPUGRID] [rr_sim] 2222049.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977210365.77G/439.78G) 16:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852810.21 16:02:00 [---] pick_jobs_to_run() start 16:02:00 [---] pick_jobs_to_run() end 16:02:00 [---] [rr_sim] end 16:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:02:00 [---] [cpu_sched_debug] preliminary job list: 16:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:02:00 [---] [cpu_sched_debug] final job list: 16:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:02:00 [---] [cpu_sched_debug] enforce_run_list: end 16:02:00 [---] choose_project(): 57720.000000 16:02:00 [WUProp@Home] doing RPC to get work 16:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377879.09 cpu 2125112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125112.90 16:02:00 [---] Got 0 tasks 16:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:03:00 [---] [rr_sim] doing sim: CPU sched 16:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977182775.41G/439.78G) 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221986.56 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 222198.66 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222198.655810 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999787.90 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 199978.79 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422177.446039 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799809.11 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 179980.91 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602158.357245 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619828.20 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 161982.82 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764141.177330 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457845.38 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 145784.54 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909925.715407 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312060.84 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 131206.08 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041131.799676 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180854.76 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 118085.48 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159217.275519 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062769.28 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 106276.93 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265494.203777 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956492.35 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 95649.24 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361143.439209 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860843.12 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 86084.31 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447227.751098 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774758.81 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 77475.88 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524703.631798 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697282.93 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 69728.29 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594431.924428 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627554.63 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 62755.46 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657187.387795 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564799.17 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 56479.92 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713667.304826 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508319.25 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 50831.93 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764499.230153 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457487.33 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 45748.73 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810247.962947 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411738.60 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 41173.86 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851421.822463 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370564.74 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 37056.47 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888478.296026 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333508.26 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 33350.83 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921829.122234 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300157.44 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 30015.74 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951844.865820 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270141.69 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 27014.17 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978859.035048 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243127.52 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 24312.75 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003171.787353 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218814.77 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 21881.48 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025053.264428 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196933.29 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 19693.33 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044746.593795 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177239.96 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 17724.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062470.590225 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159515.97 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 15951.60 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078422.187013 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143564.37 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 14356.44 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092778.624121 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129207.93 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 12920.79 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105699.417519 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116287.14 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 11628.71 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117328.131577 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104658.43 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 10465.84 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127793.974229 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94192.58 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 9419.26 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137213.232616 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84773.33 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 8477.33 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145690.565164 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76295.99 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 7629.60 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153320.164458 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68666.39 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 6866.64 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160186.803822 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61799.75 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 6179.98 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166366.779250 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55619.78 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 5561.98 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171928.757135 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50057.80 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 5005.78 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176934.537231 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45052.02 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 4505.20 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181439.739318 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40546.82 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 4054.68 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185494.421196 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36492.14 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3649.21 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189143.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32842.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192743.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29242.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196343.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25642.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199943.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22042.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203543.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18442.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207143.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14842.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210743.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11242.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214343.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7642.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217943.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4042.92 sec) 16:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221543.634886 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 442.92 sec) 16:03:00 [GPUGRID] [rr_sim] 2221986.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977182775.41G/439.78G) 16:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852807.47 16:03:00 [---] pick_jobs_to_run() start 16:03:00 [---] pick_jobs_to_run() end 16:03:00 [---] [rr_sim] end 16:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:03:00 [---] [cpu_sched_debug] preliminary job list: 16:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:03:00 [---] [cpu_sched_debug] final job list: 16:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:03:00 [---] [cpu_sched_debug] enforce_run_list: end 16:03:00 [---] choose_project(): 57780.000000 16:03:00 [WUProp@Home] doing RPC to get work 16:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377819.09 cpu 2125052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2125052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2125052.90 16:03:00 [---] Got 0 tasks 16:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:04:00 [---] [rr_sim] doing sim: CPU sched 16:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977155185.06G/439.78G) 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221923.82 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 222192.38 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222192.382122 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999731.44 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 199973.14 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422165.526031 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799758.30 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 179975.83 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602141.355550 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619782.47 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 161978.25 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764119.602116 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457804.22 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 145780.42 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909900.024026 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1312023.80 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 131202.38 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041102.403746 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180821.42 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 118082.14 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159184.545493 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062739.28 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 106273.93 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265458.473065 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956465.35 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 95646.53 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361105.007880 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860818.81 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 86081.88 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447186.889214 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774736.93 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 77473.69 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524660.582414 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697263.24 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 69726.32 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594386.906294 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627536.91 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 62753.69 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657140.597787 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564783.22 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 56478.32 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713618.920130 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508304.90 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 50830.49 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764449.410238 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457474.41 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 45747.44 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810196.851336 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411726.97 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 41172.70 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851369.548324 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370554.27 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 37055.43 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888424.975614 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333498.85 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 33349.88 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921774.860174 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300148.96 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 30014.90 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951789.756278 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270134.06 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 27013.41 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978803.162772 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243120.66 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 24312.07 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003115.228617 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218808.59 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 21880.86 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024996.087877 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196927.73 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 19692.77 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044688.861211 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177234.96 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 17723.50 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062412.357211 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159511.46 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 15951.15 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078363.503612 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143560.32 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 14356.03 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092719.535372 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129204.29 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 12920.43 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105639.963957 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116283.86 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 11628.39 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117268.349683 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104655.47 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 10465.55 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127733.896836 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94189.92 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 9418.99 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137152.889274 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84770.93 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 8477.09 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145629.982468 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76293.84 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 7629.38 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153259.366343 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68664.45 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 6866.45 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160125.811831 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61798.01 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 6179.80 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166305.612769 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55618.21 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 5561.82 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171867.433614 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50056.39 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 5005.64 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176873.072374 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45050.75 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 4505.07 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181378.147259 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40545.67 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 4054.57 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185432.714654 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36491.11 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3649.11 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189081.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32842.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192681.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29242.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196281.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25642.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199881.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22042.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203481.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18442.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207081.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14842.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210681.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11242.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214281.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7642.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217881.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4042.00 sec) 16:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221481.825311 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 442.00 sec) 16:04:00 [GPUGRID] [rr_sim] 2221923.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977155185.06G/439.78G) 16:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852804.73 16:04:00 [---] pick_jobs_to_run() start 16:04:00 [---] pick_jobs_to_run() end 16:04:00 [---] [rr_sim] end 16:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:04:00 [---] [cpu_sched_debug] preliminary job list: 16:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:04:00 [---] [cpu_sched_debug] final job list: 16:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:04:00 [---] [cpu_sched_debug] enforce_run_list: end 16:04:00 [---] choose_project(): 57840.000000 16:04:00 [WUProp@Home] doing RPC to get work 16:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377759.09 cpu 2124992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124992.90 16:04:00 [---] Got 0 tasks 16:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:05:00 [---] [rr_sim] doing sim: CPU sched 16:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977127594.70G/439.78G) 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221861.08 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 222186.11 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222186.108433 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999674.98 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 199967.50 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422153.606024 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799707.48 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 179970.75 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602124.353855 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619736.73 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 161973.67 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764098.026903 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457763.06 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 145776.31 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909874.332646 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311986.75 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 131198.68 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041073.007815 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180788.08 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 118078.81 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159151.815467 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062709.27 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 106270.93 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265422.742354 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956438.34 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 95643.83 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361066.576552 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860794.51 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 86079.45 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447146.027330 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774715.06 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 77471.51 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524617.533030 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697243.55 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 69724.36 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594341.888161 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627519.20 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 62751.92 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657093.807778 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564767.28 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 56476.73 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713570.535434 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508290.55 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 50829.05 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764399.590324 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457461.49 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 45746.15 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810145.739725 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411715.34 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 41171.53 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851317.274186 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370543.81 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 37054.38 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888371.655201 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333489.43 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 33348.94 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921720.598114 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300140.49 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 30014.05 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951734.646736 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270126.44 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 27012.64 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978747.290496 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243113.79 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 24311.38 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003058.669880 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218802.41 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 21880.24 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024938.911325 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196922.17 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 19692.22 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044631.128626 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177229.96 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 17723.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062354.124197 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159506.96 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 15950.70 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078304.820211 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143556.26 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 14355.63 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092660.446623 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129200.64 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 12920.06 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105580.510394 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116280.57 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 11628.06 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117208.567788 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104652.52 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 10465.25 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127673.819443 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94187.26 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 9418.73 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137092.545932 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84768.54 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 8476.85 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145569.399772 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76291.68 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 7629.17 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153198.568229 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68662.52 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 6866.25 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160064.819839 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61796.26 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 6179.63 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166244.446289 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55616.64 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 5561.66 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171806.110093 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50054.97 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 5005.50 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176811.607518 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45049.48 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 4504.95 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181316.555199 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40544.53 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 4054.45 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185371.008113 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36490.08 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3649.01 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189020.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32841.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192620.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29241.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196220.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25641.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199820.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22041.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203420.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18441.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207020.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14841.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210620.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11241.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214220.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7641.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217820.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4041.07 sec) 16:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221420.015735 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 441.07 sec) 16:05:00 [GPUGRID] [rr_sim] 2221861.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977127594.70G/439.78G) 16:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852802.00 16:05:00 [---] pick_jobs_to_run() start 16:05:00 [---] pick_jobs_to_run() end 16:05:00 [---] [rr_sim] end 16:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:05:00 [---] [cpu_sched_debug] preliminary job list: 16:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:05:00 [---] [cpu_sched_debug] final job list: 16:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:05:00 [---] [cpu_sched_debug] enforce_run_list: end 16:05:00 [---] choose_project(): 57900.000000 16:05:00 [WUProp@Home] doing RPC to get work 16:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377699.09 cpu 2124932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124932.90 16:05:00 [---] Got 0 tasks 16:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:06:00 [---] [rr_sim] doing sim: CPU sched 16:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977100004.34G/439.78G) 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221798.35 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 222179.83 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222179.834745 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999618.51 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 199961.85 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422141.686016 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799656.66 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 179965.67 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602107.352160 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619691.00 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 161969.10 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764076.451689 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457721.90 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 145772.19 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909848.641265 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311949.71 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 131194.97 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041043.611884 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180754.74 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 118075.47 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159119.085441 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062679.26 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 106267.93 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265387.011642 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956411.34 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 95641.13 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1361028.145223 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860770.20 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 86077.02 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447105.165446 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774693.18 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 77469.32 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524574.483647 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697223.86 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 69722.39 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594296.870027 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627501.48 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 62750.15 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657047.017770 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564751.33 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 56475.13 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713522.150738 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508276.20 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 50827.62 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764349.770409 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457448.58 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 45744.86 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810094.628114 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411703.72 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 41170.37 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851265.000048 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370533.35 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 37053.33 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888318.334788 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333480.01 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 33348.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921666.336055 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300132.01 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 30013.20 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951679.537194 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270118.81 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 27011.88 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978691.418220 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243106.93 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 24310.69 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003002.111143 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218796.24 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 21879.62 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024881.734774 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196916.61 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 19691.66 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044573.396042 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177224.95 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 17722.50 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062295.891183 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159502.46 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 15950.25 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078246.136810 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143552.21 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 14355.22 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092601.357874 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129196.99 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 12919.70 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105521.056832 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116277.29 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 11627.73 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117148.785894 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104649.56 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 10464.96 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127613.742050 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94184.61 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 9418.46 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137032.202590 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84766.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 8476.61 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145508.817076 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76289.53 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 7628.95 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153137.770114 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68660.58 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 6866.06 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160003.827848 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61794.52 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 6179.45 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166183.279808 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55615.07 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 5561.51 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171744.786573 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50053.56 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 5005.36 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176750.142661 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45048.20 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 4504.82 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181254.963140 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40543.38 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 4054.34 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185309.301571 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36489.05 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3648.90 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188958.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32840.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192558.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29240.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196158.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25640.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199758.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22040.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203358.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18440.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206958.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14840.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210558.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11240.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214158.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7640.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217758.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4040.14 sec) 16:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221358.206159 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 440.14 sec) 16:06:00 [GPUGRID] [rr_sim] 2221798.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977100004.34G/439.78G) 16:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852799.26 16:06:00 [---] pick_jobs_to_run() start 16:06:00 [---] pick_jobs_to_run() end 16:06:00 [---] [rr_sim] end 16:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:06:00 [---] [cpu_sched_debug] preliminary job list: 16:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:06:00 [---] [cpu_sched_debug] final job list: 16:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:06:00 [---] [cpu_sched_debug] enforce_run_list: end 16:06:00 [---] choose_project(): 57960.000000 16:06:00 [WUProp@Home] doing RPC to get work 16:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377639.09 cpu 2124872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124872.90 16:06:00 [---] Got 0 tasks 16:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:07:00 [---] [rr_sim] doing sim: CPU sched 16:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977072413.99G/439.78G) 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221735.61 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 222173.56 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222173.561057 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999562.05 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 199956.20 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422129.766008 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799605.84 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 179960.58 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602090.350465 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619645.26 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 161964.53 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764054.876475 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457680.73 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 145768.07 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909822.949885 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311912.66 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 131191.27 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1041014.215953 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180721.39 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 118072.14 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159086.355415 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062649.26 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 106264.93 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265351.280930 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956384.33 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 95638.43 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360989.713894 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860745.90 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 86074.59 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447064.303562 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774671.31 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 77467.13 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524531.434263 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697204.18 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 69720.42 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594251.851894 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627483.76 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 62748.38 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1657000.227761 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564735.38 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 56473.54 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713473.766042 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508261.84 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 50826.18 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764299.950495 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457435.66 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 45743.57 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1810043.516503 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411692.09 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 41169.21 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851212.725909 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370522.88 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 37052.29 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888265.014375 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333470.60 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 33347.06 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921612.073995 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300123.54 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 30012.35 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951624.427652 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270111.18 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 27011.12 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978635.545944 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243100.06 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 24310.01 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002945.552407 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218790.06 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 21879.01 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024824.558223 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196911.05 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 19691.11 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044515.663458 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177219.95 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 17721.99 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062237.658169 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159497.95 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 15949.80 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078187.453409 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143548.16 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 14354.82 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092542.269125 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129193.34 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 12919.33 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105461.603270 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116274.01 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 11627.40 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117089.004000 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104646.61 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 10464.66 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127553.664657 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94181.95 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 9418.19 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136971.859248 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84763.75 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 8476.38 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145448.234380 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76287.38 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 7628.74 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153076.971999 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68658.64 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 6865.86 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159942.835857 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61792.77 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 6179.28 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166122.113328 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55613.50 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 5561.35 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171683.463052 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50052.15 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 5005.21 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176688.677804 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45046.93 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 4504.69 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181193.371081 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40542.24 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 4054.22 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185247.595030 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36488.02 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3648.80 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188896.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32839.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192496.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29239.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196096.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25639.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199696.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22039.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203296.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18439.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206896.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14839.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210496.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11239.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214096.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7639.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217696.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4039.21 sec) 16:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221296.396584 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 439.21 sec) 16:07:00 [GPUGRID] [rr_sim] 2221735.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977072413.99G/439.78G) 16:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852796.52 16:07:00 [---] pick_jobs_to_run() start 16:07:00 [---] pick_jobs_to_run() end 16:07:00 [---] [rr_sim] end 16:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:07:00 [---] [cpu_sched_debug] preliminary job list: 16:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:07:00 [---] [cpu_sched_debug] final job list: 16:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:07:00 [---] [cpu_sched_debug] enforce_run_list: end 16:07:00 [---] choose_project(): 58020.000000 16:07:00 [WUProp@Home] doing RPC to get work 16:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377579.09 cpu 2124812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124812.90 16:07:00 [---] Got 0 tasks 16:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:08:00 [---] [rr_sim] doing sim: CPU sched 16:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977044823.63G/439.78G) 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221672.87 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 222167.29 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222167.287369 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999505.59 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 199950.56 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422117.846001 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799555.03 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 179955.50 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602073.348769 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619599.52 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 161959.95 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764033.301261 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457639.57 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 145763.96 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909797.258504 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311875.62 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 131187.56 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040984.820022 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180688.05 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 118068.81 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159053.625389 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062619.25 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 106261.92 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265315.550219 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956357.32 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 95635.73 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360951.282566 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860721.59 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 86072.16 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1447023.441678 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774649.43 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 77464.94 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524488.384879 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697184.49 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 69718.45 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594206.833760 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627466.04 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 62746.60 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656953.437753 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564719.44 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 56471.94 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713425.381346 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508247.49 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 50824.75 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764250.130581 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457422.74 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 45742.27 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809992.404891 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411680.47 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 41168.05 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851160.451771 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370512.42 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 37051.24 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888211.693963 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333461.18 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 33346.12 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921557.811935 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300115.06 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 30011.51 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951569.318111 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270103.56 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 27010.36 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978579.673668 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243093.20 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 24309.32 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002888.993670 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218783.88 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 21878.39 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024767.381672 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196905.49 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 19690.55 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044457.930874 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177214.94 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 17721.49 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062179.425155 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159493.45 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 15949.34 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078128.770008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143544.10 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 14354.41 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092483.180376 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129189.69 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 12918.97 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105402.149708 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116270.72 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 11627.07 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117029.222106 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104643.65 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 10464.37 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127493.587264 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94179.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 9417.93 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136911.515906 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84761.36 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 8476.14 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145387.651684 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76285.22 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 7628.52 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153016.173885 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68656.70 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 6865.67 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159881.843865 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61791.03 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 6179.10 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166060.946847 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55611.93 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 5561.19 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171622.139532 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50050.73 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 5005.07 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176627.212947 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45045.66 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 4504.57 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181131.779021 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40541.09 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 4054.11 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185185.888488 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36486.99 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3648.70 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188834.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32838.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192434.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29238.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196034.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25638.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199634.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22038.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203234.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18438.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206834.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14838.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210434.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11238.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214034.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7638.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217634.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4038.29 sec) 16:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221234.587008 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 438.29 sec) 16:08:00 [GPUGRID] [rr_sim] 2221672.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977044823.63G/439.78G) 16:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852793.79 16:08:00 [---] pick_jobs_to_run() start 16:08:00 [---] pick_jobs_to_run() end 16:08:00 [---] [rr_sim] end 16:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:08:00 [---] [cpu_sched_debug] preliminary job list: 16:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:08:00 [---] [cpu_sched_debug] final job list: 16:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:08:00 [---] [cpu_sched_debug] enforce_run_list: end 16:08:00 [---] choose_project(): 58080.000000 16:08:00 [WUProp@Home] doing RPC to get work 16:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377519.09 cpu 2124752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124752.90 16:08:00 [---] Got 0 tasks 16:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:09:00 [---] [rr_sim] doing sim: CPU sched 16:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (977017233.28G/439.78G) 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221610.14 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 222161.01 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222161.013681 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999449.12 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 199944.91 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422105.925993 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799504.21 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 179950.42 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602056.347074 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619553.79 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 161955.38 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 764011.726048 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457598.41 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 145759.84 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909771.567123 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311838.57 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 131183.86 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040955.424092 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180654.71 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 118065.47 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1159020.895363 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062589.24 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 106258.92 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265279.819507 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956330.32 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 95633.03 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360912.851237 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860697.29 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 86069.73 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446982.579794 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774627.56 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 77462.76 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524445.335495 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697164.80 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 69716.48 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594161.815626 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627448.32 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 62744.83 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656906.647744 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564703.49 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 56470.35 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713376.996650 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508233.14 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 50823.31 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764200.310666 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457409.83 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 45740.98 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809941.293280 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411668.84 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 41166.88 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851108.177633 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370501.96 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 37050.20 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888158.373550 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333451.76 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 33345.18 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921503.549876 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300106.59 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 30010.66 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951514.208569 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270095.93 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 27009.59 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978523.801392 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243086.34 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 24308.63 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002832.434934 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218777.70 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 21877.77 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024710.205121 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196899.93 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 19689.99 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044400.198289 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177209.94 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 17720.99 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062121.192141 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159488.94 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 15948.89 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078070.086608 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143540.05 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 14354.01 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092424.091627 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129186.05 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 12918.60 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105342.696145 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116267.44 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 11626.74 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116969.440211 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104640.70 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 10464.07 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127433.509871 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94176.63 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 9417.66 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136851.172564 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84758.96 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 8475.90 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145327.068988 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76283.07 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 7628.31 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152955.375770 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68654.76 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 6865.48 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159820.851874 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61789.28 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 6178.93 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165999.780367 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55610.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 5561.04 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171560.816011 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50049.32 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 5004.93 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176565.748090 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45044.39 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 4504.44 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181070.186962 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40539.95 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 4053.99 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185124.181946 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36485.95 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3648.60 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188772.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32837.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192372.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29237.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195972.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25637.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199572.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22037.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203172.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18437.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206772.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14837.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210372.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11237.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213972.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7637.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217572.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4037.36 sec) 16:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221172.777432 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 437.36 sec) 16:09:00 [GPUGRID] [rr_sim] 2221610.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (977017233.28G/439.78G) 16:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852791.05 16:09:00 [---] pick_jobs_to_run() start 16:09:00 [---] pick_jobs_to_run() end 16:09:00 [---] [rr_sim] end 16:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:09:00 [---] [cpu_sched_debug] preliminary job list: 16:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:09:00 [---] [cpu_sched_debug] final job list: 16:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:09:00 [---] [cpu_sched_debug] enforce_run_list: end 16:09:00 [---] choose_project(): 58140.000000 16:09:00 [WUProp@Home] doing RPC to get work 16:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377459.09 cpu 2124692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124692.90 16:09:00 [---] Got 0 tasks 16:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:10:00 [---] [rr_sim] doing sim: CPU sched 16:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976989642.92G/439.78G) 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221547.40 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 222154.74 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222154.739992 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999392.66 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 199939.27 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422094.005986 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799453.39 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 179945.34 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602039.345379 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619508.05 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 161950.81 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763990.150834 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457557.25 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 145755.72 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909745.875743 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311801.52 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 131180.15 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040926.028161 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180621.37 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 118062.14 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158988.165337 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062559.23 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 106255.92 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265244.088796 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956303.31 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 95630.33 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360874.419909 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860672.98 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 86067.30 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446941.717910 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774605.68 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 77460.57 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524402.286112 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697145.11 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 69714.51 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594116.797493 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627430.60 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 62743.06 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656859.857736 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564687.54 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 56468.75 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713328.611955 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508218.79 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 50821.88 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764150.490752 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457396.91 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 45739.69 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809890.181669 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411657.22 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 41165.72 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851055.903494 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370491.50 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 37049.15 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888105.053137 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333442.35 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 33344.23 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921449.287816 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300098.11 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 30009.81 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951459.099027 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270088.30 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 27008.83 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978467.929116 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243079.47 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 24307.95 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002775.876197 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218771.52 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 21877.15 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024653.028570 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196894.37 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 19689.44 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044342.465705 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177204.93 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 17720.49 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062062.959127 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159484.44 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 15948.44 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078011.403207 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143536.00 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 14353.60 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092365.002878 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129182.40 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 12918.24 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105283.242583 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116264.16 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 11626.42 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116909.658317 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104637.74 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 10463.77 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127373.432478 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94173.97 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 9417.40 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136790.829222 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84756.57 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 8475.66 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145266.486292 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76280.91 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 7628.09 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152894.577656 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68652.82 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 6865.28 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159759.859882 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61787.54 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 6178.75 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165938.613887 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55608.79 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 5560.88 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171499.492490 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50047.91 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 5004.79 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176504.283234 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45043.12 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 4504.31 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181008.594903 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40538.81 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 4053.88 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185062.475405 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36484.92 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3648.49 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188710.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32836.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192310.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29236.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195910.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25636.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199510.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22036.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203110.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18436.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206710.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14836.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210310.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11236.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213910.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7636.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217510.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4036.43 sec) 16:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221110.967857 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 436.43 sec) 16:10:00 [GPUGRID] [rr_sim] 2221547.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976989642.92G/439.78G) 16:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852788.31 16:10:00 [---] pick_jobs_to_run() start 16:10:00 [---] pick_jobs_to_run() end 16:10:00 [---] [rr_sim] end 16:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:10:00 [---] [cpu_sched_debug] preliminary job list: 16:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:10:00 [---] [cpu_sched_debug] final job list: 16:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:10:00 [---] [cpu_sched_debug] enforce_run_list: end 16:10:00 [---] choose_project(): 58200.000000 16:10:00 [WUProp@Home] doing RPC to get work 16:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377399.09 cpu 2124632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124632.90 16:10:00 [---] Got 0 tasks 16:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:11:00 [---] [rr_sim] doing sim: CPU sched 16:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976962052.56G/439.78G) 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221484.66 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 222148.47 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222148.466304 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999336.20 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 199933.62 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422082.085978 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799402.58 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 179940.26 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602022.343684 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619462.32 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 161946.23 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763968.575620 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457516.09 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 145751.61 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909720.184362 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311764.48 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 131176.45 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040896.632230 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180588.03 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 118058.80 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158955.435311 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062529.23 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 106252.92 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265208.358084 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956276.30 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 95627.63 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360835.988580 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860648.67 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 86064.87 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446900.856026 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774583.81 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 77458.38 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524359.236728 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697125.43 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 69712.54 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594071.779359 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627412.88 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 62741.29 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656813.067727 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564671.60 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 56467.16 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713280.227259 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508204.44 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 50820.44 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764100.670837 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457383.99 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 45738.40 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809839.070058 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411645.59 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 41164.56 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1851003.629356 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370481.03 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 37048.10 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1888051.732725 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333432.93 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 33343.29 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921395.025756 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300089.64 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 30008.96 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951403.989485 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270080.67 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 27008.07 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978412.056840 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243072.61 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 24307.26 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002719.317461 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218765.35 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 21876.53 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024595.852019 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196888.81 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 19688.88 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044284.733121 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177199.93 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 17719.99 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062004.726113 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159479.94 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 15947.99 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077952.719806 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143531.94 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 14353.19 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092305.914129 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129178.75 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 12917.87 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105223.789021 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116260.87 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 11626.09 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116849.876423 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104634.79 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 10463.48 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127313.355085 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94171.31 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 9417.13 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136730.485880 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84754.18 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 8475.42 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145205.903597 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76278.76 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 7627.88 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152833.779541 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68650.88 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 6865.09 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159698.867891 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61785.80 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 6178.58 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165877.447406 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55607.22 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 5560.72 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171438.168970 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50046.49 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 5004.65 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176442.818377 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45041.84 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 4504.18 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180947.002843 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40537.66 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 4053.77 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185000.768863 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36483.89 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3648.39 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188649.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32835.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192249.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29235.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195849.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25635.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199449.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22035.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203049.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18435.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206649.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14835.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210249.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11235.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213849.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7635.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217449.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4035.50 sec) 16:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2221049.158281 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 435.50 sec) 16:11:00 [GPUGRID] [rr_sim] 2221484.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976962052.56G/439.78G) 16:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852785.58 16:11:00 [---] pick_jobs_to_run() start 16:11:00 [---] pick_jobs_to_run() end 16:11:00 [---] [rr_sim] end 16:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:11:00 [---] [cpu_sched_debug] preliminary job list: 16:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:11:00 [---] [cpu_sched_debug] final job list: 16:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:11:00 [---] [cpu_sched_debug] enforce_run_list: end 16:11:00 [---] choose_project(): 58260.000000 16:11:00 [WUProp@Home] doing RPC to get work 16:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377339.09 cpu 2124572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124572.90 16:11:00 [---] Got 0 tasks 16:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:12:00 [---] [rr_sim] doing sim: CPU sched 16:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976934462.21G/439.78G) 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221421.93 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 222142.19 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222142.192616 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999279.73 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 199927.97 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422070.165970 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799351.76 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 179935.18 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 602005.341989 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619416.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 161941.66 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763947.000406 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457474.93 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 145747.49 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909694.492982 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311727.43 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 131172.74 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040867.236299 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180554.69 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 118055.47 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158922.705285 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062499.22 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 106249.92 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265172.627373 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956249.30 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 95624.93 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360797.557251 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860624.37 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 86062.44 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446859.994142 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774561.93 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 77456.19 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524316.187344 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697105.74 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 69710.57 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1594026.761226 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627395.16 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 62739.52 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656766.277719 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564655.65 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 56465.56 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713231.842563 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508190.08 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 50819.01 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764050.850923 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457371.08 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 45737.11 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809787.958446 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411633.97 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 41163.40 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850951.355218 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370470.57 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 37047.06 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887998.412312 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333423.51 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 33342.35 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921340.763697 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300081.16 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 30008.12 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951348.879943 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270073.05 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 27007.30 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978356.184565 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243065.74 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 24306.57 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002662.758724 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218759.17 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 21875.92 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024538.675468 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196883.25 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 19688.33 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044227.000537 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177194.93 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 17719.49 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061946.493099 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159475.43 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 15947.54 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077894.036405 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143527.89 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 14352.79 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092246.825381 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129175.10 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 12917.51 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105164.335458 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116257.59 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 11625.76 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116790.094529 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104631.83 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 10463.18 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127253.277692 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94168.65 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 9416.86 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136670.142538 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84751.78 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 8475.18 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145145.320901 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76276.61 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 7627.66 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152772.981426 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68648.94 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 6864.89 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159637.875900 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61784.05 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 6178.41 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165816.280926 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55605.65 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 5560.56 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171376.845449 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50045.08 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 5004.51 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176381.353520 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45040.57 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 4504.06 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180885.410784 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40536.52 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 4053.65 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184939.062322 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36482.86 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3648.29 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188587.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32834.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192187.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29234.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195787.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25634.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199387.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22034.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202987.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18434.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206587.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14834.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210187.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11234.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213787.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7634.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217387.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4034.58 sec) 16:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220987.348705 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 434.58 sec) 16:12:00 [GPUGRID] [rr_sim] 2221421.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976934462.21G/439.78G) 16:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852782.84 16:12:00 [---] pick_jobs_to_run() start 16:12:00 [---] pick_jobs_to_run() end 16:12:00 [---] [rr_sim] end 16:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:12:00 [---] [cpu_sched_debug] preliminary job list: 16:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:12:00 [---] [cpu_sched_debug] final job list: 16:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:12:00 [---] [cpu_sched_debug] enforce_run_list: end 16:12:00 [---] choose_project(): 58320.000000 16:12:00 [WUProp@Home] doing RPC to get work 16:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377279.09 cpu 2124512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124512.90 16:12:00 [---] Got 0 tasks 16:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:13:00 [---] [rr_sim] doing sim: CPU sched 16:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976906871.85G/439.78G) 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221359.19 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 222135.92 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222135.918928 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999223.27 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 199922.33 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422058.245963 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799300.94 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 179930.09 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601988.340294 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619370.85 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 161937.08 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763925.425192 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457433.76 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 145743.38 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909668.801601 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311690.39 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 131169.04 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040837.840369 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180521.35 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 118052.13 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158889.975259 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062469.21 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 106246.92 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265136.896661 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956222.29 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 95622.23 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360759.125923 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860600.06 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 86060.01 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446819.132258 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774540.06 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 77454.01 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524273.137960 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697086.05 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 69708.61 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593981.743092 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627377.45 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 62737.74 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656719.487710 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564639.70 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 56463.97 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713183.457867 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508175.73 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 50817.57 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1764001.031008 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457358.16 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 45735.82 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809736.846835 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411622.34 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 41162.23 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850899.081079 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370460.11 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 37046.01 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887945.091899 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333414.10 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 33341.41 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921286.501637 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300072.69 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 30007.27 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951293.770401 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270065.42 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 27006.54 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978300.312289 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243058.88 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 24305.89 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002606.199987 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218752.99 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 21875.30 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024481.498916 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196877.69 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 19687.77 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044169.267953 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177189.92 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 17718.99 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061888.260085 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159470.93 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 15947.09 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077835.353004 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143523.84 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 14352.38 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092187.736632 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129171.45 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 12917.15 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105104.881896 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116254.31 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 11625.43 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116730.312634 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104628.88 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 10462.89 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127193.200299 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94165.99 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 9416.60 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136609.799196 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84749.39 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 8474.94 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145084.738205 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76274.45 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 7627.45 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152712.183312 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68647.01 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 6864.70 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159576.883908 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61782.31 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 6178.23 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165755.114445 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55604.07 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 5560.41 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171315.521928 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50043.67 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 5004.37 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176319.888663 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45039.30 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 4503.93 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180823.818725 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40535.37 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 4053.54 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184877.355780 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36481.83 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3648.18 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188525.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32833.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192125.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29233.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195725.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25633.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199325.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22033.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202925.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18433.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206525.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14833.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210125.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11233.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213725.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7633.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217325.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4033.65 sec) 16:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220925.539130 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 433.65 sec) 16:13:00 [GPUGRID] [rr_sim] 2221359.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976906871.85G/439.78G) 16:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852780.10 16:13:00 [---] pick_jobs_to_run() start 16:13:00 [---] pick_jobs_to_run() end 16:13:00 [---] [rr_sim] end 16:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:13:00 [---] [cpu_sched_debug] preliminary job list: 16:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:13:00 [---] [cpu_sched_debug] final job list: 16:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:13:00 [---] [cpu_sched_debug] enforce_run_list: end 16:13:00 [---] choose_project(): 58380.000000 16:13:00 [WUProp@Home] doing RPC to get work 16:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377219.09 cpu 2124452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124452.90 16:13:00 [---] Got 0 tasks 16:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:14:00 [---] [rr_sim] doing sim: CPU sched 16:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976879281.50G/439.78G) 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221296.45 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 222129.65 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222129.645240 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999166.81 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 199916.68 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422046.325955 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799250.13 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 179925.01 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601971.338599 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619325.11 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 161932.51 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763903.849979 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457392.60 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 145739.26 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909643.110220 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311653.34 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 131165.33 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040808.444438 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180488.01 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 118048.80 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158857.245234 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062439.21 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 106243.92 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265101.165950 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956195.29 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 95619.53 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360720.694594 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860575.76 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 86057.58 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446778.270374 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774518.18 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 77451.82 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524230.088576 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697066.36 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 69706.64 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593936.724958 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627359.73 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 62735.97 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656672.697702 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564623.75 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 56462.38 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713135.073171 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508161.38 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 50816.14 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763951.211094 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457345.24 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 45734.52 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809685.735224 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411610.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 41161.07 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850846.806941 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370449.65 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 37044.96 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887891.771486 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333404.68 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 33340.47 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921232.239577 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300064.21 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 30006.42 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951238.660859 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270057.79 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 27005.78 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978244.440013 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243052.01 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 24305.20 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002549.641251 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218746.81 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 21874.68 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024424.322365 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196872.13 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 19687.21 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044111.535368 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177184.92 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 17718.49 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061830.027071 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159466.43 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 15946.64 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077776.669603 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143519.78 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 14351.98 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092128.647883 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129167.80 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 12916.78 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105045.428334 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116251.02 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 11625.10 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116670.530740 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104625.92 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 10462.59 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127133.122905 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94163.33 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 9416.33 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136549.455854 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84747.00 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 8474.70 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145024.155509 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76272.30 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 7627.23 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152651.385197 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68645.07 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 6864.51 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159515.891917 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61780.56 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 6178.06 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165693.947965 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55602.50 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 5560.25 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171254.198408 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50042.25 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 5004.23 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176258.423807 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45038.03 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 4503.80 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180762.226665 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40534.23 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 4053.42 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184815.649238 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36480.80 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3648.08 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188463.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32832.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192063.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29232.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195663.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25632.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199263.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22032.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202863.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18432.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206463.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14832.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210063.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11232.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213663.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7632.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217263.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4032.72 sec) 16:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220863.729554 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 432.72 sec) 16:14:00 [GPUGRID] [rr_sim] 2221296.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976879281.50G/439.78G) 16:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852777.37 16:14:00 [---] pick_jobs_to_run() start 16:14:00 [---] pick_jobs_to_run() end 16:14:00 [---] [rr_sim] end 16:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:14:00 [---] [cpu_sched_debug] preliminary job list: 16:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:14:00 [---] [cpu_sched_debug] final job list: 16:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:14:00 [---] [cpu_sched_debug] enforce_run_list: end 16:14:00 [---] choose_project(): 58440.000000 16:14:00 [WUProp@Home] doing RPC to get work 16:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377159.09 cpu 2124392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124392.90 16:14:00 [---] Got 0 tasks 16:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:15:00 [---] [rr_sim] doing sim: CPU sched 16:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976851691.14G/439.78G) 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221233.72 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 222123.37 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222123.371551 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999110.34 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 199911.03 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422034.405947 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799199.31 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 179919.93 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601954.336904 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619279.38 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 161927.94 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763882.274765 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457351.44 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 145735.14 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909617.418840 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311616.30 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 131161.63 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040779.048507 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180454.67 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 118045.47 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158824.515208 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062409.20 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 106240.92 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265065.435238 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956168.28 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 95616.83 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360682.263266 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860551.45 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 86055.15 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446737.408490 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774496.31 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 77449.63 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524187.039193 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697046.68 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 69704.67 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593891.706825 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627342.01 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 62734.20 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656625.907694 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564607.81 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 56460.78 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713086.688475 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508147.03 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 50814.70 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763901.391179 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457332.32 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 45733.23 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809634.623613 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411599.09 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 41159.91 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850794.532803 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370439.18 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 37043.92 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887838.451074 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333395.26 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 33339.53 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921177.977518 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300055.74 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 30005.57 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951183.551317 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270050.16 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 27005.02 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978188.567737 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243045.15 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 24304.51 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002493.082514 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218740.63 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 21874.06 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024367.145814 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196866.57 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 19686.66 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044053.802784 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177179.91 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 17717.99 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061771.794057 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159461.92 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 15946.19 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077717.986203 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143515.73 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 14351.57 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092069.559134 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129164.16 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 12916.42 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104985.974772 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116247.74 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 11624.77 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116610.748846 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104622.97 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 10462.30 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127073.045512 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94160.67 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 9416.07 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136489.112512 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84744.60 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 8474.46 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144963.572813 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76270.14 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 7627.01 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152590.587083 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68643.13 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 6864.31 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159454.899926 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61778.82 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 6177.88 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165632.781484 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55600.93 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 5560.09 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171192.874887 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50040.84 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 5004.08 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176196.958950 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45036.76 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 4503.68 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180700.634606 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40533.08 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 4053.31 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184753.942697 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36479.77 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3647.98 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188401.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32831.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192001.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29231.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195601.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25631.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199201.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22031.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202801.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18431.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206401.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14831.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210001.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11231.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213601.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7631.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217201.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4031.80 sec) 16:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220801.919978 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 431.80 sec) 16:15:00 [GPUGRID] [rr_sim] 2221233.72: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976851691.14G/439.78G) 16:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852774.63 16:15:00 [---] pick_jobs_to_run() start 16:15:00 [---] pick_jobs_to_run() end 16:15:00 [---] [rr_sim] end 16:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:15:00 [---] [cpu_sched_debug] preliminary job list: 16:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:15:00 [---] [cpu_sched_debug] final job list: 16:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:15:00 [---] [cpu_sched_debug] enforce_run_list: end 16:15:00 [---] choose_project(): 58500.000000 16:15:00 [WUProp@Home] doing RPC to get work 16:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377099.09 cpu 2124332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124332.90 16:15:00 [---] Got 0 tasks 16:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:16:00 [---] [rr_sim] doing sim: CPU sched 16:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976824100.78G/439.78G) 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221170.98 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 222117.10 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222117.097863 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1999053.88 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 199905.39 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422022.485940 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799148.49 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 179914.85 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601937.335209 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619233.64 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 161923.36 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763860.699551 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457310.28 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 145731.03 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909591.727459 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311579.25 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 131157.93 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040749.652576 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180421.33 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 118042.13 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158791.785182 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062379.19 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 106237.92 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1265029.704527 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956141.27 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 95614.13 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360643.831937 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860527.15 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 86052.71 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446696.546606 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774474.43 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 77447.44 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524143.989809 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697026.99 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 69702.70 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593846.688691 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627324.29 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 62732.43 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656579.117685 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564591.86 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 56459.19 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1713038.303780 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508132.67 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 50813.27 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763851.571265 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457319.41 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 45731.94 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809583.512001 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411587.47 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 41158.75 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850742.258664 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370428.72 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 37042.87 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887785.130661 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333385.85 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 33338.58 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921123.715458 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300047.26 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 30004.73 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951128.441775 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270042.54 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 27004.25 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978132.695461 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243038.28 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 24303.83 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002436.523778 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218734.45 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 21873.45 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024309.969263 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196861.01 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 19686.10 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043996.070200 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177174.91 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 17717.49 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061713.561043 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159457.42 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 15945.74 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077659.302802 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143511.68 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 14351.17 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092010.470385 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129160.51 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 12916.05 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104926.521209 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116244.46 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 11624.45 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116550.966951 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104620.01 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 10462.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127012.968119 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94158.01 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 9415.80 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136428.769171 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84742.21 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 8474.22 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144902.990117 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76267.99 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 7626.80 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152529.788968 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68641.19 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 6864.12 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159393.907934 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61777.07 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 6177.71 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165571.615004 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55599.36 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 5559.94 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171131.551367 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50039.43 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 5003.94 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176135.494093 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45035.48 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 4503.55 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180639.042547 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40531.94 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 4053.19 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184692.236155 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36478.74 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3647.87 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188340.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32830.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191940.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29230.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195540.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25630.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199140.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22030.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202740.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18430.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206340.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14830.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209940.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11230.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213540.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7630.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217140.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4030.87 sec) 16:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220740.110403 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 430.87 sec) 16:16:00 [GPUGRID] [rr_sim] 2221170.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976824100.78G/439.78G) 16:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852771.89 16:16:00 [---] pick_jobs_to_run() start 16:16:00 [---] pick_jobs_to_run() end 16:16:00 [---] [rr_sim] end 16:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:16:00 [---] [cpu_sched_debug] preliminary job list: 16:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:16:00 [---] [cpu_sched_debug] final job list: 16:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:16:00 [---] [cpu_sched_debug] enforce_run_list: end 16:16:00 [---] choose_project(): 58560.000000 16:16:00 [WUProp@Home] doing RPC to get work 16:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 377039.09 cpu 2124272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124272.90 16:16:00 [---] Got 0 tasks 16:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:17:00 [---] [rr_sim] doing sim: CPU sched 16:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976796510.43G/439.78G) 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221108.24 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 222110.82 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222110.824175 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998997.42 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 199899.74 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 422010.565932 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799097.68 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 179909.77 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601920.333514 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619187.91 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 161918.79 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763839.124337 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457269.12 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 145726.91 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909566.036078 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311542.21 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 131154.22 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040720.256646 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180387.99 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 118038.80 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158759.055156 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062349.19 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 106234.92 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264993.973815 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956114.27 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 95611.43 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360605.400608 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860502.84 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 86050.28 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446655.684722 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774452.56 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 77445.26 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524100.940425 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 697007.30 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 69700.73 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593801.670557 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627306.57 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 62730.66 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656532.327677 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564575.91 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 56457.59 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712989.919084 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508118.32 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 50811.83 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763801.751350 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457306.49 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 45730.65 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809532.400390 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411575.84 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 41157.58 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850689.984526 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370418.26 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 37041.83 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887731.810248 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333376.43 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 33337.64 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921069.453398 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300038.79 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 30003.88 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951073.332233 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270034.91 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 27003.49 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978076.823185 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243031.42 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 24303.14 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002379.965041 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218728.28 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 21872.83 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024252.792712 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196855.45 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 19685.54 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043938.337616 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177169.90 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 17716.99 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061655.328029 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159452.91 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 15945.29 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077600.619401 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143507.62 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 14350.76 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091951.381636 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129156.86 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 12915.69 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104867.067647 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116241.17 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 11624.12 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116491.185057 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104617.06 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 10461.71 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126952.890726 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94155.35 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 9415.54 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136368.425829 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84739.82 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 8473.98 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144842.407421 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76265.83 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 7626.58 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152468.990853 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68639.25 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 6863.93 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159332.915943 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61775.33 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 6177.53 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165510.448523 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55597.79 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 5559.78 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171070.227846 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50038.01 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 5003.80 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176074.029236 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45034.21 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 4503.42 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180577.450487 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40530.79 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 4053.08 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184630.529614 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36477.71 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3647.77 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188278.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32829.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191878.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29229.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195478.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25629.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199078.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22029.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202678.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18429.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206278.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14829.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209878.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11229.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213478.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7629.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217078.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4029.94 sec) 16:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220678.300827 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 429.94 sec) 16:17:00 [GPUGRID] [rr_sim] 2221108.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976796510.43G/439.78G) 16:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852769.15 16:17:00 [---] pick_jobs_to_run() start 16:17:00 [---] pick_jobs_to_run() end 16:17:00 [---] [rr_sim] end 16:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:17:00 [---] [cpu_sched_debug] preliminary job list: 16:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:17:00 [---] [cpu_sched_debug] final job list: 16:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:17:00 [---] [cpu_sched_debug] enforce_run_list: end 16:17:00 [---] choose_project(): 58620.000000 16:17:00 [WUProp@Home] doing RPC to get work 16:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376979.09 cpu 2124212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124212.90 16:17:00 [---] Got 0 tasks 16:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:18:00 [---] [rr_sim] doing sim: CPU sched 16:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976768920.07G/439.78G) 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2221045.50 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 222104.55 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222104.550487 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998940.95 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 199894.10 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421998.645925 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1799046.86 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 179904.69 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601903.331819 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619142.17 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 161914.22 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763817.549124 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457227.96 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 145722.80 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909540.344698 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311505.16 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 131150.52 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040690.860715 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180354.64 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 118035.46 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158726.325130 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062319.18 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 106231.92 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264958.243104 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956087.26 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 95608.73 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360566.969280 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860478.54 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 86047.85 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446614.822839 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774430.68 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 77443.07 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524057.891041 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696987.61 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 69698.76 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593756.652424 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627288.85 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 62728.89 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656485.537668 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564559.97 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 56456.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712941.534388 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508103.97 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 50810.40 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763751.931436 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457293.57 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 45729.36 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809481.288779 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411564.22 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 41156.42 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850637.710388 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370407.79 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 37040.78 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887678.489836 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333367.02 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 33336.70 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1921015.191339 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300030.31 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 30003.03 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951018.222691 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270027.28 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 27002.73 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978020.950909 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243024.55 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 24302.46 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002323.406305 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218722.10 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 21872.21 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024195.616161 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196849.89 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 19684.99 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043880.605031 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177164.90 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 17716.49 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061597.095015 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159448.41 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 15944.84 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077541.936000 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143503.57 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 14350.36 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091892.292887 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129153.21 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 12915.32 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104807.614085 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116237.89 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 11623.79 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116431.403163 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104614.10 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 10461.41 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126892.813333 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94152.69 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 9415.27 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136308.082487 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84737.42 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 8473.74 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144781.824725 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76263.68 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 7626.37 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152408.192739 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68637.31 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 6863.73 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159271.923952 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61773.58 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 6177.36 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165449.282043 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55596.22 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 5559.62 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171008.904325 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50036.60 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 5003.66 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176012.564379 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45032.94 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 4503.29 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180515.858428 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40529.65 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 4052.96 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184568.823072 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36476.68 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3647.67 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188216.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32829.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191816.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29229.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195416.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25629.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199016.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22029.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202616.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18429.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206216.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14829.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209816.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11229.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213416.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7629.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217016.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4029.01 sec) 16:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220616.491251 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 429.01 sec) 16:18:00 [GPUGRID] [rr_sim] 2221045.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976768920.07G/439.78G) 16:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852766.42 16:18:00 [---] pick_jobs_to_run() start 16:18:00 [---] pick_jobs_to_run() end 16:18:00 [---] [rr_sim] end 16:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:18:00 [---] [cpu_sched_debug] preliminary job list: 16:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:18:00 [---] [cpu_sched_debug] final job list: 16:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:18:00 [---] [cpu_sched_debug] enforce_run_list: end 16:18:00 [---] choose_project(): 58680.000000 16:18:00 [WUProp@Home] doing RPC to get work 16:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376919.09 cpu 2124152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124152.90 16:18:00 [---] Got 0 tasks 16:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:19:00 [---] [rr_sim] doing sim: CPU sched 16:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976741329.71G/439.78G) 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220982.77 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 222098.28 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222098.276798 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998884.49 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 199888.45 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421986.725917 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798996.04 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 179899.60 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601886.330124 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619096.44 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 161909.64 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763795.973910 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457186.79 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 145718.68 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909514.653317 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311468.11 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 131146.81 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040661.464784 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180321.30 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 118032.13 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158693.595104 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062289.17 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 106228.92 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264922.512392 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956060.26 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 95606.03 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360528.537951 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860454.23 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 86045.42 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446573.960955 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774408.81 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 77440.88 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1524014.841658 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696967.93 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 69696.79 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593711.634290 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627271.13 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 62727.11 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656438.747660 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564544.02 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 56454.40 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712893.149692 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508089.62 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 50808.96 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763702.111521 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457280.66 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 45728.07 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809430.177168 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411552.59 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 41155.26 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850585.436249 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370397.33 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 37039.73 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887625.169423 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333357.60 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 33335.76 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920960.929279 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300021.84 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 30002.18 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950963.113149 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270019.65 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 27001.97 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977965.078633 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243017.69 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 24301.77 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002266.847568 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218715.92 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 21871.59 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024138.439610 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196844.33 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 19684.43 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043822.872447 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177159.90 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 17715.99 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061538.862001 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159443.91 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 15944.39 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077483.252599 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143499.52 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 14349.95 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091833.204138 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129149.56 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 12914.96 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104748.160522 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116234.61 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 11623.46 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116371.621269 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104611.15 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 10461.11 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126832.735940 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94150.03 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 9415.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136247.739145 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84735.03 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 8473.50 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144721.242029 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76261.53 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 7626.15 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152347.394624 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68635.37 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 6863.54 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159210.931960 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61771.84 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 6177.18 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165388.115563 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55594.65 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 5559.47 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170947.580805 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50035.19 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 5003.52 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175951.099523 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45031.67 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 4503.17 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180454.266369 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40528.50 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 4052.85 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184507.116530 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36475.65 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3647.57 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188154.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32828.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191754.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29228.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195354.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25628.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198954.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22028.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202554.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18428.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206154.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14828.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209754.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11228.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213354.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7628.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216954.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4028.09 sec) 16:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220554.681676 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 428.09 sec) 16:19:00 [GPUGRID] [rr_sim] 2220982.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976741329.71G/439.78G) 16:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852763.68 16:19:00 [---] pick_jobs_to_run() start 16:19:00 [---] pick_jobs_to_run() end 16:19:00 [---] [rr_sim] end 16:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:19:00 [---] [cpu_sched_debug] preliminary job list: 16:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:19:00 [---] [cpu_sched_debug] final job list: 16:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:19:00 [---] [cpu_sched_debug] enforce_run_list: end 16:19:00 [---] choose_project(): 58740.000000 16:19:00 [WUProp@Home] doing RPC to get work 16:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376859.09 cpu 2124092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124092.90 16:19:00 [---] Got 0 tasks 16:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:20:00 [---] [rr_sim] doing sim: CPU sched 16:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976713739.36G/439.78G) 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220920.03 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 222092.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222092.003110 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998828.03 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 199882.80 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421974.805909 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798945.23 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 179894.52 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601869.328429 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619050.70 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 161905.07 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763774.398696 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457145.63 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 145714.56 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909488.961937 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311431.07 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 131143.11 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040632.068853 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180287.96 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 118028.80 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158660.865078 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062259.17 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 106225.92 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264886.781681 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956033.25 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 95603.32 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360490.106623 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860429.92 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 86042.99 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446533.099071 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774386.93 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 77438.69 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523971.792274 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696948.24 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 69694.82 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593666.616157 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627253.41 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 62725.34 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656391.957651 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564528.07 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 56452.81 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712844.764996 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508075.27 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 50807.53 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763652.291607 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457267.74 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 45726.77 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809379.065556 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411540.97 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 41154.10 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850533.162111 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370386.87 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 37038.69 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887571.849010 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333348.18 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 33334.82 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920906.667219 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300013.36 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 30001.34 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950908.003608 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270012.03 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 27001.20 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977909.206357 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243010.82 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 24301.08 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002210.288832 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218709.74 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 21870.97 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024081.263059 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196838.77 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 19683.88 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043765.139863 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177154.89 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 17715.49 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061480.628987 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159439.40 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 15943.94 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077424.569198 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143495.46 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 14349.55 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091774.115389 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129145.92 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 12914.59 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104688.706960 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116231.32 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 11623.13 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116311.839374 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104608.19 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 10460.82 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126772.658547 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94147.37 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 9414.74 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136187.395803 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84732.64 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 8473.26 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144660.659333 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76259.37 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 7625.94 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152286.596510 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68633.43 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 6863.34 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159149.939969 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61770.09 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 6177.01 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165326.949082 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55593.08 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 5559.31 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170886.257284 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50033.77 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 5003.38 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175889.634666 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45030.40 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 4503.04 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180392.674310 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40527.36 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 4052.74 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184445.409989 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36474.62 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3647.46 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188092.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32827.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191692.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29227.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195292.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25627.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198892.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22027.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202492.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18427.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206092.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14827.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209692.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11227.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213292.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7627.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216892.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4027.16 sec) 16:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220492.872100 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 427.16 sec) 16:20:00 [GPUGRID] [rr_sim] 2220920.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976713739.36G/439.78G) 16:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852760.94 16:20:00 [---] pick_jobs_to_run() start 16:20:00 [---] pick_jobs_to_run() end 16:20:00 [---] [rr_sim] end 16:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:20:00 [---] [cpu_sched_debug] preliminary job list: 16:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:20:00 [---] [cpu_sched_debug] final job list: 16:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:20:00 [---] [cpu_sched_debug] enforce_run_list: end 16:20:00 [---] choose_project(): 58800.000000 16:20:00 [WUProp@Home] doing RPC to get work 16:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376799.09 cpu 2124032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2124032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2124032.90 16:20:00 [---] Got 0 tasks 16:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:21:00 [---] [rr_sim] doing sim: CPU sched 16:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976686149.00G/439.78G) 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220857.29 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 222085.73 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222085.729422 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998771.56 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 199877.16 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421962.885902 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798894.41 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 179889.44 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601852.326734 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1619004.97 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 161900.50 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763752.823482 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457104.47 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 145710.45 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909463.270556 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311394.02 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 131139.40 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040602.672922 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180254.62 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 118025.46 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158628.135052 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062229.16 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 106222.92 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264851.050969 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 956006.24 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 95600.62 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360451.675294 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860405.62 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 86040.56 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446492.237187 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774365.06 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 77436.51 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523928.742890 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696928.55 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 69692.86 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593621.598023 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627235.70 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 62723.57 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656345.167643 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564512.13 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 56451.21 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712796.380300 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508060.91 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 50806.09 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763602.471692 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457254.82 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 45725.48 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809327.953945 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411529.34 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 41152.93 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850480.887973 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370376.41 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 37037.64 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887518.528597 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333338.77 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 33333.88 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920852.405160 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300004.89 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 30000.49 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950852.894066 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270004.40 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 27000.44 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977853.334081 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243003.96 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 24300.40 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002153.730095 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218703.56 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 21870.36 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024024.086507 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196833.21 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 19683.32 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043707.407279 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177149.89 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 17714.99 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061422.395973 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159434.90 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 15943.49 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077365.885798 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143491.41 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 14349.14 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091715.026640 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129142.27 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 12914.23 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104629.253398 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116228.04 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 11622.80 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116252.057480 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104605.24 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 10460.52 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126712.581154 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94144.71 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 9414.47 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136127.052461 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84730.24 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 8473.02 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144600.076637 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76257.22 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 7625.72 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152225.798395 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68631.50 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 6863.15 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159088.947977 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61768.35 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 6176.83 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165265.782602 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55591.51 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 5559.15 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170824.933764 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50032.36 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 5003.24 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175828.169809 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45029.12 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 4502.91 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180331.082250 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40526.21 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 4052.62 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184383.703447 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36473.59 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3647.36 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188031.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32826.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191631.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29226.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195231.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25626.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198831.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22026.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202431.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18426.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206031.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14826.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209631.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11226.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213231.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7626.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216831.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4026.23 sec) 16:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220431.062525 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 426.23 sec) 16:21:00 [GPUGRID] [rr_sim] 2220857.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976686149.00G/439.78G) 16:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852758.21 16:21:00 [---] pick_jobs_to_run() start 16:21:00 [---] pick_jobs_to_run() end 16:21:00 [---] [rr_sim] end 16:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:21:00 [---] [cpu_sched_debug] preliminary job list: 16:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:21:00 [---] [cpu_sched_debug] final job list: 16:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:21:00 [---] [cpu_sched_debug] enforce_run_list: end 16:21:00 [---] choose_project(): 58860.000000 16:21:00 [WUProp@Home] doing RPC to get work 16:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376739.09 cpu 2123972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123972.90 16:21:00 [---] Got 0 tasks 16:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:22:00 [---] [rr_sim] doing sim: CPU sched 16:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976658558.65G/439.78G) 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220794.56 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 222079.46 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222079.455734 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998715.10 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 199871.51 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421950.965894 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798843.59 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 179884.36 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601835.325039 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618959.23 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 161895.92 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763731.248268 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457063.31 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 145706.33 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909437.579175 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311356.98 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 131135.70 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040573.276992 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180221.28 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 118022.13 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158595.405026 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062199.15 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 106219.92 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264815.320257 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955979.24 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 95597.92 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360413.243965 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860381.31 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 86038.13 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446451.375303 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774343.18 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 77434.32 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523885.693506 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696908.86 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 69690.89 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593576.579889 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627217.98 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 62721.80 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656298.377634 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564496.18 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 56449.62 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712747.995605 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508046.56 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 50804.66 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763552.651778 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457241.91 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 45724.19 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809276.842334 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411517.72 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 41151.77 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850428.613834 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370365.94 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 37036.59 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887465.208185 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333329.35 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 33332.93 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920798.143100 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299996.41 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 29999.64 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950797.784524 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269996.77 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 26999.68 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977797.461805 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242997.10 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 24299.71 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002097.171358 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218697.39 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 21869.74 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023966.909956 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196827.65 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 19682.76 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043649.674695 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177144.88 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 17714.49 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061364.162959 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159430.39 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 15943.04 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077307.202397 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143487.35 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 14348.74 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091655.937891 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129138.62 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 12913.86 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104569.799836 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116224.76 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 11622.48 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116192.275586 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104602.28 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 10460.23 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126652.503761 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94142.05 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 9414.21 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136066.709119 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84727.85 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 8472.78 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144539.493941 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76255.06 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 7625.51 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152165.000280 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68629.56 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 6862.96 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159027.955986 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61766.60 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 6176.66 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165204.616121 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55589.94 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 5558.99 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170763.610243 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50030.95 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 5003.09 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175766.704952 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45027.85 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 4502.79 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180269.490191 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40525.07 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 4052.51 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184321.996906 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36472.56 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3647.26 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187969.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32825.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191569.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29225.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195169.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25625.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198769.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22025.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202369.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18425.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205969.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14825.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209569.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11225.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213169.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7625.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216769.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4025.30 sec) 16:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220369.252949 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 425.30 sec) 16:22:00 [GPUGRID] [rr_sim] 2220794.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976658558.65G/439.78G) 16:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852755.47 16:22:00 [---] pick_jobs_to_run() start 16:22:00 [---] pick_jobs_to_run() end 16:22:00 [---] [rr_sim] end 16:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:22:00 [---] [cpu_sched_debug] preliminary job list: 16:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:22:00 [---] [cpu_sched_debug] final job list: 16:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:22:00 [---] [cpu_sched_debug] enforce_run_list: end 16:22:00 [---] choose_project(): 58920.000000 16:22:00 [WUProp@Home] doing RPC to get work 16:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376679.09 cpu 2123912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123912.90 16:22:00 [---] Got 0 tasks 16:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:23:00 [---] [rr_sim] doing sim: CPU sched 16:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976630968.29G/439.78G) 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220731.82 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 222073.18 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222073.182046 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998658.64 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 199865.86 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421939.045887 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798792.77 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 179879.28 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601818.323343 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618913.50 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 161891.35 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763709.673055 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1457022.15 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 145702.21 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909411.887795 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311319.93 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 131131.99 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040543.881061 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180187.94 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 118018.79 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158562.675000 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062169.15 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 106216.91 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264779.589546 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955952.23 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 95595.22 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360374.812637 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860357.01 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 86035.70 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446410.513419 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774321.31 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 77432.13 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523842.644122 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696889.18 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 69688.92 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593531.561756 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627200.26 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 62720.03 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656251.587626 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564480.23 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 56448.02 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712699.610909 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508032.21 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 50803.22 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763502.831863 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457228.99 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 45722.90 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809225.730723 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411506.09 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 41150.61 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850376.339696 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370355.48 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 37035.55 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887411.887772 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333319.93 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 33331.99 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920743.881040 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299987.94 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 29998.79 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950742.674982 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269989.15 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 26998.91 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977741.589529 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242990.23 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 24299.02 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002040.612622 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218691.21 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 21869.12 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023909.733405 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196822.09 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 19682.21 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043591.942110 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177139.88 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 17713.99 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061305.929945 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159425.89 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 15942.59 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077248.518996 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143483.30 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 14348.33 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091596.849142 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129134.97 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 12913.50 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104510.346273 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116221.47 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 11622.15 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116132.493692 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104599.33 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 10459.93 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126592.426368 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94139.39 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 9413.94 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136006.365777 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84725.45 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 8472.55 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144478.911245 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76252.91 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 7625.29 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152104.202166 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68627.62 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 6862.76 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158966.963995 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61764.86 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 6176.49 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165143.449641 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55588.37 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 5558.84 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170702.286722 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50029.53 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 5002.95 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175705.240096 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45026.58 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 4502.66 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180207.898132 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40523.92 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 4052.39 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184260.290364 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36471.53 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3647.15 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187907.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32824.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191507.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29224.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195107.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25624.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198707.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22024.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202307.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18424.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205907.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14824.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209507.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11224.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213107.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7624.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216707.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4024.38 sec) 16:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220307.443373 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 424.38 sec) 16:23:00 [GPUGRID] [rr_sim] 2220731.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976630968.29G/439.78G) 16:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852752.73 16:23:00 [---] pick_jobs_to_run() start 16:23:00 [---] pick_jobs_to_run() end 16:23:00 [---] [rr_sim] end 16:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:23:00 [---] [cpu_sched_debug] preliminary job list: 16:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:23:00 [---] [cpu_sched_debug] final job list: 16:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:23:00 [---] [cpu_sched_debug] enforce_run_list: end 16:23:00 [---] choose_project(): 58980.000000 16:23:00 [WUProp@Home] doing RPC to get work 16:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376619.09 cpu 2123852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123852.90 16:23:00 [---] Got 0 tasks 16:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:24:00 [---] [rr_sim] doing sim: CPU sched 16:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976603377.93G/439.78G) 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220669.08 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 222066.91 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222066.908357 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998602.18 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 199860.22 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421927.125879 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798741.96 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 179874.20 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601801.321648 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618867.76 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 161886.78 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763688.097841 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456980.99 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 145698.10 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909386.196414 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311282.89 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 131128.29 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040514.485130 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180154.60 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 118015.46 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158529.944974 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062139.14 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 106213.91 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264743.858834 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955925.22 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 95592.52 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360336.381308 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860332.70 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 86033.27 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446369.651535 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774299.43 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 77429.94 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523799.594739 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696869.49 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 69686.95 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593486.543622 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627182.54 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 62718.25 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656204.797617 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564464.29 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 56446.43 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712651.226213 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508017.86 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 50801.79 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763453.011949 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457216.07 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 45721.61 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809174.619111 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411494.46 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 41149.45 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850324.065558 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370345.02 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 37034.50 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887358.567359 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333310.52 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 33331.05 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920689.618981 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299979.46 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 29997.95 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950687.565440 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269981.52 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 26998.15 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977685.717253 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242983.37 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 24298.34 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001984.053885 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218685.03 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 21868.50 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023852.556854 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196816.53 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 19681.65 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043534.209526 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177134.87 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 17713.49 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061247.696931 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159421.39 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 15942.14 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077189.835595 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143479.25 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 14347.92 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091537.760393 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129131.32 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 12913.13 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104450.892711 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116218.19 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 11621.82 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116072.711797 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104596.37 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 10459.64 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126532.348975 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94136.73 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 9413.67 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135946.022435 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84723.06 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 8472.31 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144418.328549 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76250.76 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 7625.08 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152043.404051 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68625.68 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 6862.57 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158905.972003 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61763.11 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 6176.31 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165082.283160 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55586.80 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 5558.68 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170640.963202 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50028.12 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 5002.81 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175643.775239 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45025.31 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 4502.53 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180146.306072 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40522.78 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 4052.28 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184198.583822 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36470.50 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3647.05 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187845.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32823.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191445.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29223.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195045.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25623.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198645.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22023.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202245.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18423.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205845.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14823.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209445.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11223.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213045.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7623.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216645.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4023.45 sec) 16:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220245.633798 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 423.45 sec) 16:24:00 [GPUGRID] [rr_sim] 2220669.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976603377.93G/439.78G) 16:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852750.00 16:24:00 [---] pick_jobs_to_run() start 16:24:00 [---] pick_jobs_to_run() end 16:24:00 [---] [rr_sim] end 16:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:24:00 [---] [cpu_sched_debug] preliminary job list: 16:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:24:00 [---] [cpu_sched_debug] final job list: 16:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:24:00 [---] [cpu_sched_debug] enforce_run_list: end 16:24:00 [---] choose_project(): 59040.000000 16:24:00 [WUProp@Home] doing RPC to get work 16:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376559.09 cpu 2123792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123792.90 16:24:00 [---] Got 0 tasks 16:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:25:00 [---] [rr_sim] doing sim: CPU sched 16:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976575787.58G/439.78G) 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220606.35 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 222060.63 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222060.634669 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998545.71 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 199854.57 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421915.205871 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798691.14 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 179869.11 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601784.319953 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618822.03 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 161882.20 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763666.522627 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456939.82 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 145693.98 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909360.505034 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311245.84 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 131124.58 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040485.089199 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180121.26 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 118012.13 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158497.214949 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062109.13 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 106210.91 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264708.128123 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955898.22 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 95589.82 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360297.949980 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860308.40 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 86030.84 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446328.789651 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774277.56 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 77427.76 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523756.545355 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696849.80 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 69684.98 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593441.525489 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627164.82 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 62716.48 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656158.007609 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564448.34 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 56444.83 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712602.841517 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 508003.51 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 50800.35 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763403.192034 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457203.15 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 45720.32 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809123.507500 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411482.84 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 41148.28 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850271.791419 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370334.56 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 37033.46 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887305.246947 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333301.10 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 33330.11 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920635.356921 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299970.99 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 29997.10 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950632.455898 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269973.89 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 26997.39 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977629.844977 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242976.50 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 24297.65 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001927.495149 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218678.85 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 21867.89 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023795.380303 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196810.97 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 19681.10 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043476.476942 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177129.87 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 17712.99 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061189.463917 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159416.88 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 15941.69 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077131.152194 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143475.19 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 14347.52 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091478.671644 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129127.68 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 12912.77 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104391.439149 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116214.91 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 11621.49 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116012.929903 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104593.42 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 10459.34 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126472.271582 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94134.08 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 9413.41 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135885.679093 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84720.67 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 8472.07 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144357.745853 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76248.60 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 7624.86 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151982.605936 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68623.74 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 6862.37 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158844.980012 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61761.37 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 6176.14 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165021.116680 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55585.23 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 5558.52 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170579.639681 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50026.71 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 5002.67 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175582.310382 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45024.04 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 4502.40 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180084.714013 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40521.63 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 4052.16 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184136.877281 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36469.47 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3646.95 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187783.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32822.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191383.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29222.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194983.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25622.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198583.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22022.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202183.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18422.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205783.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14822.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209383.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11222.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212983.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7622.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216583.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4022.52 sec) 16:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220183.824222 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 422.52 sec) 16:25:00 [GPUGRID] [rr_sim] 2220606.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976575787.58G/439.78G) 16:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852747.26 16:25:00 [---] pick_jobs_to_run() start 16:25:00 [---] pick_jobs_to_run() end 16:25:00 [---] [rr_sim] end 16:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:25:00 [---] [cpu_sched_debug] preliminary job list: 16:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:25:00 [---] [cpu_sched_debug] final job list: 16:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:25:00 [---] [cpu_sched_debug] enforce_run_list: end 16:25:00 [---] choose_project(): 59100.000000 16:25:00 [WUProp@Home] doing RPC to get work 16:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376499.09 cpu 2123732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123732.90 16:25:00 [---] Got 0 tasks 16:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:26:00 [---] [rr_sim] doing sim: CPU sched 16:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976548197.22G/439.78G) 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220543.61 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 222054.36 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222054.360981 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998489.25 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 199848.92 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421903.285864 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798640.32 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 179864.03 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601767.318258 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618776.29 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 161877.63 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763644.947413 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456898.66 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 145689.87 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909334.813653 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311208.80 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 131120.88 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040455.693269 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180087.92 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 118008.79 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158464.484923 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062079.12 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 106207.91 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264672.397411 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955871.21 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 95587.12 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360259.518651 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860284.09 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 86028.41 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446287.927767 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774255.68 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 77425.57 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523713.495971 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696830.11 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 69683.01 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593396.507355 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627147.10 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 62714.71 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656111.217600 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564432.39 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 56443.24 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712554.456821 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507989.15 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 50798.92 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763353.372120 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457190.24 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 45719.02 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809072.395889 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411471.21 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 41147.12 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850219.517281 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370324.09 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 37032.41 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887251.926534 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333291.68 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 33329.17 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920581.094861 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299962.51 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 29996.25 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950577.346356 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269966.26 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 26996.63 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977573.972701 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242969.64 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 24296.96 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001870.936412 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218672.67 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 21867.27 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023738.203752 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196805.41 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 19680.54 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043418.744358 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177124.87 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 17712.49 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061131.230903 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159412.38 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 15941.24 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077072.468793 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143471.14 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 14347.11 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091419.582895 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129124.03 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 12912.40 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104331.985586 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116211.62 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 11621.16 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115953.148009 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104590.46 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 10459.05 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126412.194189 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94131.42 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 9413.14 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135825.335751 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84718.27 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 8471.83 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144297.163157 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76246.45 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 7624.64 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151921.807822 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68621.80 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 6862.18 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158783.988021 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61759.62 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 6175.96 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164959.950199 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55583.66 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 5558.37 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170518.316160 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50025.29 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 5002.53 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175520.845525 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45022.76 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 4502.28 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180023.121954 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40520.49 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 4052.05 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184075.170739 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36468.44 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3646.84 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187722.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32821.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191322.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29221.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194922.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25621.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198522.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22021.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202122.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18421.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205722.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14821.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209322.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11221.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212922.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7621.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216522.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4021.60 sec) 16:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220122.014646 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 421.60 sec) 16:26:00 [GPUGRID] [rr_sim] 2220543.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976548197.22G/439.78G) 16:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852744.52 16:26:00 [---] pick_jobs_to_run() start 16:26:00 [---] pick_jobs_to_run() end 16:26:00 [---] [rr_sim] end 16:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:26:00 [---] [cpu_sched_debug] preliminary job list: 16:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:26:00 [---] [cpu_sched_debug] final job list: 16:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:26:00 [---] [cpu_sched_debug] enforce_run_list: end 16:26:00 [---] choose_project(): 59160.000000 16:26:00 [WUProp@Home] doing RPC to get work 16:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376439.09 cpu 2123672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123672.90 16:26:00 [---] Got 0 tasks 16:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:27:00 [---] [rr_sim] doing sim: CPU sched 16:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976520606.86G/439.78G) 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220480.87 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 222048.09 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222048.087293 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998432.79 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 199843.28 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421891.365856 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798589.51 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 179858.95 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601750.316563 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618730.56 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 161873.06 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763623.372200 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456857.50 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 145685.75 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909309.122272 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311171.75 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 131117.18 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040426.297338 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180054.58 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 118005.46 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158431.754897 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062049.12 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 106204.91 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264636.666700 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955844.21 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 95584.42 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360221.087322 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860259.79 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 86025.98 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446247.065883 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774233.81 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 77423.38 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523670.446587 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696810.43 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 69681.04 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593351.489221 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627129.38 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 62712.94 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656064.427592 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564416.45 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 56441.64 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712506.072125 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507974.80 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 50797.48 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763303.552206 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457177.32 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 45717.73 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1809021.284278 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411459.59 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 41145.96 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850167.243143 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370313.63 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 37031.36 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887198.606121 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333282.27 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 33328.23 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920526.832802 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299954.04 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 29995.40 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950522.236814 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269958.64 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 26995.86 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977518.100425 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242962.77 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 24296.28 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001814.377676 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218666.50 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 21866.65 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023681.027201 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196799.85 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 19679.98 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043361.011773 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177119.86 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 17711.99 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061072.997889 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159407.88 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 15940.79 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077013.785393 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143467.09 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 14346.71 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091360.494146 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129120.38 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 12912.04 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104272.532024 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116208.34 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 11620.83 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115893.366114 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104587.51 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 10458.75 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126352.116796 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94128.76 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 9412.88 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135764.992409 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84715.88 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 8471.59 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144236.580461 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76244.29 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 7624.43 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151861.009707 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68619.86 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 6861.99 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158722.996029 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61757.88 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 6175.79 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164898.783719 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55582.09 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 5558.21 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170456.992640 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50023.88 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 5002.39 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175459.380669 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45021.49 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 4502.15 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179961.529894 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40519.34 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 4051.93 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184013.464198 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36467.41 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3646.74 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187660.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32820.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191260.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29220.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194860.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25620.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198460.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22020.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202060.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18420.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205660.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14820.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209260.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11220.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212860.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7620.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216460.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4020.67 sec) 16:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2220060.205071 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 420.67 sec) 16:27:00 [GPUGRID] [rr_sim] 2220480.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976520606.86G/439.78G) 16:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852741.79 16:27:00 [---] pick_jobs_to_run() start 16:27:00 [---] pick_jobs_to_run() end 16:27:00 [---] [rr_sim] end 16:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:27:00 [---] [cpu_sched_debug] preliminary job list: 16:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:27:00 [---] [cpu_sched_debug] final job list: 16:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:27:00 [---] [cpu_sched_debug] enforce_run_list: end 16:27:00 [---] choose_project(): 59220.000000 16:27:00 [WUProp@Home] doing RPC to get work 16:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376379.09 cpu 2123612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123612.90 16:27:00 [---] Got 0 tasks 16:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:28:00 [---] [rr_sim] doing sim: CPU sched 16:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976493016.51G/439.78G) 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220418.14 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 222041.81 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222041.813604 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998376.32 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 199837.63 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421879.445849 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798538.69 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 179853.87 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601733.314868 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618684.82 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 161868.48 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763601.796986 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456816.34 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 145681.63 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909283.430892 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311134.71 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 131113.47 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040396.901407 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1180021.23 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 118002.12 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158399.024871 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1062019.11 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 106201.91 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264600.935988 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955817.20 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 95581.72 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360182.655994 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860235.48 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 86023.55 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446206.203999 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774211.93 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 77421.19 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523627.397204 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696790.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 69679.07 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593306.471088 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627111.66 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 62711.17 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1656017.637583 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564400.50 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 56440.05 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712457.687430 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507960.45 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 50796.04 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763253.732291 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457164.40 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 45716.44 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808970.172666 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411447.96 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 41144.80 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850114.969004 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370303.17 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 37030.32 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887145.285708 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333272.85 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 33327.29 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920472.570742 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299945.57 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 29994.56 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950467.127272 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269951.01 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 26995.10 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977462.228150 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242955.91 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 24295.59 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001757.818939 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218660.32 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 21866.03 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023623.850650 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196794.29 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 19679.43 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043303.279189 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177114.86 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 17711.49 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061014.764875 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159403.37 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 15940.34 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076955.101992 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143463.03 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 14346.30 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091301.405397 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129116.73 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 12911.67 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104213.078462 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116205.06 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 11620.51 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115833.584220 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104584.55 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 10458.46 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126292.039403 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94126.10 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 9412.61 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135704.649067 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84713.49 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 8471.35 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144175.997765 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76242.14 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 7624.21 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151800.211593 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68617.92 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 6861.79 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158662.004038 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61756.13 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 6175.61 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164837.617239 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55580.52 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 5558.05 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170395.669119 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50022.47 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 5002.25 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175397.915812 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45020.22 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 4502.02 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179899.937835 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40518.20 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 4051.82 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183951.757656 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36466.38 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3646.64 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187598.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32819.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191198.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29219.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194798.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25619.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198398.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22019.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201998.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18419.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205598.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14819.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209198.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11219.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212798.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7619.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216398.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4019.74 sec) 16:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219998.395495 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 419.74 sec) 16:28:00 [GPUGRID] [rr_sim] 2220418.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976493016.51G/439.78G) 16:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852739.05 16:28:00 [---] pick_jobs_to_run() start 16:28:00 [---] pick_jobs_to_run() end 16:28:00 [---] [rr_sim] end 16:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:28:00 [---] [cpu_sched_debug] preliminary job list: 16:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:28:00 [---] [cpu_sched_debug] final job list: 16:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:28:00 [---] [cpu_sched_debug] enforce_run_list: end 16:28:00 [---] choose_project(): 59280.000000 16:28:00 [WUProp@Home] doing RPC to get work 16:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376319.09 cpu 2123552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123552.90 16:28:00 [---] Got 0 tasks 16:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:29:00 [---] [rr_sim] doing sim: CPU sched 16:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976465426.15G/439.78G) 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220355.40 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 222035.54 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222035.539916 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998319.86 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 199831.99 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421867.525841 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798487.87 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 179848.79 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601716.313173 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618639.09 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 161863.91 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763580.221772 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456775.18 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 145677.52 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909257.739511 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311097.66 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 131109.77 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040367.505476 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179987.89 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 117998.79 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158366.294845 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061989.10 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 106198.91 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264565.205277 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955790.19 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 95579.02 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360144.224665 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860211.17 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 86021.12 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446165.342115 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774190.06 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 77419.01 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523584.347820 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696771.05 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 69677.11 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593261.452954 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627093.95 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 62709.39 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655970.847575 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564384.55 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 56438.46 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712409.302734 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507946.10 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 50794.61 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763203.912377 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457151.49 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 45715.15 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808919.061055 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411436.34 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 41143.63 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850062.694866 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370292.70 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 37029.27 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887091.965296 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333263.43 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 33326.34 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920418.308682 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299937.09 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 29993.71 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950412.017730 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269943.38 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 26994.34 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977406.355874 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242949.04 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 24294.90 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001701.260203 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218654.14 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 21865.41 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023566.674099 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196788.73 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 19678.87 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043245.546605 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177109.85 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 17710.99 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060956.531861 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159398.87 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 15939.89 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076896.418591 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143458.98 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 14345.90 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091242.316648 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129113.08 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 12911.31 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104153.624900 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116201.77 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 11620.18 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115773.802326 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104581.60 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 10458.16 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126231.962010 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94123.44 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 9412.34 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135644.305725 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84711.09 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 8471.11 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144115.415069 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76239.98 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 7624.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151739.413478 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68615.99 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 6861.60 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158601.012047 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61754.39 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 6175.44 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164776.450758 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55578.95 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 5557.89 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170334.345599 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50021.05 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 5002.11 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175336.450955 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45018.95 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 4501.89 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179838.345776 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40517.05 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 4051.71 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183890.051114 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36465.35 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3646.53 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187536.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32818.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191136.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29218.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194736.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25618.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198336.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22018.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201936.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18418.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205536.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14818.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209136.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11218.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212736.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7618.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216336.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4018.81 sec) 16:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219936.585919 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 418.81 sec) 16:29:00 [GPUGRID] [rr_sim] 2220355.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976465426.15G/439.78G) 16:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852736.31 16:29:00 [---] pick_jobs_to_run() start 16:29:00 [---] pick_jobs_to_run() end 16:29:00 [---] [rr_sim] end 16:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:29:00 [---] [cpu_sched_debug] preliminary job list: 16:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:29:00 [---] [cpu_sched_debug] final job list: 16:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:29:00 [---] [cpu_sched_debug] enforce_run_list: end 16:29:00 [---] choose_project(): 59340.000000 16:29:00 [WUProp@Home] doing RPC to get work 16:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376259.09 cpu 2123492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123492.90 16:29:00 [---] Got 0 tasks 16:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:30:00 [---] [rr_sim] doing sim: CPU sched 16:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976437835.80G/439.78G) 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220292.66 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 222029.27 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222029.266228 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998263.40 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 199826.34 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421855.605833 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798437.06 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 179843.71 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601699.311478 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618593.35 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 161859.34 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763558.646558 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456734.02 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 145673.40 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909232.048130 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311060.61 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 131106.06 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040338.109545 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179954.55 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 117995.46 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158333.564819 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061959.10 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 106195.91 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264529.474565 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955763.19 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 95576.32 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360105.793337 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860186.87 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 86018.69 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446124.480231 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774168.18 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 77416.82 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523541.298436 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696751.36 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 69675.14 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593216.434820 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627076.23 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 62707.62 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655924.057566 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564368.60 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 56436.86 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712360.918038 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507931.74 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 50793.17 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763154.092462 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457138.57 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 45713.86 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808867.949444 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411424.71 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 41142.47 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1850010.420728 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370282.24 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 37028.22 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1887038.644883 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333254.02 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 33325.40 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920364.046623 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299928.62 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 29992.86 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950356.908188 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269935.75 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 26993.58 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977350.483598 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242942.18 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 24294.22 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001644.701466 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218647.96 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 21864.80 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023509.497547 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196783.16 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 19678.32 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043187.814021 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177104.85 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 17710.48 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060898.298847 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159394.36 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 15939.44 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076837.735190 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143454.93 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 14345.49 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091183.227899 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129109.43 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 12910.94 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104094.171337 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116198.49 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 11619.85 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115714.020432 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104578.64 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 10457.86 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126171.884616 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94120.78 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 9412.08 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135583.962383 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84708.70 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 8470.87 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144054.832373 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76237.83 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 7623.78 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151678.615363 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68614.05 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 6861.40 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158540.020055 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61752.64 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 6175.26 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164715.284278 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55577.38 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 5557.74 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170273.022078 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50019.64 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 5001.96 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175274.986098 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45017.68 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 4501.77 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179776.753716 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40515.91 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 4051.59 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183828.344573 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36464.32 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3646.43 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187474.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32817.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191074.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29217.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194674.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25617.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198274.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22017.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201874.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18417.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205474.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14817.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209074.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11217.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212674.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7617.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216274.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4017.89 sec) 16:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219874.776344 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 417.89 sec) 16:30:00 [GPUGRID] [rr_sim] 2220292.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976437835.80G/439.78G) 16:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852733.57 16:30:00 [---] pick_jobs_to_run() start 16:30:00 [---] pick_jobs_to_run() end 16:30:00 [---] [rr_sim] end 16:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:30:00 [---] [cpu_sched_debug] preliminary job list: 16:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:30:00 [---] [cpu_sched_debug] final job list: 16:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:30:00 [---] [cpu_sched_debug] enforce_run_list: end 16:30:00 [---] choose_project(): 59400.000000 16:30:00 [WUProp@Home] doing RPC to get work 16:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376199.09 cpu 2123432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123432.90 16:30:00 [---] Got 0 tasks 16:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:31:00 [---] [rr_sim] doing sim: CPU sched 16:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976410245.44G/439.78G) 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220229.93 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 222022.99 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222022.992540 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998206.93 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 199820.69 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421843.685826 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798386.24 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 179838.62 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601682.309783 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618547.62 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 161854.76 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763537.071344 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456692.85 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 145669.29 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909206.356750 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1311023.57 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 131102.36 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040308.713615 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179921.21 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 117992.12 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158300.834793 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061929.09 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 106192.91 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264493.743854 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955736.18 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 95573.62 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360067.362008 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860162.56 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 86016.26 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446083.618347 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774146.31 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 77414.63 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523498.249052 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696731.68 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 69673.17 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593171.416687 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627058.51 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 62705.85 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655877.267558 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564352.66 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 56435.27 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712312.533342 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507917.39 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 50791.74 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763104.272548 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457125.65 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 45712.57 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808816.837833 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411413.09 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 41141.31 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849958.146589 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370271.78 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 37027.18 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886985.324470 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333244.60 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 33324.46 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920309.784563 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299920.14 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 29992.01 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950301.798647 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269928.13 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 26992.81 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977294.611322 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242935.31 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 24293.53 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001588.142729 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218641.78 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 21864.18 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023452.320996 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196777.60 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 19677.76 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043130.081436 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177099.84 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 17709.98 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060840.065833 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159389.86 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 15938.99 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076779.051789 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143450.87 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 14345.09 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091124.139150 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129105.79 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 12910.58 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104034.717775 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116195.21 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 11619.52 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115654.238537 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104575.69 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 10457.57 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126111.807223 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94118.12 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 9411.81 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135523.619041 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84706.31 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 8470.63 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143994.249677 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76235.68 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 7623.57 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151617.817249 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68612.11 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 6861.21 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158479.028064 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61750.90 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 6175.09 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164654.117797 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55575.81 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 5557.58 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170211.698557 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50018.23 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 5001.82 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175213.521241 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45016.40 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 4501.64 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179715.161657 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40514.76 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 4051.48 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183766.638031 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36463.29 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3646.33 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187412.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32816.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191012.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29216.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194612.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25616.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198212.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22016.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201812.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18416.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205412.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14816.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209012.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11216.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212612.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7616.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216212.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4016.96 sec) 16:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219812.966768 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416.96 sec) 16:31:00 [GPUGRID] [rr_sim] 2220229.93: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976410245.44G/439.78G) 16:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852730.84 16:31:00 [---] pick_jobs_to_run() start 16:31:00 [---] pick_jobs_to_run() end 16:31:00 [---] [rr_sim] end 16:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:31:00 [---] [cpu_sched_debug] preliminary job list: 16:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:31:00 [---] [cpu_sched_debug] final job list: 16:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:31:00 [---] [cpu_sched_debug] enforce_run_list: end 16:31:00 [---] choose_project(): 59460.000000 16:31:00 [WUProp@Home] doing RPC to get work 16:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376139.09 cpu 2123372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123372.90 16:31:00 [---] Got 0 tasks 16:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:32:00 [---] [rr_sim] doing sim: CPU sched 16:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976382655.08G/439.78G) 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220167.19 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 222016.72 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222016.718852 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998150.47 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 199815.05 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421831.765818 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798335.42 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 179833.54 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601665.308088 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618501.88 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 161850.19 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763515.496131 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456651.69 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 145665.17 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909180.665369 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310986.52 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 131098.65 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040279.317684 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179887.87 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 117988.79 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158268.104767 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061899.08 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 106189.91 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264458.013142 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955709.18 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 95570.92 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1360028.930679 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860138.26 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 86013.83 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446042.756463 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774124.43 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 77412.44 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523455.199668 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696711.99 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 69671.20 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593126.398553 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627040.79 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 62704.08 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655830.477550 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564336.71 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 56433.67 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712264.148646 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507903.04 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 50790.30 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763054.452633 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457112.74 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 45711.27 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808765.726221 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411401.46 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 41140.15 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849905.872451 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370261.32 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 37026.13 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886932.004057 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333235.18 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 33323.52 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920255.522503 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299911.67 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 29991.17 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950246.689105 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269920.50 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 26992.05 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977238.739046 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242928.45 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 24292.84 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001531.583993 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218635.60 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 21863.56 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023395.144445 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196772.04 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 19677.20 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043072.348852 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177094.84 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 17709.48 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060781.832819 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159385.36 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 15938.54 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076720.368388 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143446.82 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 14344.68 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091065.050401 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129102.14 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 12910.21 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103975.264213 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116191.92 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 11619.19 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115594.456643 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104572.73 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 10457.27 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126051.729830 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94115.46 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 9411.55 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135463.275699 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84703.91 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 8470.39 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143933.666981 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76233.52 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 7623.35 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151557.019134 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68610.17 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 6861.02 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158418.036072 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61749.15 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 6174.92 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164592.951317 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55574.24 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 5557.42 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170150.375037 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50016.81 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 5001.68 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175152.056385 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45015.13 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 4501.51 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179653.569598 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40513.62 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 4051.36 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183704.931490 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36462.26 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3646.23 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187351.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32816.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190951.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29216.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194551.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25616.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198151.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22016.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201751.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18416.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205351.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14816.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208951.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11216.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212551.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7616.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216151.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4016.03 sec) 16:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219751.157192 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 416.03 sec) 16:32:00 [GPUGRID] [rr_sim] 2220167.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976382655.08G/439.78G) 16:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852728.10 16:32:00 [---] pick_jobs_to_run() start 16:32:00 [---] pick_jobs_to_run() end 16:32:00 [---] [rr_sim] end 16:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:32:00 [---] [cpu_sched_debug] preliminary job list: 16:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:32:00 [---] [cpu_sched_debug] final job list: 16:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:32:00 [---] [cpu_sched_debug] enforce_run_list: end 16:32:00 [---] choose_project(): 59520.000000 16:32:00 [WUProp@Home] doing RPC to get work 16:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376079.09 cpu 2123312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123312.90 16:32:00 [---] Got 0 tasks 16:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:33:00 [---] [rr_sim] doing sim: CPU sched 16:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976355064.73G/439.78G) 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220104.45 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 222010.45 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222010.445163 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998094.01 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 199809.40 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421819.845810 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798284.61 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 179828.46 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601648.306393 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618456.15 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 161845.61 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763493.920917 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456610.53 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 145661.05 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909154.973989 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310949.48 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 131094.95 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040249.921753 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179854.53 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 117985.45 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158235.374741 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061869.08 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 106186.91 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264422.282431 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955682.17 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 95568.22 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359990.499351 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860113.95 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 86011.40 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1446001.894579 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774102.56 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 77410.26 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523412.150285 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696692.30 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 69669.23 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593081.380420 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627023.07 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 62702.31 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655783.687541 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564320.76 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 56432.08 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712215.763950 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507888.69 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 50788.87 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1763004.632719 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457099.82 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 45709.98 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808714.614610 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411389.84 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 41138.98 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849853.598313 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370250.85 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 37025.09 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886878.683645 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333225.77 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 33322.58 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920201.260444 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299903.19 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 29990.32 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950191.579563 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269912.87 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 26991.29 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977182.866770 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242921.58 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 24292.16 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001475.025256 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218629.43 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 21862.94 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023337.967894 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196766.48 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 19676.65 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043014.616268 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177089.84 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 17708.98 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060723.599805 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159380.85 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 15938.09 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076661.684988 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143442.77 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 14344.28 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091005.961652 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129098.49 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 12909.85 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103915.810650 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116188.64 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 11618.86 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115534.674749 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104569.78 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 10456.98 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125991.652437 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94112.80 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 9411.28 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135402.932357 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84701.52 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 8470.15 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143873.084285 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76231.37 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 7623.14 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151496.221020 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68608.23 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 6860.82 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158357.044081 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61747.41 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 6174.74 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164531.784836 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55572.67 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 5557.27 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170089.051516 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50015.40 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 5001.54 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175090.591528 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45013.86 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 4501.39 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179591.977539 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40512.47 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 4051.25 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183643.224948 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36461.23 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3646.12 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187289.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32815.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190889.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29215.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194489.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25615.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198089.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22015.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201689.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18415.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205289.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14815.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208889.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11215.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212489.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7615.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216089.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4015.10 sec) 16:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219689.347617 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 415.10 sec) 16:33:00 [GPUGRID] [rr_sim] 2220104.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976355064.73G/439.78G) 16:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852725.36 16:33:00 [---] pick_jobs_to_run() start 16:33:00 [---] pick_jobs_to_run() end 16:33:00 [---] [rr_sim] end 16:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:33:00 [---] [cpu_sched_debug] preliminary job list: 16:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:33:00 [---] [cpu_sched_debug] final job list: 16:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:33:00 [---] [cpu_sched_debug] enforce_run_list: end 16:33:00 [---] choose_project(): 59580.000000 16:33:00 [WUProp@Home] doing RPC to get work 16:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 376019.09 cpu 2123252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123252.90 16:33:00 [---] Got 0 tasks 16:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:34:00 [---] [rr_sim] doing sim: CPU sched 16:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976327474.37G/439.78G) 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2220041.71 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 222004.17 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 222004.171475 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1998037.54 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 199803.75 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421807.925803 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798233.79 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 179823.38 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601631.304698 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618410.41 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 161841.04 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763472.345703 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456569.37 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 145656.94 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909129.282608 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310912.43 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 131091.24 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040220.525822 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179821.19 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 117982.12 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158202.644715 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061839.07 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 106183.91 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264386.551719 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955655.16 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 95565.52 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359952.068022 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860089.65 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 86008.96 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445961.032695 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774080.68 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 77408.07 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523369.100901 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696672.61 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 69667.26 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1593036.362286 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 627005.35 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 62700.54 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655736.897533 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564304.82 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 56430.48 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712167.379255 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507874.34 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 50787.43 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762954.812804 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457086.90 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 45708.69 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808663.502999 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411378.21 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 41137.82 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849801.324174 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370240.39 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 37024.04 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886825.363232 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333216.35 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 33321.64 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920146.998384 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299894.72 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 29989.47 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950136.470021 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269905.24 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 26990.52 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977126.994494 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242914.72 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 24291.47 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001418.466520 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218623.25 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 21862.32 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023280.791343 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196760.92 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 19676.09 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042956.883684 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177084.83 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 17708.48 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060665.366791 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159376.35 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 15937.63 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076603.001587 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143438.71 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 14343.87 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090946.872903 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129094.84 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 12909.48 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103856.357088 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116185.36 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 11618.54 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115474.892854 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104566.82 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 10456.68 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125931.575044 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94110.14 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 9411.01 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135342.589015 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84699.13 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 8469.91 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143812.501589 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76229.21 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 7622.92 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151435.422905 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68606.29 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 6860.63 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158296.052090 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61745.66 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 6174.57 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164470.618356 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55571.10 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 5557.11 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170027.727995 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50013.99 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 5001.40 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175029.126671 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45012.59 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 4501.26 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179530.385479 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40511.33 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 4051.13 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183581.518406 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36460.20 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3646.02 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187227.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32814.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190827.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29214.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194427.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25614.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198027.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22014.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201627.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18414.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205227.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14814.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208827.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11214.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212427.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7614.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216027.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4014.18 sec) 16:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219627.538041 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 414.18 sec) 16:34:00 [GPUGRID] [rr_sim] 2220041.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976327474.37G/439.78G) 16:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852722.63 16:34:00 [---] pick_jobs_to_run() start 16:34:00 [---] pick_jobs_to_run() end 16:34:00 [---] [rr_sim] end 16:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:34:00 [---] [cpu_sched_debug] preliminary job list: 16:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:34:00 [---] [cpu_sched_debug] final job list: 16:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:34:00 [---] [cpu_sched_debug] enforce_run_list: end 16:34:00 [---] choose_project(): 59640.000000 16:34:00 [WUProp@Home] doing RPC to get work 16:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375959.09 cpu 2123192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123192.90 16:34:00 [---] Got 0 tasks 16:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:35:00 [---] [rr_sim] doing sim: CPU sched 16:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976299884.01G/439.78G) 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219978.98 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 221997.90 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221997.897787 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997981.08 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 199798.11 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421796.005795 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798182.97 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 179818.30 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601614.303003 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618364.67 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 161836.47 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763450.770489 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456528.21 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 145652.82 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909103.591227 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310875.39 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 131087.54 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040191.129892 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179787.85 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 117978.78 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158169.914689 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061809.06 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 106180.91 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264350.821007 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955628.16 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 95562.82 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359913.636694 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860065.34 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 86006.53 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445920.170811 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774058.81 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 77405.88 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523326.051517 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696652.93 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 69665.29 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592991.344152 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626987.63 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 62698.76 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655690.107524 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564288.87 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 56428.89 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712118.994559 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507859.98 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 50786.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762904.992890 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457073.98 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 45707.40 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808612.391388 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411366.59 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 41136.66 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849749.050036 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370229.93 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 37022.99 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886772.042819 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333206.94 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 33320.69 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920092.736324 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299886.24 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 29988.62 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950081.360479 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269897.62 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 26989.76 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977071.122218 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242907.86 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 24290.79 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001361.907783 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218617.07 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 21861.71 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023223.614792 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196755.36 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 19675.54 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042899.151100 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177079.83 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 17707.98 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060607.133777 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159371.84 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 15937.18 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076544.318186 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143434.66 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 14343.47 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090887.784154 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129091.19 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 12909.12 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103796.903526 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116182.07 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 11618.21 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115415.110960 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104563.87 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 10456.39 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125871.497651 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94107.48 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 9410.75 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135282.245673 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84696.73 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 8469.67 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143751.918893 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76227.06 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 7622.71 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151374.624790 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68604.35 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 6860.44 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158235.060098 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61743.92 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 6174.39 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164409.451875 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55569.53 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 5556.95 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169966.404475 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50012.57 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 5001.26 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174967.661814 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45011.32 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 4501.13 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179468.793420 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40510.18 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 4051.02 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183519.811865 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36459.17 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3645.92 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187165.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32813.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190765.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29213.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194365.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25613.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197965.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22013.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201565.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18413.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205165.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14813.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208765.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11213.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212365.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7613.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215965.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4013.25 sec) 16:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219565.728465 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 413.25 sec) 16:35:00 [GPUGRID] [rr_sim] 2219978.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976299884.01G/439.78G) 16:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852719.89 16:35:00 [---] pick_jobs_to_run() start 16:35:00 [---] pick_jobs_to_run() end 16:35:00 [---] [rr_sim] end 16:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:35:00 [---] [cpu_sched_debug] preliminary job list: 16:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:35:00 [---] [cpu_sched_debug] final job list: 16:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:35:00 [---] [cpu_sched_debug] enforce_run_list: end 16:35:00 [---] choose_project(): 59700.000000 16:35:00 [WUProp@Home] doing RPC to get work 16:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375899.09 cpu 2123132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123132.90 16:35:00 [---] Got 0 tasks 16:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:36:00 [---] [rr_sim] doing sim: CPU sched 16:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976272293.66G/439.78G) 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219916.24 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 221991.62 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221991.624099 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997924.62 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 199792.46 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421784.085788 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798132.16 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 179813.22 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601597.301308 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618318.94 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 161831.89 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763429.195276 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456487.05 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 145648.70 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909077.899847 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310838.34 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 131083.83 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040161.733961 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179754.51 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 117975.45 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158137.184664 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061779.06 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 106177.91 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264315.090296 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955601.15 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 95560.12 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359875.205365 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860041.04 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 86004.10 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445879.308927 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774036.93 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 77403.69 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523283.002133 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696633.24 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 69663.32 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592946.326019 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626969.91 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 62696.99 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655643.317516 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564272.92 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 56427.29 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712070.609863 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507845.63 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 50784.56 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762855.172975 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457061.07 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 45706.11 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808561.279777 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411354.96 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 41135.50 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849696.775898 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370219.47 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 37021.95 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886718.722407 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333197.52 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 33319.75 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1920038.474265 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299877.77 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 29987.78 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1950026.250937 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269889.99 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 26989.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977015.249942 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242900.99 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 24290.10 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001305.349047 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218610.89 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 21861.09 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023166.438241 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196749.80 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 19674.98 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042841.418515 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177074.82 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 17707.48 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060548.900763 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159367.34 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 15936.73 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076485.634785 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143430.61 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 14343.06 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090828.695405 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129087.55 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 12908.75 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103737.449964 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116178.79 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 11617.88 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115355.329066 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104560.91 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 10456.09 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125811.420258 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94104.82 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 9410.48 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135221.902331 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84694.34 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 8469.43 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143691.336197 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76224.90 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 7622.49 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151313.826676 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68602.41 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 6860.24 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158174.068107 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61742.17 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 6174.22 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164348.285395 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55567.96 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 5556.80 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169905.080954 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50011.16 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 5001.12 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174906.196958 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45010.04 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 4501.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179407.201361 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40509.04 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 4050.90 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183458.105323 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36458.14 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3645.81 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187103.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32812.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190703.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29212.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194303.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25612.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197903.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22012.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201503.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18412.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205103.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14812.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208703.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11212.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212303.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7612.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215903.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4012.32 sec) 16:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219503.918890 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 412.32 sec) 16:36:00 [GPUGRID] [rr_sim] 2219916.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976272293.66G/439.78G) 16:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852717.15 16:36:00 [---] pick_jobs_to_run() start 16:36:00 [---] pick_jobs_to_run() end 16:36:00 [---] [rr_sim] end 16:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:36:00 [---] [cpu_sched_debug] preliminary job list: 16:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:36:00 [---] [cpu_sched_debug] final job list: 16:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:36:00 [---] [cpu_sched_debug] enforce_run_list: end 16:36:00 [---] choose_project(): 59760.000000 16:36:00 [WUProp@Home] doing RPC to get work 16:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375839.09 cpu 2123072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123072.90 16:36:00 [---] Got 0 tasks 16:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:37:00 [---] [rr_sim] doing sim: CPU sched 16:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976244703.30G/439.78G) 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219853.50 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 221985.35 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221985.350411 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997868.15 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 199786.82 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421772.165780 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798081.34 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 179808.13 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601580.299613 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618273.20 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 161827.32 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763407.620062 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456445.88 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 145644.59 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909052.208466 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310801.30 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 131080.13 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040132.338030 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179721.17 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 117972.12 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158104.454638 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061749.05 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 106174.90 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264279.359584 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955574.14 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 95557.41 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359836.774036 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 860016.73 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 86001.67 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445838.447043 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 774015.06 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 77401.51 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523239.952750 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696613.55 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 69661.36 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592901.307885 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626952.20 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 62695.22 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655596.527507 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564256.98 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 56425.70 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1712022.225167 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507831.28 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 50783.13 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762805.353061 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457048.15 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 45704.82 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808510.168165 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411343.34 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 41134.33 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849644.501759 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370209.00 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 37020.90 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886665.401994 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333188.10 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 33318.81 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919984.212205 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299869.29 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 29986.93 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949971.141395 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269882.36 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 26988.24 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976959.377666 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242894.13 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 24289.41 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001248.790310 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218604.71 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 21860.47 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023109.261690 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196744.24 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 19674.42 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042783.685931 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177069.82 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 17706.98 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060490.667749 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159362.84 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 15936.28 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076426.951384 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143426.55 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 14342.66 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090769.606656 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129083.90 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 12908.39 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103677.996401 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116175.51 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 11617.55 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115295.547172 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104557.96 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 10455.80 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125751.342865 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94102.16 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 9410.22 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135161.558989 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84691.95 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 8469.19 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143630.753501 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76222.75 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 7622.28 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151253.028561 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68600.48 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 6860.05 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158113.076116 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61740.43 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 6174.04 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164287.118915 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55566.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 5556.64 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169843.757434 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50009.75 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 5000.97 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174844.732101 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45008.77 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 4500.88 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179345.609301 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40507.89 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 4050.79 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183396.398782 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36457.11 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3645.71 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187042.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32811.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190642.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29211.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194242.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25611.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197842.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22011.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201442.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18411.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205042.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14811.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208642.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11211.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212242.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7611.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215842.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4011.39 sec) 16:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219442.109314 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411.39 sec) 16:37:00 [GPUGRID] [rr_sim] 2219853.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976244703.30G/439.78G) 16:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852714.42 16:37:00 [---] pick_jobs_to_run() start 16:37:00 [---] pick_jobs_to_run() end 16:37:00 [---] [rr_sim] end 16:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:37:00 [---] [cpu_sched_debug] preliminary job list: 16:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:37:00 [---] [cpu_sched_debug] final job list: 16:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:37:00 [---] [cpu_sched_debug] enforce_run_list: end 16:37:00 [---] choose_project(): 59820.000000 16:37:00 [WUProp@Home] doing RPC to get work 16:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375779.09 cpu 2123012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2123012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2123012.90 16:37:00 [---] Got 0 tasks 16:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:38:00 [---] [rr_sim] doing sim: CPU sched 16:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976217112.95G/439.78G) 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219790.77 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 221979.08 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221979.076722 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997811.69 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 199781.17 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421760.245772 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1798030.52 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 179803.05 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601563.297917 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618227.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 161822.75 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763386.044848 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456404.72 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 145640.47 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909026.517086 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310764.25 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 131076.43 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040102.942099 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179687.83 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 117968.78 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158071.724612 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061719.04 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 106171.90 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264243.628873 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955547.14 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 95554.71 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359798.342708 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859992.42 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 85999.24 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445797.585159 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773993.18 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 77399.32 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523196.903366 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696593.86 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 69659.39 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592856.289752 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626934.48 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 62693.45 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655549.737499 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564241.03 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 56424.10 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711973.840471 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507816.93 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 50781.69 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762755.533146 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457035.23 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 45703.52 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808459.056554 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411331.71 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 41133.17 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849592.227621 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370198.54 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 37019.85 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886612.081581 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333178.69 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 33317.87 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919929.950145 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299860.82 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 29986.08 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949916.031853 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269874.74 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 26987.47 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976903.505390 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242887.26 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 24288.73 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001192.231573 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218598.54 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 21859.85 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023052.085138 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196738.68 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 19673.87 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042725.953347 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177064.81 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 17706.48 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060432.434735 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159358.33 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 15935.83 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076368.267983 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143422.50 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 14342.25 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090710.517907 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129080.25 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 12908.02 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103618.542839 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116172.22 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 11617.22 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115235.765277 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104555.00 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 10455.50 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125691.265472 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94099.50 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 9409.95 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135101.215647 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84689.55 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 8468.96 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143570.170805 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76220.60 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 7622.06 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151192.230447 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68598.54 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 6859.85 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158052.084124 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61738.68 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 6173.87 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164225.952434 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55564.81 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 5556.48 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169782.433913 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50008.33 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 5000.83 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174783.267244 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45007.50 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 4500.75 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179284.017242 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40506.75 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 4050.67 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183334.692240 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36456.07 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3645.61 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186980.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32810.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190580.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29210.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194180.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25610.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197780.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22010.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201380.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18410.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204980.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14810.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208580.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11210.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212180.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7610.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215780.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4010.47 sec) 16:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219380.299738 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410.47 sec) 16:38:00 [GPUGRID] [rr_sim] 2219790.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976217112.95G/439.78G) 16:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852711.68 16:38:00 [---] pick_jobs_to_run() start 16:38:00 [---] pick_jobs_to_run() end 16:38:00 [---] [rr_sim] end 16:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:38:00 [---] [cpu_sched_debug] preliminary job list: 16:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:38:00 [---] [cpu_sched_debug] final job list: 16:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:38:00 [---] [cpu_sched_debug] enforce_run_list: end 16:38:00 [---] choose_project(): 59880.000000 16:38:00 [WUProp@Home] doing RPC to get work 16:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375719.09 cpu 2122952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122952.90 16:38:00 [---] Got 0 tasks 16:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:39:00 [---] [rr_sim] doing sim: CPU sched 16:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976189522.59G/439.78G) 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219728.03 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 221972.80 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221972.803034 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997755.23 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 199775.52 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421748.325765 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797979.70 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 179797.97 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601546.296222 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618181.73 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 161818.17 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763364.469634 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456363.56 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 145636.36 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 909000.825705 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310727.20 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 131072.72 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040073.546169 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179654.48 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 117965.45 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158038.994586 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061689.04 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 106168.90 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264207.898161 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955520.13 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 95552.01 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359759.911379 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859968.12 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 85996.81 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445756.723275 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773971.31 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 77397.13 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523153.853982 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696574.18 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 69657.42 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592811.271618 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626916.76 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 62691.68 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655502.947490 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564225.08 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 56422.51 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711925.455775 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507802.57 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 50780.26 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762705.713232 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457022.32 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 45702.23 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808407.944943 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411320.09 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 41132.01 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849539.953483 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370188.08 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 37018.81 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886558.761168 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333169.27 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 33316.93 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919875.688086 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299852.34 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 29985.23 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949860.922311 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269867.11 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 26986.71 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976847.633114 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242880.40 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 24288.04 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001135.672837 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218592.36 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 21859.24 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022994.908587 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196733.12 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 19673.31 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042668.220763 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177059.81 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 17705.98 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060374.201721 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159353.83 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 15935.38 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076309.584583 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143418.45 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 14341.84 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090651.429158 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129076.60 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 12907.66 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103559.089277 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116168.94 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 11616.89 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115175.983383 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104552.05 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 10455.20 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125631.188079 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94096.84 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 9409.68 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135040.872305 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84687.16 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 8468.72 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143509.588109 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76218.44 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 7621.84 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151131.432332 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68596.60 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 6859.66 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157991.092133 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61736.94 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 6173.69 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164164.785954 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55563.24 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 5556.32 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169721.110392 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50006.92 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 5000.69 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174721.802387 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45006.23 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 4500.62 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179222.425183 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40505.61 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 4050.56 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183272.985698 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36455.04 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3645.50 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186918.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32809.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190518.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29209.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194118.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25609.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197718.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22009.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201318.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18409.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204918.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14809.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208518.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11209.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212118.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7609.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215718.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4009.54 sec) 16:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219318.490163 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409.54 sec) 16:39:00 [GPUGRID] [rr_sim] 2219728.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976189522.59G/439.78G) 16:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852708.94 16:39:00 [---] pick_jobs_to_run() start 16:39:00 [---] pick_jobs_to_run() end 16:39:00 [---] [rr_sim] end 16:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:39:00 [---] [cpu_sched_debug] preliminary job list: 16:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:39:00 [---] [cpu_sched_debug] final job list: 16:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:39:00 [---] [cpu_sched_debug] enforce_run_list: end 16:39:00 [---] choose_project(): 59940.000000 16:39:00 [WUProp@Home] doing RPC to get work 16:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375659.09 cpu 2122892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122892.90 16:39:00 [---] Got 0 tasks 16:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:40:00 [---] [rr_sim] doing sim: CPU sched 16:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976161932.23G/439.78G) 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219665.29 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 221966.53 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221966.529346 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997698.76 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 199769.88 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421736.405757 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797928.89 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 179792.89 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601529.294527 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618136.00 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 161813.60 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763342.894421 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456322.40 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 145632.24 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908975.134324 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310690.16 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 131069.02 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040044.150238 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179621.14 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 117962.11 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1158006.264560 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061659.03 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 106165.90 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264172.167450 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955493.13 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 95549.31 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359721.480051 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859943.81 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 85994.38 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445715.861392 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773949.43 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 77394.94 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523110.804598 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696554.49 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 69655.45 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592766.253484 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626899.04 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 62689.90 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655456.157482 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564209.14 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 56420.91 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711877.071079 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507788.22 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 50778.82 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762655.893317 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 457009.40 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 45700.94 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808356.833332 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411308.46 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 41130.85 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849487.679344 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370177.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 37017.76 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886505.440756 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333159.85 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 33315.99 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919821.426026 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299843.87 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 29984.39 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949805.812769 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269859.48 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 26985.95 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976791.760838 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242873.53 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 24287.35 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001079.114100 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218586.18 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 21858.62 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022937.732036 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196727.56 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 19672.76 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042610.488178 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177054.81 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 17705.48 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060315.968707 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159349.32 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 15934.93 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076250.901182 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143414.39 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 14341.44 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090592.340409 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129072.95 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 12907.30 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103499.635714 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116165.66 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 11616.57 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115116.201489 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104549.09 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 10454.91 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125571.110686 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94094.18 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 9409.42 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134980.528963 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84684.76 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 8468.48 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143449.005413 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76216.29 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 7621.63 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151070.634217 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68594.66 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 6859.47 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157930.100141 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61735.19 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 6173.52 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164103.619473 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55561.67 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 5556.17 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169659.786872 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50005.51 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 5000.55 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174660.337531 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45004.96 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 4500.50 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179160.833123 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40504.46 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 4050.45 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183211.279157 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36454.01 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3645.40 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186856.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32808.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190456.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29208.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194056.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25608.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197656.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22008.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201256.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18408.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204856.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14808.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208456.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11208.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212056.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7608.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215656.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4008.61 sec) 16:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219256.680587 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408.61 sec) 16:40:00 [GPUGRID] [rr_sim] 2219665.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976161932.23G/439.78G) 16:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852706.21 16:40:00 [---] pick_jobs_to_run() start 16:40:00 [---] pick_jobs_to_run() end 16:40:00 [---] [rr_sim] end 16:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:40:00 [---] [cpu_sched_debug] preliminary job list: 16:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:40:00 [---] [cpu_sched_debug] final job list: 16:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:40:00 [---] [cpu_sched_debug] enforce_run_list: end 16:40:00 [---] choose_project(): 60000.000000 16:40:00 [WUProp@Home] doing RPC to get work 16:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375599.09 cpu 2122832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122832.90 16:40:00 [---] Got 0 tasks 16:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:41:00 [---] [rr_sim] doing sim: CPU sched 16:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976134341.88G/439.78G) 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219602.56 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 221960.26 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221960.255658 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997642.30 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 199764.23 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421724.485750 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797878.07 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 179787.81 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601512.292832 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618090.26 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 161809.03 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763321.319207 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456281.24 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 145628.12 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908949.442944 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310653.11 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 131065.31 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040014.754307 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179587.80 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 117958.78 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157973.534534 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061629.02 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 106162.90 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264136.436738 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955466.12 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 95546.61 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359683.048722 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859919.51 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 85991.95 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445674.999508 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773927.56 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 77392.76 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523067.755214 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696534.80 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 69653.48 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592721.235351 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626881.32 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 62688.13 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655409.367473 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564193.19 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 56419.32 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711828.686384 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507773.87 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 50777.39 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762606.073403 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456996.48 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 45699.65 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808305.721720 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411296.83 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 41129.68 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849435.405206 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370167.15 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 37016.72 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886452.120343 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333150.44 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 33315.04 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919767.163966 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299835.39 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 29983.54 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949750.703227 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269851.85 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 26985.19 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976735.888562 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242866.67 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 24286.67 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001022.555364 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218580.00 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 21858.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022880.555485 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196722.00 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 19672.20 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042552.755594 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177049.80 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 17704.98 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060257.735692 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159344.82 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 15934.48 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076192.217781 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143410.34 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 14341.03 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090533.251660 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129069.30 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 12906.93 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103440.182152 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116162.37 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 11616.24 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115056.419595 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104546.14 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 10454.61 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125511.033293 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94091.52 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 9409.15 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134920.185621 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84682.37 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 8468.24 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143388.422717 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76214.13 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 7621.41 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151009.836103 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68592.72 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 6859.27 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157869.108150 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61733.45 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 6173.34 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164042.452993 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55560.10 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 5556.01 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169598.463351 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50004.09 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 5000.41 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174598.872674 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45003.68 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 4500.37 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179099.241064 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40503.32 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 4050.33 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183149.572615 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36452.98 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3645.30 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186794.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32807.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190394.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29207.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193994.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25607.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197594.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22007.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201194.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18407.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204794.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14807.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208394.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11207.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211994.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7607.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215594.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4007.69 sec) 16:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219194.871011 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407.69 sec) 16:41:00 [GPUGRID] [rr_sim] 2219602.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976134341.88G/439.78G) 16:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852703.47 16:41:00 [---] pick_jobs_to_run() start 16:41:00 [---] pick_jobs_to_run() end 16:41:00 [---] [rr_sim] end 16:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:41:00 [---] [cpu_sched_debug] preliminary job list: 16:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:41:00 [---] [cpu_sched_debug] final job list: 16:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:41:00 [---] [cpu_sched_debug] enforce_run_list: end 16:41:00 [---] choose_project(): 60060.000000 16:41:00 [WUProp@Home] doing RPC to get work 16:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375539.09 cpu 2122772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122772.90 16:41:00 [---] Got 0 tasks 16:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:42:00 [---] [rr_sim] doing sim: CPU sched 16:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976134341.88G/439.78G) 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219602.56 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 221960.26 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221960.255658 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997642.30 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 199764.23 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421724.485750 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797878.07 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 179787.81 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601512.292832 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618090.26 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 161809.03 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763321.319207 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456281.24 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 145628.12 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908949.442944 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310653.11 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 131065.31 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1040014.754307 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179587.80 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 117958.78 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157973.534534 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061629.02 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 106162.90 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264136.436738 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955466.12 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 95546.61 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359683.048722 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859919.51 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 85991.95 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445674.999508 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773927.56 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 77392.76 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523067.755214 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696534.80 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 69653.48 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592721.235351 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626881.32 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 62688.13 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655409.367473 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564193.19 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 56419.32 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711828.686384 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507773.87 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 50777.39 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762606.073403 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456996.48 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 45699.65 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808305.721720 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411296.83 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 41129.68 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849435.405206 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370167.15 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 37016.72 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886452.120343 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333150.44 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 33315.04 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919767.163966 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299835.39 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 29983.54 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949750.703227 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269851.85 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 26985.19 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976735.888562 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242866.67 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 24286.67 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001022.555364 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218580.00 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 21858.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022880.555485 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196722.00 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 19672.20 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042552.755594 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177049.80 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 17704.98 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060257.735692 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159344.82 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 15934.48 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076192.217781 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143410.34 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 14341.03 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090533.251660 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129069.30 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 12906.93 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103440.182152 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116162.37 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 11616.24 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115056.419595 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104546.14 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 10454.61 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125511.033293 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94091.52 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 9409.15 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134920.185621 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84682.37 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 8468.24 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143388.422717 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76214.13 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 7621.41 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151009.836103 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68592.72 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 6859.27 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157869.108150 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61733.45 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 6173.34 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164042.452993 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55560.10 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 5556.01 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169598.463351 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50004.09 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 5000.41 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174598.872674 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45003.68 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 4500.37 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179099.241064 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40503.32 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 4050.33 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183149.572615 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36452.98 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3645.30 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186794.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32807.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190394.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29207.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193994.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25607.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197594.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22007.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201194.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18407.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204794.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14807.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208394.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11207.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211994.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7607.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215594.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4007.69 sec) 16:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219194.871011 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407.69 sec) 16:42:00 [GPUGRID] [rr_sim] 2219602.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976134341.88G/439.78G) 16:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852763.47 16:42:00 [---] pick_jobs_to_run() start 16:42:00 [---] pick_jobs_to_run() end 16:42:00 [---] [rr_sim] end 16:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:42:00 [---] [cpu_sched_debug] preliminary job list: 16:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:42:00 [---] [cpu_sched_debug] final job list: 16:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:42:00 [---] [cpu_sched_debug] enforce_run_list: end 16:42:00 [---] choose_project(): 60120.000000 16:42:00 [WUProp@Home] doing RPC to get work 16:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375479.09 cpu 2122772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122772.90 16:42:00 [---] Got 0 tasks 16:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:43:00 [---] [rr_sim] doing sim: CPU sched 16:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976106751.52G/439.78G) 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219539.82 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 221953.98 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221953.981969 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997585.84 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 199758.58 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421712.565742 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797827.25 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 179782.73 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601495.291137 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618044.53 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 161804.45 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763299.743993 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456240.08 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 145624.01 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908923.751563 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310616.07 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 131061.61 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039985.358376 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179554.46 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 117955.45 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157940.804508 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061599.02 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 106159.90 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264100.706027 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955439.11 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 95543.91 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359644.617394 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859895.20 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 85989.52 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445634.137624 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773905.68 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 77390.57 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523024.705831 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696515.11 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 69651.51 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592676.217217 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626863.60 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 62686.36 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655362.577465 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564177.24 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 56417.72 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711780.301688 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507759.52 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 50775.95 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762556.253488 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456983.57 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 45698.36 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808254.610109 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411285.21 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 41128.52 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849383.131068 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370156.69 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 37015.67 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886398.799930 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333141.02 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 33314.10 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919712.901907 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299826.92 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 29982.69 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949695.593686 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269844.23 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 26984.42 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976680.016286 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242859.80 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 24285.98 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000965.996627 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218573.82 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 21857.38 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022823.378934 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196716.44 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 19671.64 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042495.023010 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177044.80 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 17704.48 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060199.502678 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159340.32 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 15934.03 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076133.534380 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143406.29 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 14340.63 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090474.162912 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129065.66 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 12906.57 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103380.728590 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116159.09 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 11615.91 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114996.637700 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104543.18 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 10454.32 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125450.955900 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94088.86 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 9408.89 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134859.842279 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84679.98 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 8468.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143327.840021 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76211.98 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 7621.20 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150949.037988 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68590.78 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 6859.08 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157808.116159 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61731.70 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 6173.17 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163981.286512 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55558.53 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 5555.85 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169537.139831 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50002.68 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 5000.27 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174537.407817 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45002.41 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 4500.24 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179037.649005 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40502.17 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 4050.22 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183087.866074 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36451.95 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3645.20 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186733.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32806.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190333.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29206.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193933.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25606.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197533.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22006.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201133.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18406.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204733.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14806.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208333.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11206.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211933.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7606.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215533.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4006.76 sec) 16:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219133.061436 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406.76 sec) 16:43:00 [GPUGRID] [rr_sim] 2219539.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976106751.52G/439.78G) 16:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852760.73 16:43:00 [---] pick_jobs_to_run() start 16:43:00 [---] pick_jobs_to_run() end 16:43:00 [---] [rr_sim] end 16:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:43:00 [---] [cpu_sched_debug] preliminary job list: 16:43:00 [---] [cpu_sched_debug] final job list: 16:43:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 16:43:00 [---] [work_fetch] Request work fetch: CPUs idle 16:43:00 [---] [cpu_sched_debug] enforce_run_list: end 16:43:00 [---] choose_project(): 60180.000000 16:43:00 [WUProp@Home] doing RPC to get work 16:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375419.09 cpu 2122712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122712.90 16:43:00 [---] Got 0 tasks 16:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:44:00 [---] [rr_sim] doing sim: CPU sched 16:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976106751.52G/439.78G) 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219539.82 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 221953.98 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221953.981969 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997585.84 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 199758.58 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421712.565742 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797827.25 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 179782.73 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601495.291137 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1618044.53 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 161804.45 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763299.743993 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456240.08 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 145624.01 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908923.751563 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310616.07 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 131061.61 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039985.358376 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179554.46 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 117955.45 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157940.804508 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061599.02 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 106159.90 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264100.706027 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955439.11 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 95543.91 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359644.617394 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859895.20 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 85989.52 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445634.137624 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773905.68 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 77390.57 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1523024.705831 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696515.11 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 69651.51 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592676.217217 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626863.60 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 62686.36 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655362.577465 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564177.24 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 56417.72 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711780.301688 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507759.52 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 50775.95 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762556.253488 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456983.57 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 45698.36 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808254.610109 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411285.21 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 41128.52 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849383.131068 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370156.69 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 37015.67 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886398.799930 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333141.02 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 33314.10 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919712.901907 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299826.92 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 29982.69 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949695.593686 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269844.23 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 26984.42 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976680.016286 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242859.80 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 24285.98 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000965.996627 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218573.82 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 21857.38 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022823.378934 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196716.44 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 19671.64 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042495.023010 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177044.80 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 17704.48 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060199.502678 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159340.32 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 15934.03 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076133.534380 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143406.29 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 14340.63 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090474.162912 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129065.66 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 12906.57 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103380.728590 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116159.09 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 11615.91 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114996.637700 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104543.18 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 10454.32 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125450.955900 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94088.86 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 9408.89 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134859.842279 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84679.98 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 8468.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143327.840021 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76211.98 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 7621.20 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150949.037988 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68590.78 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 6859.08 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157808.116159 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61731.70 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 6173.17 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163981.286512 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55558.53 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 5555.85 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169537.139831 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50002.68 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 5000.27 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174537.407817 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45002.41 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 4500.24 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179037.649005 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40502.17 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 4050.22 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183087.866074 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36451.95 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3645.20 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186733.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32806.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190333.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29206.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193933.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25606.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197533.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22006.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201133.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18406.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204733.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14806.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208333.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11206.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211933.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7606.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215533.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4006.76 sec) 16:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219133.061436 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406.76 sec) 16:44:00 [GPUGRID] [rr_sim] 2219539.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976106751.52G/439.78G) 16:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852820.73 16:44:00 [---] pick_jobs_to_run() start 16:44:00 [---] pick_jobs_to_run() end 16:44:00 [---] [rr_sim] end 16:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:44:00 [---] [cpu_sched_debug] preliminary job list: 16:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:44:00 [---] [cpu_sched_debug] final job list: 16:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:44:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 976106751.52G 16:44:00 [---] [cpu_sched_debug] enforce_run_list: end 16:44:00 [---] choose_project(): 60240.000000 16:44:00 [WUProp@Home] doing RPC to get work 16:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375359.09 cpu 2122712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122712.90 16:44:00 [---] Got 0 tasks 16:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:45:00 [---] [rr_sim] doing sim: CPU sched 16:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976079161.16G/439.78G) 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219477.08 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 221947.71 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221947.708281 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997529.37 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 199752.94 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421700.645734 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797776.44 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 179777.64 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601478.289442 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617998.79 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 161799.88 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763278.168779 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456198.91 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 145619.89 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908898.060182 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310579.02 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 131057.90 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039955.962445 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179521.12 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 117952.11 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157908.074482 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061569.01 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 106156.90 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264064.975315 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955412.11 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 95541.21 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359606.186065 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859870.90 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 85987.09 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445593.275740 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773883.81 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 77388.38 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522981.656447 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696495.43 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 69649.54 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592631.199083 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626845.88 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 62684.59 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655315.787456 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564161.30 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 56416.13 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711731.916992 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507745.17 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 50774.52 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762506.433574 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456970.65 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 45697.06 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808203.498498 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411273.58 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 41127.36 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849330.856929 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370146.23 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 37014.62 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886345.479518 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333131.60 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 33313.16 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919658.639847 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299818.44 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 29981.84 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949640.484144 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269836.60 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 26983.66 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976624.144010 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242852.94 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 24285.29 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000909.437891 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218567.64 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 21856.76 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022766.202383 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196710.88 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 19671.09 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042437.290426 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177039.79 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 17703.98 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060141.269664 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159335.81 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 15933.58 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076074.850979 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143402.23 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 14340.22 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090415.074163 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129062.01 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 12906.20 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103321.275028 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116155.81 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 11615.58 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114936.855806 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104540.23 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 10454.02 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125390.878507 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94086.20 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 9408.62 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134799.498937 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84677.58 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 8467.76 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143267.257325 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76209.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 7620.98 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150888.239874 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68588.84 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 6858.88 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157747.124167 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61729.96 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 6173.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163920.120032 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55556.96 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 5555.70 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169475.816310 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50001.27 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 5000.13 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174475.942960 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45001.14 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 4500.11 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178976.056945 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40501.03 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 4050.10 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183026.159532 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36450.92 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3645.09 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186671.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32805.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190271.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29205.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193871.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25605.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197471.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22005.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201071.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18405.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204671.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14805.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208271.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11205.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211871.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7605.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215471.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4005.83 sec) 16:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219071.251860 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 405.83 sec) 16:45:00 [GPUGRID] [rr_sim] 2219477.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976079161.16G/439.78G) 16:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852818.00 16:45:00 [---] pick_jobs_to_run() start 16:45:00 [---] pick_jobs_to_run() end 16:45:00 [---] [rr_sim] end 16:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:45:00 [---] [cpu_sched_debug] preliminary job list: 16:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 16:45:00 [---] [cpu_sched_debug] final job list: 16:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:45:00 [---] [cpu_sched_debug] enforce_run_list: end 16:45:00 [---] choose_project(): 60300.000000 16:45:00 [WUProp@Home] doing RPC to get work 16:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375299.09 cpu 2122652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122652.90 16:45:00 [---] Got 0 tasks 16:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:46:00 [---] [rr_sim] doing sim: CPU sched 16:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976051570.81G/439.78G) 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219414.35 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 221941.43 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221941.434593 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997472.91 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 199747.29 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421688.725727 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797725.62 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 179772.56 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601461.287747 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617953.06 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 161795.31 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763256.593565 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456157.75 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 145615.78 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908872.368802 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310541.98 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 131054.20 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039926.566515 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179487.78 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 117948.78 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157875.344456 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061539.00 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 106153.90 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1264029.244604 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955385.10 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 95538.51 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359567.754736 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859846.59 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 85984.66 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445552.413856 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773861.93 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 77386.19 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522938.607063 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696475.74 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 69647.57 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592586.180950 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626828.16 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 62682.82 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655268.997448 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564145.35 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 56414.53 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711683.532296 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507730.81 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 50773.08 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762456.613660 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456957.73 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 45695.77 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808152.386887 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411261.96 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 41126.20 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849278.582791 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370135.76 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 37013.58 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886292.159105 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333122.19 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 33312.22 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919604.377787 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299809.97 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 29981.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949585.374602 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269828.97 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 26982.90 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976568.271735 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242846.07 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 24284.61 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000852.879154 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218561.47 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 21856.15 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022709.025832 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196705.32 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 19670.53 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042379.557842 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177034.79 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 17703.48 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060083.036650 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159331.31 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 15933.13 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076016.167578 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143398.18 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 14339.82 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090355.985414 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129058.36 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 12905.84 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103261.821465 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116152.52 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 11615.25 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114877.073912 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104537.27 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 10453.73 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125330.801114 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94083.54 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 9408.35 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134739.155595 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84675.19 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 8467.52 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143206.674629 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76207.67 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 7620.77 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150827.441759 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68586.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 6858.69 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157686.132176 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61728.21 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 6172.82 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163858.953551 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55555.39 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 5555.54 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169414.492789 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49999.85 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 4999.99 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174414.478103 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44999.87 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 4499.99 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178914.464886 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40499.88 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 4049.99 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182964.452991 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36449.89 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3644.99 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186609.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32804.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190209.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29204.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193809.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25604.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197409.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22004.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201009.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18404.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204609.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14804.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208209.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11204.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211809.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7604.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215409.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4004.90 sec) 16:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2219009.442284 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 404.90 sec) 16:46:00 [GPUGRID] [rr_sim] 2219414.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976051570.81G/439.78G) 16:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852815.26 16:46:00 [---] pick_jobs_to_run() start 16:46:00 [---] pick_jobs_to_run() end 16:46:00 [---] [rr_sim] end 16:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:46:00 [---] [cpu_sched_debug] preliminary job list: 16:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:46:00 [---] [cpu_sched_debug] final job list: 16:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:46:00 [---] [cpu_sched_debug] enforce_run_list: end 16:46:00 [---] choose_project(): 60360.000000 16:46:00 [WUProp@Home] doing RPC to get work 16:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375239.09 cpu 2122592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122592.90 16:46:00 [---] Got 0 tasks 16:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:47:00 [---] [rr_sim] doing sim: CPU sched 16:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (976023980.45G/439.78G) 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219351.61 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 221935.16 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221935.160905 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997416.45 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 199741.64 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421676.805719 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797674.80 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 179767.48 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601444.286052 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617907.32 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 161790.73 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763235.018352 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456116.59 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 145611.66 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908846.677421 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310504.93 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 131050.49 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039897.170584 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179454.44 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 117945.44 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157842.614430 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061508.99 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 106150.90 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263993.513892 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955358.10 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 95535.81 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359529.323408 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859822.29 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 85982.23 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445511.551972 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773840.06 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 77384.01 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522895.557679 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696456.05 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 69645.61 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592541.162816 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626810.45 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 62681.04 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655222.207439 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564129.40 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 56412.94 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711635.147600 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507716.46 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 50771.65 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762406.793745 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456944.82 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 45694.48 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808101.275275 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411250.33 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 41125.03 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849226.308653 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370125.30 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 37012.53 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886238.838692 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333112.77 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 33311.28 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919550.115728 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299801.49 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 29980.15 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949530.265060 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269821.34 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 26982.13 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976512.399459 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242839.21 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 24283.92 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000796.320418 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218555.29 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 21855.53 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022651.849281 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196699.76 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 19669.98 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042321.825257 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177029.78 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 17702.98 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060024.803636 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159326.81 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 15932.68 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075957.484178 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143394.12 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 14339.41 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090296.896665 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129054.71 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 12905.47 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103202.367903 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116149.24 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 11614.92 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114817.292017 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104534.32 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 10453.43 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125270.723721 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94080.89 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 9408.09 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134678.812253 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84672.80 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 8467.28 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143146.091933 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76205.52 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 7620.55 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150766.643644 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68584.97 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 6858.50 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157625.140185 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61726.47 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 6172.65 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163797.787071 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55553.82 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 5555.38 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169353.169269 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49998.44 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 4999.84 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174353.013247 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44998.60 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 4499.86 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178852.872827 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40498.74 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 4049.87 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182902.746449 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36448.86 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3644.89 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186547.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32803.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190147.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29203.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193747.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25603.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197347.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22003.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200947.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18403.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204547.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14803.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208147.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11203.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211747.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7603.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215347.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4003.98 sec) 16:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218947.632709 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 403.98 sec) 16:47:00 [GPUGRID] [rr_sim] 2219351.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (976023980.45G/439.78G) 16:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852812.52 16:47:00 [---] pick_jobs_to_run() start 16:47:00 [---] pick_jobs_to_run() end 16:47:00 [---] [rr_sim] end 16:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:47:00 [---] [cpu_sched_debug] preliminary job list: 16:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:47:00 [---] [cpu_sched_debug] final job list: 16:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:47:00 [---] [cpu_sched_debug] enforce_run_list: end 16:47:00 [---] choose_project(): 60420.000000 16:47:00 [WUProp@Home] doing RPC to get work 16:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375179.09 cpu 2122532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122532.90 16:47:00 [---] Got 0 tasks 16:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:48:00 [---] [rr_sim] doing sim: CPU sched 16:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975996390.10G/439.78G) 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219288.87 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 221928.89 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221928.887217 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997359.98 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 199736.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421664.885712 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797623.99 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 179762.40 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601427.284357 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617861.59 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 161786.16 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763213.443138 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456075.43 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 145607.54 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908820.986041 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310467.89 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 131046.79 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039867.774653 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179421.10 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 117942.11 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157809.884404 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061478.99 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 106147.90 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263957.783181 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955331.09 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 95533.11 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359490.892079 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859797.98 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 85979.80 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445470.690088 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773818.18 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 77381.82 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522852.508296 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696436.36 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 69643.64 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592496.144683 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626792.73 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 62679.27 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655175.417431 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564113.45 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 56411.35 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711586.762904 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507702.11 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 50770.21 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762356.973831 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456931.90 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 45693.19 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1808050.163664 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411238.71 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 41123.87 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849174.034514 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370114.84 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 37011.48 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886185.518279 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333103.35 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 33310.34 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919495.853668 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299793.02 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 29979.30 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949475.155518 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269813.72 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 26981.37 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976456.527183 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242832.34 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 24283.23 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000739.761681 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218549.11 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 21854.91 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022594.672729 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196694.20 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 19669.42 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042264.092673 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177024.78 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 17702.48 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059966.570622 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159322.30 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 15932.23 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075898.800777 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143390.07 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 14339.01 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090237.807916 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129051.06 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 12905.11 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103142.914341 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116145.96 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 11614.60 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114757.510123 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104531.36 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 10453.14 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125210.646327 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94078.23 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 9407.82 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134618.468911 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84670.40 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 8467.04 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143085.509237 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76203.36 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 7620.34 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150705.845530 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68583.03 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 6858.30 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157564.148193 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61724.72 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 6172.47 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163736.620591 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55552.25 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 5555.23 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169291.845748 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49997.03 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 4999.70 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174291.548390 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44997.32 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 4499.73 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178791.280767 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40497.59 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 4049.76 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182841.039907 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36447.83 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3644.78 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186485.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32803.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190085.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29203.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193685.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25603.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197285.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22003.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200885.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18403.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204485.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14803.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208085.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11203.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211685.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7603.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215285.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4003.05 sec) 16:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218885.823133 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 403.05 sec) 16:48:00 [GPUGRID] [rr_sim] 2219288.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975996390.10G/439.78G) 16:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852809.78 16:48:00 [---] pick_jobs_to_run() start 16:48:00 [---] pick_jobs_to_run() end 16:48:00 [---] [rr_sim] end 16:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:48:00 [---] [cpu_sched_debug] preliminary job list: 16:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:48:00 [---] [cpu_sched_debug] final job list: 16:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:48:00 [---] [cpu_sched_debug] enforce_run_list: end 16:48:00 [---] choose_project(): 60480.000000 16:48:00 [WUProp@Home] doing RPC to get work 16:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375119.09 cpu 2122472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122472.90 16:48:00 [---] Got 0 tasks 16:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:49:00 [---] [rr_sim] doing sim: CPU sched 16:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975968799.74G/439.78G) 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219226.14 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 221922.61 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221922.613528 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997303.52 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 199730.35 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421652.965704 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797573.17 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 179757.32 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601410.282662 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617815.85 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 161781.59 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763191.867924 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1456034.27 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 145603.43 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908795.294660 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310430.84 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 131043.08 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039838.378722 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179387.76 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 117938.78 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157777.154379 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061448.98 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 106144.90 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263922.052469 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955304.08 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 95530.41 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359452.460751 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859773.67 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 85977.37 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445429.828204 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773796.31 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 77379.63 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522809.458912 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696416.68 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 69641.67 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592451.126549 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626775.01 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 62677.50 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655128.627422 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564097.51 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 56409.75 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711538.378209 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507687.76 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 50768.78 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762307.153916 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456918.98 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 45691.90 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807999.052053 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411227.08 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 41122.71 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849121.760376 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370104.37 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 37010.44 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886132.197867 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333093.94 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 33309.39 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919441.591608 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299784.54 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 29978.45 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949420.045976 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269806.09 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 26980.61 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976400.654907 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242825.48 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 24282.55 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000683.202944 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218542.93 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 21854.29 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022537.496178 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196688.64 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 19668.86 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042206.360089 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177019.78 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 17701.98 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059908.337608 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159317.80 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 15931.78 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075840.117376 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143386.02 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 14338.60 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090178.719167 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129047.42 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 12904.74 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103083.460778 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116142.67 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 11614.27 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114697.728229 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104528.41 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 10452.84 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125150.568934 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94075.57 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 9407.56 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134558.125569 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84668.01 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 8466.80 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143024.926541 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76201.21 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 7620.12 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150645.047415 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68581.09 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 6858.11 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157503.156202 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61722.98 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 6172.30 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163675.454110 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55550.68 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 5555.07 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169230.522227 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49995.61 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 4999.56 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174230.083533 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44996.05 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 4499.61 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178729.688708 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40496.45 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 4049.64 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182779.333366 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36446.80 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3644.68 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186424.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32802.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190024.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29202.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193624.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25602.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197224.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22002.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200824.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18402.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204424.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14802.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208024.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11202.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211624.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7602.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215224.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4002.12 sec) 16:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218824.013558 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 402.12 sec) 16:49:00 [GPUGRID] [rr_sim] 2219226.14: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975968799.74G/439.78G) 16:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852807.05 16:49:00 [---] pick_jobs_to_run() start 16:49:00 [---] pick_jobs_to_run() end 16:49:00 [---] [rr_sim] end 16:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:49:00 [---] [cpu_sched_debug] preliminary job list: 16:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:49:00 [---] [cpu_sched_debug] final job list: 16:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:49:00 [---] [cpu_sched_debug] enforce_run_list: end 16:49:00 [---] choose_project(): 60540.000000 16:49:00 [WUProp@Home] doing RPC to get work 16:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 375059.09 cpu 2122412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122412.90 16:49:00 [---] Got 0 tasks 16:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:50:00 [---] [rr_sim] doing sim: CPU sched 16:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975941209.38G/439.78G) 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219163.40 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 221916.34 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221916.339840 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997247.06 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 199724.71 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421641.045696 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797522.35 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 179752.24 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601393.280967 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617770.12 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 161777.01 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763170.292710 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455993.11 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 145599.31 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908769.603279 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310393.80 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 131039.38 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039808.982792 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179354.42 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 117935.44 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157744.424353 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061418.97 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 106141.90 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263886.321758 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955277.08 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 95527.71 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359414.029422 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859749.37 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 85974.94 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445388.966320 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773774.43 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 77377.44 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522766.409528 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696396.99 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 69639.70 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592406.108415 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626757.29 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 62675.73 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655081.837414 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564081.56 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 56408.16 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711489.993513 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507673.40 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 50767.34 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762257.334002 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456906.06 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 45690.61 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807947.940442 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411215.46 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 41121.55 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849069.486238 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370093.91 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 37009.39 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886078.877454 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333084.52 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 33308.45 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919387.329549 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299776.07 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 29977.61 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949364.936434 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269798.46 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 26979.85 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976344.782631 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242818.62 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 24281.86 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000626.644208 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218536.75 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 21853.68 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022480.319627 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196683.08 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 19668.31 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042148.627505 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177014.77 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 17701.48 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059850.104594 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159313.29 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 15931.33 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075781.433975 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143381.96 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 14338.20 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090119.630418 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129043.77 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 12904.38 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103024.007216 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116139.39 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 11613.94 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114637.946335 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104525.45 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 10452.55 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125090.491541 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94072.91 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 9407.29 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134497.782227 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84665.62 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 8466.56 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142964.343845 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76199.05 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 7619.91 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150584.249300 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68579.15 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 6857.91 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157442.164211 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61721.23 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 6172.12 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163614.287630 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55549.11 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 5554.91 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169169.198707 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49994.20 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 4999.42 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174168.618676 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44994.78 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 4499.48 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178668.096649 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40495.30 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 4049.53 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182717.626824 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36445.77 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3644.58 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186362.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32801.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189962.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29201.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193562.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25601.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197162.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22001.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200762.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18401.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204362.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14801.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207962.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11201.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211562.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7601.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215162.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4001.19 sec) 16:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218762.203982 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 401.19 sec) 16:50:00 [GPUGRID] [rr_sim] 2219163.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975941209.38G/439.78G) 16:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852804.31 16:50:00 [---] pick_jobs_to_run() start 16:50:00 [---] pick_jobs_to_run() end 16:50:00 [---] [rr_sim] end 16:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:50:00 [---] [cpu_sched_debug] preliminary job list: 16:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:50:00 [---] [cpu_sched_debug] final job list: 16:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:50:00 [---] [cpu_sched_debug] enforce_run_list: end 16:50:00 [---] choose_project(): 60600.000000 16:50:00 [WUProp@Home] doing RPC to get work 16:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374999.09 cpu 2122352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122352.90 16:50:00 [---] Got 0 tasks 16:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:51:00 [---] [rr_sim] doing sim: CPU sched 16:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975913619.03G/439.78G) 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219100.66 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 221910.07 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221910.066152 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997190.60 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 199719.06 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421629.125689 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797471.54 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 179747.15 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601376.279272 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617724.38 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 161772.44 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763148.717497 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455951.94 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 145595.19 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908743.911899 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310356.75 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 131035.67 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039779.586861 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179321.07 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 117932.11 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157711.694327 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061388.97 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 106138.90 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263850.591046 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955250.07 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 95525.01 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359375.598093 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859725.06 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 85972.51 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445348.104436 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773752.56 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 77375.26 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522723.360144 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696377.30 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 69637.73 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592361.090282 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626739.57 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 62673.96 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1655035.047406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564065.61 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 56406.56 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711441.608817 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507659.05 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 50765.91 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762207.514087 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456893.15 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 45689.31 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807896.828830 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411203.83 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 41120.38 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1849017.212099 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370083.45 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 37008.34 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1886025.557041 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333075.10 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 33307.51 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919333.067489 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299767.59 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 29976.76 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949309.826892 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269790.83 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 26979.08 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976288.910355 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242811.75 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 24281.18 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000570.085471 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218530.58 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 21853.06 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022423.143076 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196677.52 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 19667.75 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042090.894920 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177009.77 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 17700.98 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059791.871580 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159308.79 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 15930.88 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075722.750574 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143377.91 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 14337.79 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090060.541669 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129040.12 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 12904.01 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102964.553654 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116136.11 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 11613.61 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114578.164440 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104522.50 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 10452.25 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125030.414148 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94070.25 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 9407.02 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134437.438885 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84663.22 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 8466.32 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142903.761149 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76196.90 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 7619.69 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150523.451186 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68577.21 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 6857.72 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157381.172219 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61719.49 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 6171.95 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163553.121149 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55547.54 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 5554.75 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169107.875186 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49992.79 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 4999.28 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174107.153820 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44993.51 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 4499.35 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178606.504590 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40494.16 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 4049.42 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182655.920283 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36444.74 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3644.47 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186300.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32800.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189900.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29200.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193500.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25600.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197100.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22000.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200700.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18400.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204300.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14800.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207900.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11200.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211500.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7600.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215100.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4000.27 sec) 16:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218700.394406 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 400.27 sec) 16:51:00 [GPUGRID] [rr_sim] 2219100.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975913619.03G/439.78G) 16:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852801.57 16:51:00 [---] pick_jobs_to_run() start 16:51:00 [---] pick_jobs_to_run() end 16:51:00 [---] [rr_sim] end 16:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:51:00 [---] [cpu_sched_debug] preliminary job list: 16:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:51:00 [---] [cpu_sched_debug] final job list: 16:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:51:00 [---] [cpu_sched_debug] enforce_run_list: end 16:51:00 [---] choose_project(): 60660.000000 16:51:00 [WUProp@Home] doing RPC to get work 16:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374939.09 cpu 2122292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122292.90 16:51:00 [---] Got 0 tasks 16:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:52:00 [---] [rr_sim] doing sim: CPU sched 16:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975886028.67G/439.78G) 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2219037.92 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 221903.79 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221903.792464 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997134.13 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 199713.41 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421617.205681 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797420.72 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 179742.07 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601359.277577 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617678.65 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 161767.86 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763127.142283 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455910.78 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 145591.08 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908718.220518 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310319.70 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 131031.97 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039750.190930 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179287.73 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 117928.77 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157678.964301 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061358.96 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 106135.90 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263814.860334 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955223.06 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 95522.31 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359337.166765 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859700.76 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 85970.08 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445307.242552 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773730.68 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 77373.07 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522680.310760 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696357.61 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 69635.76 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592316.072148 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626721.85 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 62672.19 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654988.257397 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564049.67 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 56404.97 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711393.224121 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507644.70 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 50764.47 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762157.694173 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456880.23 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 45688.02 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807845.717219 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411192.21 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 41119.22 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848964.937961 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370072.99 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 37007.30 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885972.236629 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333065.69 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 33306.57 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919278.805429 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299759.12 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 29975.91 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949254.717350 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269783.21 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 26978.32 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976233.038079 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242804.89 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 24280.49 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000513.526735 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218524.40 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 21852.44 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022365.966525 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196671.96 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 19667.20 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042033.162336 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177004.76 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 17700.48 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059733.638566 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159304.29 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 15930.43 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075664.067173 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143373.86 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 14337.39 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090001.452920 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129036.47 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 12903.65 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102905.100092 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116132.82 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 11613.28 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114518.382546 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104519.54 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 10451.95 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124970.336755 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94067.59 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 9406.76 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134377.095543 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84660.83 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 8466.08 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142843.178453 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76194.75 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 7619.47 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150462.653071 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68575.27 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 6857.53 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157320.180228 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61717.74 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 6171.77 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163491.954669 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55545.97 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 5554.60 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169046.551666 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49991.37 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 4999.14 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174045.688963 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44992.24 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 4499.22 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178544.912530 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40493.01 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 4049.30 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182594.213741 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36443.71 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3644.37 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186238.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32799.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189838.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29199.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193438.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25599.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197038.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21999.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200638.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18399.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204238.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14799.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207838.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11199.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211438.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7599.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215038.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3999.34 sec) 16:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218638.584831 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 399.34 sec) 16:52:00 [GPUGRID] [rr_sim] 2219037.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975886028.67G/439.78G) 16:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852798.84 16:52:00 [---] pick_jobs_to_run() start 16:52:00 [---] pick_jobs_to_run() end 16:52:00 [---] [rr_sim] end 16:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:52:00 [---] [cpu_sched_debug] preliminary job list: 16:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:52:00 [---] [cpu_sched_debug] final job list: 16:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:52:00 [---] [cpu_sched_debug] enforce_run_list: end 16:52:00 [---] choose_project(): 60720.000000 16:52:00 [WUProp@Home] doing RPC to get work 16:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374879.09 cpu 2122232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122232.90 16:52:00 [---] Got 0 tasks 16:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 16:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 16:53:00 [---] [rr_sim] doing sim: CPU sched 16:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975858438.32G/439.78G) 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218975.19 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 221897.52 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221897.518776 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997077.67 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 199707.77 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421605.285673 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797369.90 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 179736.99 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601342.275882 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617632.91 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 161763.29 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763105.567069 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455869.62 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 145586.96 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908692.529138 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310282.66 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 131028.27 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039720.794999 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179254.39 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 117925.44 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157646.234275 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061328.95 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 106132.90 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263779.129623 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955196.06 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 95519.61 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359298.735436 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859676.45 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 85967.65 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445266.380668 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773708.81 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 77370.88 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522637.261377 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696337.93 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 69633.79 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592271.054015 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626704.13 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 62670.41 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654941.467389 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564033.72 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 56403.37 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711344.839425 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507630.35 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 50763.03 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762107.874258 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456867.31 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 45686.73 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807794.605608 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411180.58 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 41118.06 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848912.663823 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370062.52 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 37006.25 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885918.916216 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333056.27 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 33305.63 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919224.543370 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299750.64 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 29975.06 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949199.607808 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269775.58 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 26977.56 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976177.165803 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242798.02 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 24279.80 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000456.967998 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218518.22 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 21851.82 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022308.789974 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196666.40 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 19666.64 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041975.429752 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176999.76 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 17699.98 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059675.405552 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159299.78 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 15929.98 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075605.383773 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143369.80 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 14336.98 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089942.364171 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129032.82 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 12903.28 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102845.646529 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116129.54 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 11612.95 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114458.600652 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104516.59 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 10451.66 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124910.259362 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94064.93 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 9406.49 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134316.752201 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84658.44 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 8465.84 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142782.595757 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76192.59 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 7619.26 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150401.854957 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68573.33 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 6857.33 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157259.188236 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61716.00 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 6171.60 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163430.788188 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55544.40 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 5554.44 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168985.228145 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49989.96 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 4999.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173984.224106 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44990.96 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 4499.10 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178483.320471 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40491.87 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 4049.19 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182532.507199 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36442.68 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3644.27 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186176.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32798.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189776.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29198.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193376.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25598.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196976.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21998.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200576.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18398.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204176.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14798.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207776.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11198.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211376.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7598.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214976.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3998.41 sec) 16:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218576.775255 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 398.41 sec) 16:53:00 [GPUGRID] [rr_sim] 2218975.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975858438.32G/439.78G) 16:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1852796.10 16:53:00 [---] pick_jobs_to_run() start 16:53:00 [---] pick_jobs_to_run() end 16:53:00 [---] [rr_sim] end 16:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 16:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 16:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 16:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 16:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 16:53:00 [---] [cpu_sched_debug] preliminary job list: 16:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:53:00 [---] [cpu_sched_debug] final job list: 16:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 16:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 16:53:00 [---] [cpu_sched_debug] enforce_run_list: end 16:53:00 [---] choose_project(): 60780.000000 16:53:00 [WUProp@Home] doing RPC to get work 16:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374819.09 cpu 2122172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122172.90 16:53:00 [---] Got 0 tasks 17:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:00:00 [---] [rr_sim] doing sim: CPU sched 17:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975830847.96G/439.78G) 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218912.45 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 221891.25 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221891.245087 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1997021.21 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 199702.12 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421593.365666 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797319.09 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 179731.91 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601325.274187 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617587.18 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 161758.72 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763083.991855 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455828.46 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 145582.85 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908666.837757 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310245.61 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 131024.56 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039691.399069 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179221.05 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 117922.11 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157613.504249 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061298.95 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 106129.89 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263743.398911 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955169.05 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 95516.91 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359260.304108 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859652.15 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 85965.21 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445225.518784 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773686.93 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 77368.69 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522594.211993 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696318.24 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 69631.82 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592226.035881 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626686.41 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 62668.64 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654894.677380 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564017.77 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 56401.78 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711296.454729 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507616.00 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 50761.60 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762058.054344 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456854.40 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 45685.44 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807743.493997 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411168.96 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 41116.90 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848860.389684 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370052.06 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 37005.21 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885865.595803 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333046.86 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 33304.69 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919170.281310 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299742.17 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 29974.22 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949144.498266 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269767.95 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 26976.80 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976121.293527 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242791.16 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 24279.12 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000400.409262 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218512.04 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 21851.20 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022251.613423 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196660.84 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 19666.08 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041917.697168 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176994.75 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 17699.48 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059617.172538 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159295.28 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 15929.53 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075546.700372 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143365.75 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 14336.58 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089883.275422 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129029.18 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 12902.92 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102786.192967 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116126.26 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 11612.63 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114398.818758 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104513.63 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 10451.36 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124850.181969 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94062.27 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 9406.23 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134256.408859 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84656.04 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 8465.60 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142722.013061 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76190.44 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 7619.04 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150341.056842 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68571.39 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 6857.14 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157198.196245 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61714.25 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 6171.43 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163369.621708 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55542.83 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 5554.28 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168923.904624 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49988.55 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 4998.85 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173922.759249 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44989.69 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 4498.97 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178421.728412 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40490.72 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 4049.07 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182470.800658 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36441.65 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3644.17 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186114.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32797.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189714.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29197.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193314.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25597.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196914.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21997.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200514.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18397.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204114.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14797.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207714.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11197.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211314.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7597.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214914.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3997.49 sec) 17:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218514.965679 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 397.49 sec) 17:00:00 [GPUGRID] [rr_sim] 2218912.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975830847.96G/439.78G) 17:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853153.36 17:00:00 [---] pick_jobs_to_run() start 17:00:00 [---] pick_jobs_to_run() end 17:00:00 [---] [rr_sim] end 17:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:00:00 [---] [cpu_sched_debug] preliminary job list: 17:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:00:00 [---] [cpu_sched_debug] final job list: 17:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:00:00 [---] [cpu_sched_debug] enforce_run_list: end 17:00:00 [---] choose_project(): 61200.000000 17:00:00 [WUProp@Home] doing RPC to get work 17:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374399.09 cpu 2122112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122112.90 17:00:00 [---] Got 0 tasks 17:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:01:00 [---] [rr_sim] doing sim: CPU sched 17:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975803257.60G/439.78G) 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218849.71 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 221884.97 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221884.971399 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996964.74 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 199696.47 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421581.445658 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797268.27 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 179726.83 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601308.272491 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617541.44 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 161754.14 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763062.416641 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455787.30 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 145578.73 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908641.146376 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310208.57 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 131020.86 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039662.003138 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179187.71 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 117918.77 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157580.774223 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061268.94 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 106126.89 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263707.668200 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955142.05 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 95514.20 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359221.872779 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859627.84 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 85962.78 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445184.656900 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773665.06 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 77366.51 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522551.162609 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696298.55 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 69629.86 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592181.017747 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626668.70 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 62666.87 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654847.887372 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 564001.83 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 56400.18 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711248.070034 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507601.64 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 50760.16 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1762008.234429 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456841.48 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 45684.15 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807692.382385 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411157.33 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 41115.73 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848808.115546 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370041.60 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 37004.16 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885812.275390 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333037.44 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 33303.74 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919116.019250 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299733.69 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 29973.37 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949089.388724 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269760.33 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 26976.03 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976065.421251 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242784.29 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 24278.43 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000343.850525 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218505.86 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 21850.59 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022194.436872 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196655.28 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 19665.53 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041859.964584 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176989.75 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 17698.97 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059558.939524 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159290.77 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 15929.08 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075488.016971 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143361.70 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 14336.17 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089824.186673 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129025.53 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 12902.55 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102726.739405 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116122.97 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 11612.30 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114339.036863 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104510.68 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 10451.07 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124790.104576 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94059.61 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 9405.96 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134196.065517 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84653.65 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 8465.36 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142661.430365 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76188.28 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 7618.83 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150280.258727 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68569.46 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 6856.95 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157137.204254 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61712.51 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 6171.25 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163308.455227 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55541.26 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 5554.13 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168862.581104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49987.13 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 4998.71 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173861.294392 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44988.42 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 4498.84 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178360.136352 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40489.58 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 4048.96 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182409.094116 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36440.62 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3644.06 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186053.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32796.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189653.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29196.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193253.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25596.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196853.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21996.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200453.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18396.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204053.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14796.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207653.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11196.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211253.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7596.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214853.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3996.56 sec) 17:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218453.156104 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 396.56 sec) 17:01:00 [GPUGRID] [rr_sim] 2218849.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975803257.60G/439.78G) 17:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853150.63 17:01:00 [---] pick_jobs_to_run() start 17:01:00 [---] pick_jobs_to_run() end 17:01:00 [---] [rr_sim] end 17:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:01:00 [---] [cpu_sched_debug] preliminary job list: 17:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:01:00 [---] [cpu_sched_debug] final job list: 17:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:01:00 [---] [cpu_sched_debug] enforce_run_list: end 17:01:00 [---] choose_project(): 61260.000000 17:01:00 [WUProp@Home] doing RPC to get work 17:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374339.09 cpu 2122052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2122052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2122052.90 17:01:00 [---] Got 0 tasks 17:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:02:00 [---] [rr_sim] doing sim: CPU sched 17:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975775667.25G/439.78G) 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218786.98 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 221878.70 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221878.697711 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996908.28 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 199690.83 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421569.525651 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797217.45 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 179721.75 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601291.270796 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617495.71 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 161749.57 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763040.841428 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455746.14 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 145574.61 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908615.454996 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310171.52 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 131017.15 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039632.607207 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179154.37 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 117915.44 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157548.044197 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061238.93 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 106123.89 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263671.937488 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955115.04 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 95511.50 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359183.441450 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859603.54 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 85960.35 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445143.795016 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773643.18 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 77364.32 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522508.113225 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696278.86 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 69627.89 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592135.999614 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626650.98 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 62665.10 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654801.097363 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563985.88 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 56398.59 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711199.685338 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507587.29 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 50758.73 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761958.414515 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456828.56 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 45682.86 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807641.270774 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411145.71 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 41114.57 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848755.841408 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370031.14 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 37003.11 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885758.954978 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333028.02 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 33302.80 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919061.757191 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299725.22 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 29972.52 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949034.279183 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269752.70 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 26975.27 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976009.548975 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242777.43 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 24277.74 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000287.291789 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218499.69 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 21849.97 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022137.260321 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196649.72 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 19664.97 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041802.231999 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176984.75 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 17698.47 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059500.706510 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159286.27 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 15928.63 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075429.333570 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143357.64 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 14335.76 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089765.097924 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129021.88 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 12902.19 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102667.285842 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116119.69 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 11611.97 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114279.254969 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104507.72 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 10450.77 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124730.027183 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94056.95 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 9405.69 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134135.722176 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84651.25 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 8465.13 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142600.847669 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76186.13 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 7618.61 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150219.460613 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68567.52 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 6856.75 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157076.212262 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61710.76 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 6171.08 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163247.288747 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55539.69 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 5553.97 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168801.257583 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49985.72 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 4998.57 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173799.829536 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44987.15 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 4498.71 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178298.544293 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40488.43 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 4048.84 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182347.387575 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36439.59 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3643.96 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185991.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32795.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189591.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29195.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193191.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25595.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196791.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21995.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200391.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18395.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203991.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14795.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207591.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11195.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211191.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7595.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214791.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3995.63 sec) 17:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218391.346528 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 395.63 sec) 17:02:00 [GPUGRID] [rr_sim] 2218786.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975775667.25G/439.78G) 17:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853147.89 17:02:00 [---] pick_jobs_to_run() start 17:02:00 [---] pick_jobs_to_run() end 17:02:00 [---] [rr_sim] end 17:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:02:00 [---] [cpu_sched_debug] preliminary job list: 17:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:02:00 [---] [cpu_sched_debug] final job list: 17:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:02:00 [---] [cpu_sched_debug] enforce_run_list: end 17:02:00 [---] choose_project(): 61320.000000 17:02:00 [WUProp@Home] doing RPC to get work 17:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374279.09 cpu 2121992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121992.90 17:02:00 [---] Got 0 tasks 17:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:03:00 [---] [rr_sim] doing sim: CPU sched 17:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975775667.25G/439.78G) 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218786.98 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 221878.70 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221878.697711 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996908.28 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 199690.83 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421569.525651 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797217.45 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 179721.75 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601291.270796 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617495.71 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 161749.57 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763040.841428 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455746.14 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 145574.61 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908615.454996 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310171.52 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 131017.15 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039632.607207 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179154.37 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 117915.44 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157548.044197 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061238.93 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 106123.89 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263671.937488 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955115.04 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 95511.50 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359183.441450 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859603.54 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 85960.35 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445143.795016 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773643.18 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 77364.32 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522508.113225 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696278.86 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 69627.89 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592135.999614 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626650.98 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 62665.10 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654801.097363 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563985.88 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 56398.59 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711199.685338 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507587.29 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 50758.73 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761958.414515 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456828.56 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 45682.86 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807641.270774 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411145.71 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 41114.57 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848755.841408 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370031.14 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 37003.11 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885758.954978 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333028.02 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 33302.80 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919061.757191 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299725.22 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 29972.52 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1949034.279183 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269752.70 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 26975.27 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976009.548975 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242777.43 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 24277.74 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000287.291789 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218499.69 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 21849.97 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022137.260321 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196649.72 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 19664.97 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041802.231999 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176984.75 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 17698.47 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059500.706510 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159286.27 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 15928.63 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075429.333570 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143357.64 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 14335.76 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089765.097924 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129021.88 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 12902.19 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102667.285842 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116119.69 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 11611.97 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114279.254969 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104507.72 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 10450.77 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124730.027183 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94056.95 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 9405.69 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134135.722176 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84651.25 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 8465.13 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142600.847669 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76186.13 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 7618.61 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150219.460613 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68567.52 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 6856.75 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157076.212262 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61710.76 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 6171.08 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163247.288747 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55539.69 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 5553.97 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168801.257583 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49985.72 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 4998.57 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173799.829536 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44987.15 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 4498.71 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178298.544293 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40488.43 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 4048.84 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182347.387575 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36439.59 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3643.96 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185991.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32795.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189591.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29195.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193191.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25595.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196791.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21995.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200391.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18395.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203991.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14795.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207591.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11195.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211191.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7595.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214791.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3995.63 sec) 17:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218391.346528 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 395.63 sec) 17:03:00 [GPUGRID] [rr_sim] 2218786.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975775667.25G/439.78G) 17:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853207.89 17:03:00 [---] pick_jobs_to_run() start 17:03:00 [---] pick_jobs_to_run() end 17:03:00 [---] [rr_sim] end 17:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:03:00 [---] [cpu_sched_debug] preliminary job list: 17:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:03:00 [---] [cpu_sched_debug] final job list: 17:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:03:00 [---] [cpu_sched_debug] enforce_run_list: end 17:03:00 [---] choose_project(): 61380.000000 17:03:00 [WUProp@Home] doing RPC to get work 17:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374219.09 cpu 2121992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121992.90 17:03:00 [---] Got 0 tasks 17:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:04:00 [---] [rr_sim] doing sim: CPU sched 17:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975748076.89G/439.78G) 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218724.24 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 221872.42 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221872.424023 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996851.82 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 199685.18 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421557.605643 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797166.63 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 179716.66 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601274.269101 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617449.97 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 161745.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763019.266214 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455704.97 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 145570.50 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908589.763615 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310134.48 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 131013.45 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039603.211276 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179121.03 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 117912.10 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157515.314171 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061208.93 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 106120.89 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263636.206777 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955088.03 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 95508.80 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359145.010122 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859579.23 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 85957.92 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445102.933132 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773621.31 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 77362.13 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522465.063842 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696259.18 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 69625.92 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592090.981480 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626633.26 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 62663.33 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654754.307355 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563969.93 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 56396.99 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711151.300642 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507572.94 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 50757.29 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761908.594600 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456815.65 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 45681.56 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807590.159163 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411134.08 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 41113.41 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848703.567269 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370020.67 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 37002.07 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885705.634565 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333018.61 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 33301.86 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919007.495131 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299716.75 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 29971.67 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948979.169641 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269745.07 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 26974.51 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975953.676699 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242770.56 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 24277.06 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000230.733052 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218493.51 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 21849.35 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022080.083769 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196644.16 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 19664.42 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041744.499415 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176979.74 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 17697.97 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059442.473496 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159281.77 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 15928.18 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075370.650169 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143353.59 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 14335.36 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089706.009175 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129018.23 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 12901.82 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102607.832280 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116116.41 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 11611.64 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114219.473075 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104504.77 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 10450.48 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124669.949790 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94054.29 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 9405.43 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134075.378834 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84648.86 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 8464.89 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142540.264973 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76183.98 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 7618.40 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150158.662498 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68565.58 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 6856.56 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157015.220271 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61709.02 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 6170.90 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163186.122267 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55538.12 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 5553.81 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168739.934063 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49984.31 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 4998.43 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173738.364679 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44985.88 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 4498.59 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178236.952234 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40487.29 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 4048.73 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182285.681033 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36438.56 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3643.86 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185929.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32794.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189529.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29194.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193129.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25594.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196729.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21994.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200329.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18394.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203929.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14794.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207529.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11194.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211129.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7594.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214729.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3994.70 sec) 17:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218329.536952 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 394.70 sec) 17:04:00 [GPUGRID] [rr_sim] 2218724.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975748076.89G/439.78G) 17:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853205.15 17:04:00 [---] pick_jobs_to_run() start 17:04:00 [---] pick_jobs_to_run() end 17:04:00 [---] [rr_sim] end 17:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:04:00 [---] [cpu_sched_debug] preliminary job list: 17:04:00 [---] [cpu_sched_debug] final job list: 17:04:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 17:04:00 [---] [work_fetch] Request work fetch: CPUs idle 17:04:00 [---] [cpu_sched_debug] enforce_run_list: end 17:04:00 [---] choose_project(): 61440.000000 17:04:00 [WUProp@Home] doing RPC to get work 17:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374159.09 cpu 2121932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121932.90 17:04:00 [---] Got 0 tasks 17:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:05:00 [---] [rr_sim] doing sim: CPU sched 17:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975748076.89G/439.78G) 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218724.24 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 221872.42 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221872.424023 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996851.82 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 199685.18 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421557.605643 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797166.63 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 179716.66 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601274.269101 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617449.97 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 161745.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 763019.266214 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455704.97 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 145570.50 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908589.763615 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310134.48 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 131013.45 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039603.211276 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179121.03 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 117912.10 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157515.314171 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061208.93 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 106120.89 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263636.206777 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955088.03 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 95508.80 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359145.010122 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859579.23 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 85957.92 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445102.933132 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773621.31 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 77362.13 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522465.063842 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696259.18 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 69625.92 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592090.981480 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626633.26 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 62663.33 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654754.307355 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563969.93 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 56396.99 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711151.300642 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507572.94 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 50757.29 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761908.594600 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456815.65 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 45681.56 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807590.159163 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411134.08 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 41113.41 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848703.567269 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370020.67 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 37002.07 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885705.634565 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333018.61 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 33301.86 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1919007.495131 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299716.75 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 29971.67 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948979.169641 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269745.07 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 26974.51 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975953.676699 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242770.56 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 24277.06 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000230.733052 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218493.51 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 21849.35 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022080.083769 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196644.16 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 19664.42 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041744.499415 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176979.74 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 17697.97 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059442.473496 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159281.77 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 15928.18 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075370.650169 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143353.59 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 14335.36 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089706.009175 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129018.23 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 12901.82 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102607.832280 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116116.41 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 11611.64 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114219.473075 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104504.77 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 10450.48 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124669.949790 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94054.29 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 9405.43 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134075.378834 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84648.86 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 8464.89 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142540.264973 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76183.98 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 7618.40 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150158.662498 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68565.58 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 6856.56 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157015.220271 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61709.02 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 6170.90 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163186.122267 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55538.12 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 5553.81 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168739.934063 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49984.31 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 4998.43 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173738.364679 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44985.88 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 4498.59 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178236.952234 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40487.29 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 4048.73 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182285.681033 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36438.56 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3643.86 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185929.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32794.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189529.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29194.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193129.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25594.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196729.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21994.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200329.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18394.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203929.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14794.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207529.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11194.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211129.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7594.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214729.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3994.70 sec) 17:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218329.536952 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 394.70 sec) 17:05:00 [GPUGRID] [rr_sim] 2218724.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975748076.89G/439.78G) 17:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853265.15 17:05:00 [---] pick_jobs_to_run() start 17:05:00 [---] pick_jobs_to_run() end 17:05:00 [---] [rr_sim] end 17:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:05:00 [---] [cpu_sched_debug] preliminary job list: 17:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:05:00 [---] [cpu_sched_debug] final job list: 17:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 17:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:05:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 975748076.89G 17:05:00 [---] [cpu_sched_debug] enforce_run_list: end 17:05:00 [---] choose_project(): 61500.000000 17:05:00 [WUProp@Home] doing RPC to get work 17:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374099.09 cpu 2121932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121932.90 17:05:00 [---] Got 0 tasks 17:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:06:00 [---] [rr_sim] doing sim: CPU sched 17:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975720486.53G/439.78G) 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218661.50 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 221866.15 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221866.150334 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996795.35 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 199679.54 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421545.685635 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797115.82 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 179711.58 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601257.267406 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617404.24 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 161740.42 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762997.691000 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455663.81 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 145566.38 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908564.072234 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310097.43 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 131009.74 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039573.815345 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179087.69 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 117908.77 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157482.584145 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061178.92 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 106117.89 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263600.476065 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955061.03 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 95506.10 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359106.578793 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859554.92 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 85955.49 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445062.071248 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773599.43 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 77359.94 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522422.014458 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696239.49 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 69623.95 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592045.963346 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626615.54 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 62661.55 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654707.517346 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563953.99 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 56395.40 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711102.915946 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507558.59 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 50755.86 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761858.774686 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456802.73 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 45680.27 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807539.047552 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411122.46 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 41112.25 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848651.293131 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370010.21 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 37001.02 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885652.314152 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333009.19 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 33300.92 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918953.233071 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299708.27 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 29970.83 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948924.060099 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269737.44 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 26973.74 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975897.804423 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242763.70 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 24276.37 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000174.174315 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218487.33 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 21848.73 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022022.907218 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196638.60 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 19663.86 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041686.766831 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176974.74 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 17697.47 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059384.240482 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159277.26 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 15927.73 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075311.966768 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143349.54 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 14334.95 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089646.920426 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129014.58 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 12901.46 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102548.378718 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116113.12 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 11611.31 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114159.691180 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104501.81 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 10450.18 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124609.872397 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94051.63 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 9405.16 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134015.035492 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84646.47 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 8464.65 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142479.682277 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76181.82 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 7618.18 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150097.864384 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68563.64 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 6856.36 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156954.228280 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61707.28 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 6170.73 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163124.955786 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55536.55 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 5553.65 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168678.610542 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49982.89 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 4998.29 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173676.899822 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44984.60 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 4498.46 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178175.360174 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40486.14 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 4048.61 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182223.974491 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36437.53 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3643.75 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185867.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32793.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189467.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29193.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193067.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25593.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196667.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21993.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200267.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18393.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203867.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14793.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207467.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11193.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211067.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7593.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214667.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3993.78 sec) 17:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218267.727377 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 393.78 sec) 17:06:00 [GPUGRID] [rr_sim] 2218661.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975720486.53G/439.78G) 17:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853262.42 17:06:00 [---] pick_jobs_to_run() start 17:06:00 [---] pick_jobs_to_run() end 17:06:00 [---] [rr_sim] end 17:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:06:00 [---] [cpu_sched_debug] preliminary job list: 17:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 17:06:00 [---] [cpu_sched_debug] final job list: 17:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:06:00 [---] [cpu_sched_debug] enforce_run_list: end 17:06:00 [---] choose_project(): 61560.000000 17:06:00 [WUProp@Home] doing RPC to get work 17:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 374039.09 cpu 2121872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121872.90 17:06:00 [---] Got 0 tasks 17:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:07:00 [---] [rr_sim] doing sim: CPU sched 17:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975692896.18G/439.78G) 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218598.77 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 221859.88 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221859.876646 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996738.89 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 199673.89 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421533.765628 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797065.00 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 179706.50 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601240.265711 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617358.50 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 161735.85 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762976.115786 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455622.65 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 145562.27 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908538.380854 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310060.39 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 131006.04 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039544.419415 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179054.35 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 117905.43 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157449.854119 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061148.91 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 106114.89 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263564.745354 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955034.02 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 95503.40 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359068.147465 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859530.62 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 85953.06 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1445021.209364 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773577.56 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 77357.76 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522378.965074 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696219.80 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 69621.98 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1592000.945213 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626597.82 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 62659.78 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654660.727338 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563938.04 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 56393.80 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711054.531250 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507544.24 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 50754.42 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761808.954771 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456789.81 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 45678.98 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807487.935940 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411110.83 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 41111.08 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848599.018993 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369999.75 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 36999.97 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885598.993740 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332999.77 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 33299.98 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918898.971012 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299699.80 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 29969.98 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948868.950557 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269729.82 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 26972.98 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975841.932147 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242756.83 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 24275.68 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000117.615579 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218481.15 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 21848.12 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021965.730667 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196633.04 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 19663.30 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041629.034247 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176969.73 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 17696.97 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059326.007468 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159272.76 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 15927.28 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075253.283368 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143345.48 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 14334.55 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089587.831677 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129010.93 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 12901.09 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102488.925156 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116109.84 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 11610.98 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114099.909286 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104498.86 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 10449.89 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124549.795004 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94048.97 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 9404.90 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133954.692150 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84644.07 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 8464.41 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142419.099581 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76179.67 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 7617.97 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150037.066269 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68561.70 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 6856.17 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156893.236288 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61705.53 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 6170.55 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163063.789306 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55534.98 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 5553.50 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168617.287021 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49981.48 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 4998.15 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173615.434965 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44983.33 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 4498.33 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178113.768115 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40485.00 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 4048.50 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182162.267950 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36436.50 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3643.65 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185805.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32792.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189405.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29192.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193005.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25592.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196605.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21992.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200205.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18392.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203805.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14792.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207405.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11192.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211005.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7592.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214605.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3992.85 sec) 17:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218205.917801 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 392.85 sec) 17:07:00 [GPUGRID] [rr_sim] 2218598.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975692896.18G/439.78G) 17:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853259.68 17:07:00 [---] pick_jobs_to_run() start 17:07:00 [---] pick_jobs_to_run() end 17:07:00 [---] [rr_sim] end 17:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:07:00 [---] [cpu_sched_debug] preliminary job list: 17:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:07:00 [---] [cpu_sched_debug] final job list: 17:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:07:00 [---] [cpu_sched_debug] enforce_run_list: end 17:07:00 [---] choose_project(): 61620.000000 17:07:00 [WUProp@Home] doing RPC to get work 17:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373979.09 cpu 2121812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121812.90 17:07:00 [---] Got 0 tasks 17:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:08:00 [---] [rr_sim] doing sim: CPU sched 17:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975665305.82G/439.78G) 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218536.03 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 221853.60 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221853.602958 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996682.43 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 199668.24 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421521.845620 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1797014.18 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 179701.42 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601223.264016 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617312.77 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 161731.28 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762954.540573 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455581.49 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 145558.15 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908512.689473 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1310023.34 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 131002.33 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039515.023484 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1179021.01 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 117902.10 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157417.124094 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061118.91 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 106111.89 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263529.014642 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 955007.01 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 95500.70 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1359029.716136 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859506.31 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 85950.63 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444980.347480 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773555.68 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 77355.57 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522335.915690 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696200.11 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 69620.01 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591955.927079 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626580.10 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 62658.01 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654613.937329 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563922.09 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 56392.21 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1711006.146554 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507529.88 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 50752.99 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761759.134857 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456776.89 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 45677.69 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807436.824329 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411099.21 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 41109.92 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848546.744854 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369989.28 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 36998.93 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885545.673327 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332990.36 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 33299.04 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918844.708952 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299691.32 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 29969.13 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948813.841015 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269722.19 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 26972.22 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975786.059871 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242749.97 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 24275.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000061.056842 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218474.97 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 21847.50 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021908.554116 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196627.48 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 19662.75 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041571.301662 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176964.73 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 17696.47 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059267.774454 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159268.26 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 15926.83 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075194.599967 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143341.43 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 14334.14 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089528.742928 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129007.29 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 12900.73 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102429.471593 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116106.56 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 11610.66 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114040.127392 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104495.90 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 10449.59 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124489.717611 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94046.31 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 9404.63 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133894.348808 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84641.68 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 8464.17 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142358.516885 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76177.51 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 7617.75 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149976.268154 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68559.76 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 6855.98 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156832.244297 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61703.79 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 6170.38 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163002.622825 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55533.41 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 5553.34 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168555.963501 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49980.07 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 4998.01 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173553.970109 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44982.06 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 4498.21 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178052.176056 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40483.85 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 4048.39 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182100.561408 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36435.47 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3643.55 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185744.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32791.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189344.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29191.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192944.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25591.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196544.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21991.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200144.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18391.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203744.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14791.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207344.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11191.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210944.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7591.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214544.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3991.92 sec) 17:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218144.108225 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 391.92 sec) 17:08:00 [GPUGRID] [rr_sim] 2218536.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975665305.82G/439.78G) 17:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853256.94 17:08:00 [---] pick_jobs_to_run() start 17:08:00 [---] pick_jobs_to_run() end 17:08:00 [---] [rr_sim] end 17:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:08:00 [---] [cpu_sched_debug] preliminary job list: 17:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:08:00 [---] [cpu_sched_debug] final job list: 17:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:08:00 [---] [cpu_sched_debug] enforce_run_list: end 17:08:00 [---] choose_project(): 61680.000000 17:08:00 [WUProp@Home] doing RPC to get work 17:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373919.09 cpu 2121752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121752.90 17:08:00 [---] Got 0 tasks 17:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:09:00 [---] [rr_sim] doing sim: CPU sched 17:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975637715.47G/439.78G) 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218473.29 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 221847.33 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221847.329270 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996625.96 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 199662.60 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421509.925613 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796963.37 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 179696.34 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601206.262321 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617267.03 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 161726.70 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762932.965359 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455540.33 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 145554.03 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908486.998093 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309986.29 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 130998.63 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039485.627553 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178987.67 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 117898.77 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157384.394068 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061088.90 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 106108.89 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263493.283931 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954980.01 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 95498.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358991.284807 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859482.01 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 85948.20 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444939.485596 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773533.81 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 77353.38 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522292.866307 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696180.43 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 69618.04 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591910.908946 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626562.38 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 62656.24 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654567.147321 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563906.15 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 56390.61 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710957.761859 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507515.53 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 50751.55 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761709.314942 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456763.98 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 45676.40 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807385.712718 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411087.58 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 41108.76 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848494.470716 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369978.82 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 36997.88 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885492.352914 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332980.94 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 33298.09 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918790.446893 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299682.85 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 29968.28 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948758.731473 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269714.56 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 26971.46 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975730.187595 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242743.11 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 24274.31 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000004.498106 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218468.79 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 21846.88 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021851.377565 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196621.92 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 19662.19 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041513.569078 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176959.72 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 17695.97 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059209.541440 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159263.75 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 15926.38 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075135.916566 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143337.38 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 14333.74 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089469.654179 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129003.64 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 12900.36 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102370.018031 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116103.27 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 11610.33 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113980.345498 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104492.95 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 10449.29 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124429.640218 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94043.65 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 9404.37 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133834.005466 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84639.29 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 8463.93 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142297.934189 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76175.36 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 7617.54 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149915.470040 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68557.82 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 6855.78 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156771.252306 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61702.04 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 6170.20 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162941.456345 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55531.84 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 5553.18 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168494.639980 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49978.65 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 4997.87 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173492.505252 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44980.79 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 4498.08 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177990.583996 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40482.71 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 4048.27 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182038.854867 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36434.44 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3643.44 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185682.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32790.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189282.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29190.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192882.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25590.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196482.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21990.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200082.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18390.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203682.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14790.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207282.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11190.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210882.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7590.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214482.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3990.99 sec) 17:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218082.298650 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 390.99 sec) 17:09:00 [GPUGRID] [rr_sim] 2218473.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975637715.47G/439.78G) 17:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853254.21 17:09:00 [---] pick_jobs_to_run() start 17:09:00 [---] pick_jobs_to_run() end 17:09:00 [---] [rr_sim] end 17:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:09:00 [---] [cpu_sched_debug] preliminary job list: 17:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:09:00 [---] [cpu_sched_debug] final job list: 17:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:09:00 [---] [cpu_sched_debug] enforce_run_list: end 17:09:00 [---] choose_project(): 61740.000000 17:09:00 [WUProp@Home] doing RPC to get work 17:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373859.09 cpu 2121692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121692.90 17:09:00 [---] Got 0 tasks 17:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:10:00 [---] [rr_sim] doing sim: CPU sched 17:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975610125.11G/439.78G) 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218410.56 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 221841.06 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221841.055582 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996569.50 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 199656.95 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421498.005605 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796912.55 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 179691.26 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601189.260626 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617221.30 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 161722.13 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762911.390145 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455499.17 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 145549.92 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908461.306712 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309949.25 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 130994.92 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039456.231622 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178954.32 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 117895.43 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157351.664042 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061058.89 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 106105.89 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263457.553219 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954953.00 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 95495.30 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358952.853479 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859457.70 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 85945.77 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444898.623712 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773511.93 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 77351.19 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522249.816923 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696160.74 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 69616.07 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591865.890812 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626544.67 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 62654.47 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654520.357312 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563890.20 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 56389.02 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710909.377163 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507501.18 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 50750.12 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761659.495028 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456751.06 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 45675.11 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807334.601107 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411075.95 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 41107.60 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848442.196578 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369968.36 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 36996.84 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885439.032501 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332971.52 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 33297.15 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918736.184833 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299674.37 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 29967.44 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948703.621931 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269706.93 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 26970.69 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975674.315320 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242736.24 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 24273.62 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999947.939369 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218462.62 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 21846.26 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021794.201014 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196616.35 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 19661.64 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041455.836494 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176954.72 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 17695.47 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059151.308426 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159259.25 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 15925.92 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075077.233165 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143333.32 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 14333.33 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089410.565430 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128999.99 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 12900.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102310.564469 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116099.99 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 11610.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113920.563603 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104489.99 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 10449.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124369.562825 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94040.99 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 9404.10 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133773.662124 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84636.89 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 8463.69 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142237.351493 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76173.20 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 7617.32 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149854.671925 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68555.88 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 6855.59 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156710.260314 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61700.30 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 6170.03 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162880.289864 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55530.27 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 5553.03 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168433.316459 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49977.24 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 4997.72 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173431.040395 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44979.52 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 4497.95 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177928.991937 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40481.56 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 4048.16 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181977.148325 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36433.41 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3643.34 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185620.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32790.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189220.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29190.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192820.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25590.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196420.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21990.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200020.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18390.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203620.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14790.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207220.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11190.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210820.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7590.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214420.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3990.07 sec) 17:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2218020.489074 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 390.07 sec) 17:10:00 [GPUGRID] [rr_sim] 2218410.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975610125.11G/439.78G) 17:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853251.47 17:10:00 [---] pick_jobs_to_run() start 17:10:00 [---] pick_jobs_to_run() end 17:10:00 [---] [rr_sim] end 17:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:10:00 [---] [cpu_sched_debug] preliminary job list: 17:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:10:00 [---] [cpu_sched_debug] final job list: 17:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:10:00 [---] [cpu_sched_debug] enforce_run_list: end 17:10:00 [---] choose_project(): 61800.000000 17:10:00 [WUProp@Home] doing RPC to get work 17:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373799.09 cpu 2121632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121632.90 17:10:00 [---] Got 0 tasks 17:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:11:00 [---] [rr_sim] doing sim: CPU sched 17:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975582534.75G/439.78G) 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218347.82 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 221834.78 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221834.781893 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996513.04 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 199651.30 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421486.085597 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796861.73 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 179686.17 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601172.258931 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617175.56 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 161717.56 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762889.814931 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455458.00 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 145545.80 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908435.615331 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309912.20 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 130991.22 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039426.835692 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178920.98 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 117892.10 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157318.934016 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1061028.88 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 106102.89 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263421.822508 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954926.00 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 95492.60 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358914.422150 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859433.40 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 85943.34 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444857.761828 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773490.06 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 77349.01 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522206.767539 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696141.05 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 69614.11 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591820.872678 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626526.95 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 62652.69 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654473.567304 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563874.25 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 56387.43 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710860.992467 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507486.83 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 50748.68 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761609.675114 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456738.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 45673.81 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807283.489496 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411064.33 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 41106.43 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848389.922439 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369957.90 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 36995.79 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885385.712089 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332962.11 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 33296.21 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918681.922773 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299665.90 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 29966.59 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948648.512389 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269699.31 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 26969.93 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975618.443044 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242729.38 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 24272.94 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999891.380633 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218456.44 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 21845.64 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021737.024463 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196610.79 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 19661.08 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041398.103910 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176949.72 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 17694.97 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059093.075412 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159254.74 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 15925.47 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075018.549764 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143329.27 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 14332.93 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089351.476681 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128996.34 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 12899.63 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102251.110906 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116096.71 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 11609.67 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113860.781709 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104487.04 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 10448.70 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124309.485432 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94038.33 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 9403.83 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133713.318782 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84634.50 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 8463.45 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142176.768797 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76171.05 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 7617.11 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149793.873811 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68553.95 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 6855.39 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156649.268323 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61698.55 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 6169.86 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162819.123384 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55528.70 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 5552.87 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168371.992939 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49975.83 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 4997.58 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173369.575538 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44978.24 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 4497.82 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177867.399878 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40480.42 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 4048.04 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181915.441783 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36432.38 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3643.24 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185558.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32789.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189158.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29189.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192758.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25589.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196358.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21989.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199958.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18389.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203558.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14789.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207158.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11189.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210758.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7589.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214358.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3989.14 sec) 17:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217958.679498 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 389.14 sec) 17:11:00 [GPUGRID] [rr_sim] 2218347.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975582534.75G/439.78G) 17:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853248.73 17:11:00 [---] pick_jobs_to_run() start 17:11:00 [---] pick_jobs_to_run() end 17:11:00 [---] [rr_sim] end 17:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:11:00 [---] [cpu_sched_debug] preliminary job list: 17:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:11:00 [---] [cpu_sched_debug] final job list: 17:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:11:00 [---] [cpu_sched_debug] enforce_run_list: end 17:11:00 [---] choose_project(): 61860.000000 17:11:00 [WUProp@Home] doing RPC to get work 17:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373739.09 cpu 2121572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121572.90 17:11:00 [---] Got 0 tasks 17:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:12:00 [---] [rr_sim] doing sim: CPU sched 17:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975554944.40G/439.78G) 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218285.08 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 221828.51 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221828.508205 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996456.57 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 199645.66 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421474.165590 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796810.92 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 179681.09 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601155.257236 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617129.82 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 161712.98 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762868.239717 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455416.84 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 145541.68 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908409.923951 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309875.16 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 130987.52 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039397.439761 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178887.64 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 117888.76 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157286.203990 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060998.88 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 106099.89 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263386.091796 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954898.99 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 95489.90 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358875.990822 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859409.09 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 85940.91 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444816.899945 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773468.18 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 77346.82 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522163.718155 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696121.36 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 69612.14 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591775.854545 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626509.23 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 62650.92 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654426.777295 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563858.30 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 56385.83 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710812.607771 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507472.47 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 50747.25 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761559.855199 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456725.23 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 45672.52 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807232.377884 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411052.70 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 41105.27 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848337.648301 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369947.43 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 36994.74 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885332.391676 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332952.69 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 33295.27 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918627.660714 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299657.42 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 29965.74 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948593.402847 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269691.68 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 26969.17 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975562.570768 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242722.51 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 24272.25 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999834.821896 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218450.26 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 21845.03 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021679.847912 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196605.23 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 19660.52 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041340.371326 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176944.71 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 17694.47 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059034.842398 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159250.24 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 15925.02 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074959.866363 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143325.22 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 14332.52 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089292.387932 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128992.69 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 12899.27 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102191.657344 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116093.42 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 11609.34 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113800.999815 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104484.08 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 10448.41 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124249.408038 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94035.67 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 9403.57 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133652.975440 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84632.11 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 8463.21 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142116.186101 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76168.90 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 7616.89 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149733.075696 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68552.01 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 6855.20 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156588.276331 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61696.81 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 6169.68 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162757.956903 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55527.13 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 5552.71 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168310.669418 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49974.41 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 4997.44 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173308.110682 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44976.97 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 4497.70 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177805.807818 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40479.27 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 4047.93 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181853.735242 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36431.35 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3643.13 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185496.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32788.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189096.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29188.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192696.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25588.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196296.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21988.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199896.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18388.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203496.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14788.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207096.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11188.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210696.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7588.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214296.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3988.21 sec) 17:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217896.869923 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 388.21 sec) 17:12:00 [GPUGRID] [rr_sim] 2218285.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975554944.40G/439.78G) 17:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853245.99 17:12:00 [---] pick_jobs_to_run() start 17:12:00 [---] pick_jobs_to_run() end 17:12:00 [---] [rr_sim] end 17:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:12:00 [---] [cpu_sched_debug] preliminary job list: 17:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:12:00 [---] [cpu_sched_debug] final job list: 17:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:12:00 [---] [cpu_sched_debug] enforce_run_list: end 17:12:00 [---] choose_project(): 61920.000000 17:12:00 [WUProp@Home] doing RPC to get work 17:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373679.09 cpu 2121512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121512.90 17:12:00 [---] Got 0 tasks 17:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:13:00 [---] [rr_sim] doing sim: CPU sched 17:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975527354.04G/439.78G) 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218222.35 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 221822.23 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221822.234517 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996400.11 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 199640.01 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421462.245582 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796760.10 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 179676.01 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601138.255541 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617084.09 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 161708.41 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762846.664504 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455375.68 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 145537.57 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908384.232570 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309838.11 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 130983.81 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039368.043830 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178854.30 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 117885.43 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157253.473964 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060968.87 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 106096.89 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263350.361084 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954871.98 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 95487.20 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358837.559493 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859384.79 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 85938.48 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444776.038061 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773446.31 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 77344.63 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522120.668771 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696101.68 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 69610.17 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591730.836411 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626491.51 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 62649.15 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654379.987287 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563842.36 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 56384.24 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710764.223075 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507458.12 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 50745.81 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761510.035285 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456712.31 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 45671.23 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807181.266273 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411041.08 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 41104.11 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848285.374163 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369936.97 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 36993.70 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885279.071263 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332943.27 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 33294.33 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918573.398654 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299648.95 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 29964.89 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948538.293305 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269684.05 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 26968.41 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975506.698492 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242715.65 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 24271.56 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999778.263159 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218444.08 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 21844.41 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021622.671360 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196599.67 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 19659.97 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041282.638741 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176939.71 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 17693.97 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058976.609384 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159245.74 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 15924.57 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074901.182963 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143321.16 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 14332.12 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089233.299183 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128989.05 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 12898.90 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102132.203782 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116090.14 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 11609.01 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113741.217921 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104481.13 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 10448.11 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124189.330645 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94033.01 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 9403.30 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133592.632098 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84629.71 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 8462.97 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142055.603405 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76166.74 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 7616.67 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149672.277581 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68550.07 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 6855.01 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156527.284340 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61695.06 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 6169.51 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162696.790423 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55525.55 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 5552.56 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168249.345898 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49973.00 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 4997.30 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173246.645825 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44975.70 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 4497.57 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177744.215759 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40478.13 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 4047.81 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181792.028700 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36430.32 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3643.03 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185435.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32787.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189035.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29187.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192635.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25587.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196235.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21987.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199835.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18387.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203435.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14787.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207035.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11187.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210635.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7587.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214235.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3987.28 sec) 17:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217835.060347 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 387.28 sec) 17:13:00 [GPUGRID] [rr_sim] 2218222.35: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975527354.04G/439.78G) 17:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853243.26 17:13:00 [---] pick_jobs_to_run() start 17:13:00 [---] pick_jobs_to_run() end 17:13:00 [---] [rr_sim] end 17:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:13:00 [---] [cpu_sched_debug] preliminary job list: 17:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:13:00 [---] [cpu_sched_debug] final job list: 17:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:13:00 [---] [cpu_sched_debug] enforce_run_list: end 17:13:00 [---] choose_project(): 61980.000000 17:13:00 [WUProp@Home] doing RPC to get work 17:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373619.09 cpu 2121452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121452.90 17:13:00 [---] Got 0 tasks 17:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:14:00 [---] [rr_sim] doing sim: CPU sched 17:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975499763.68G/439.78G) 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218159.61 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 221815.96 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221815.960829 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996343.65 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 199634.36 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421450.325575 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796709.28 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 179670.93 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601121.253846 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1617038.35 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 161703.84 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762825.089290 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455334.52 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 145533.45 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908358.541190 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309801.07 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 130980.11 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039338.647899 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178820.96 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 117882.10 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157220.743938 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060938.86 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 106093.89 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263314.630373 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954844.98 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 95484.50 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358799.128164 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859360.48 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 85936.05 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444735.176177 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773424.43 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 77342.44 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522077.619388 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696081.99 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 69608.20 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591685.818278 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626473.79 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 62647.38 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654333.197279 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563826.41 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 56382.64 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710715.838379 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507443.77 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 50744.38 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761460.215370 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456699.39 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 45669.94 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807130.154662 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411029.45 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 41102.95 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848233.100024 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369926.51 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 36992.65 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885225.750851 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332933.86 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 33293.39 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918519.136594 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299640.47 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 29964.05 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948483.183763 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269676.42 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 26967.64 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975450.826216 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242708.78 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 24270.88 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999721.704423 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218437.90 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 21843.79 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021565.494809 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196594.11 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 19659.41 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041224.906157 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176934.70 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 17693.47 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058918.376370 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159241.23 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 15924.12 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074842.499562 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143317.11 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 14331.71 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089174.210434 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128985.40 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 12898.54 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102072.750220 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116086.86 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 11608.69 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113681.436026 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104478.17 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 10447.82 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124129.253252 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94030.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 9403.04 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133532.288756 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84627.32 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 8462.73 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141995.020709 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76164.59 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 7616.46 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149611.479467 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68548.13 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 6854.81 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156466.292349 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61693.32 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 6169.33 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162635.623943 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55523.98 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 5552.40 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168188.022377 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49971.59 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 4997.16 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173185.180968 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44974.43 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 4497.44 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177682.623700 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40476.98 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 4047.70 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181730.322159 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36429.29 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3642.93 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185373.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32786.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188973.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29186.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192573.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25586.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196173.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21986.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199773.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18386.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203373.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14786.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206973.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11186.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210573.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7586.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214173.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3986.36 sec) 17:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217773.250771 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 386.36 sec) 17:14:00 [GPUGRID] [rr_sim] 2218159.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975499763.68G/439.78G) 17:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853240.52 17:14:00 [---] pick_jobs_to_run() start 17:14:00 [---] pick_jobs_to_run() end 17:14:00 [---] [rr_sim] end 17:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:14:00 [---] [cpu_sched_debug] preliminary job list: 17:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:14:00 [---] [cpu_sched_debug] final job list: 17:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:14:00 [---] [cpu_sched_debug] enforce_run_list: end 17:14:00 [---] choose_project(): 62040.000000 17:14:00 [WUProp@Home] doing RPC to get work 17:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373559.09 cpu 2121392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121392.90 17:14:00 [---] Got 0 tasks 17:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:15:00 [---] [rr_sim] doing sim: CPU sched 17:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975472173.33G/439.78G) 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218096.87 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 221809.69 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221809.687140 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996287.18 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 199628.72 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421438.405567 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796658.47 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 179665.85 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601104.252151 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616992.62 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 161699.26 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762803.514076 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455293.36 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 145529.34 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908332.849809 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309764.02 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 130976.40 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039309.251969 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178787.62 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 117878.76 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157188.013912 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060908.86 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 106090.89 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263278.899661 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954817.97 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 95481.80 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358760.696836 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859336.17 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 85933.62 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444694.314293 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773402.56 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 77340.26 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1522034.570004 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696062.30 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 69606.23 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591640.800144 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626456.07 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 62645.61 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654286.407270 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563810.46 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 56381.05 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710667.453684 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507429.42 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 50742.94 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761410.395456 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456686.48 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 45668.65 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807079.043051 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411017.83 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 41101.78 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848180.825886 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369916.05 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 36991.60 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885172.430438 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332924.44 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 33292.44 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918464.874535 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299632.00 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 29963.20 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948428.074222 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269668.80 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 26966.88 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975394.953940 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242701.92 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 24270.19 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999665.145686 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218431.73 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 21843.17 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021508.318258 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196588.55 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 19658.86 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041167.173573 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176929.70 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 17692.97 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058860.143356 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159236.73 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 15923.67 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074783.816161 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143313.06 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 14331.31 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089115.121685 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128981.75 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 12898.17 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102013.296657 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116083.57 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 11608.36 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113621.654132 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104475.22 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 10447.52 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124069.175859 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94027.70 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 9402.77 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133471.945414 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84624.93 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 8462.49 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141934.438013 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76162.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 7616.24 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149550.681352 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68546.19 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 6854.62 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156405.300357 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61691.57 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 6169.16 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162574.457462 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55522.41 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 5552.24 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168126.698856 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49970.17 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 4997.02 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173123.716111 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44973.16 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 4497.32 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177621.031641 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40475.84 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 4047.58 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181668.615617 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36428.26 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3642.83 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185311.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32785.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188911.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29185.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192511.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25585.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196111.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21985.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199711.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18385.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203311.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14785.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206911.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11185.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210511.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7585.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214111.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3985.43 sec) 17:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217711.441196 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 385.43 sec) 17:15:00 [GPUGRID] [rr_sim] 2218096.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975472173.33G/439.78G) 17:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853237.78 17:15:00 [---] pick_jobs_to_run() start 17:15:00 [---] pick_jobs_to_run() end 17:15:00 [---] [rr_sim] end 17:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:15:00 [---] [cpu_sched_debug] preliminary job list: 17:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:15:00 [---] [cpu_sched_debug] final job list: 17:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:15:00 [---] [cpu_sched_debug] enforce_run_list: end 17:15:00 [---] choose_project(): 62100.000000 17:15:00 [WUProp@Home] doing RPC to get work 17:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373499.09 cpu 2121332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121332.90 17:15:00 [---] Got 0 tasks 17:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:16:00 [---] [rr_sim] doing sim: CPU sched 17:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975444582.97G/439.78G) 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2218034.13 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 221803.41 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221803.413452 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996230.72 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 199623.07 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421426.485559 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796607.65 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 179660.76 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601087.250456 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616946.88 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 161694.69 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762781.938862 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455252.20 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 145525.22 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908307.158428 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309726.98 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 130972.70 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039279.856038 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178754.28 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 117875.43 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157155.283886 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060878.85 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 106087.89 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263243.168950 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954790.97 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 95479.10 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358722.265507 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859311.87 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 85931.19 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444653.452409 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773380.68 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 77338.07 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521991.520620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696042.61 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 69604.26 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591595.782010 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626438.35 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 62643.84 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654239.617262 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563794.52 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 56379.45 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710619.068988 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507415.07 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 50741.51 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761360.575541 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456673.56 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 45667.36 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1807027.931439 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 411006.20 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 41100.62 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848128.551748 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369905.58 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 36990.56 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885119.110025 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332915.02 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 33291.50 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918410.612475 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299623.52 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 29962.35 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948372.964680 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269661.17 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 26966.12 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975339.081664 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242695.05 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 24269.51 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999608.586950 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218425.55 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 21842.55 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021451.141707 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196582.99 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 19658.30 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041109.440989 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176924.69 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 17692.47 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058801.910342 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159232.22 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 15923.22 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074725.132760 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143309.00 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 14330.90 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089056.032936 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128978.10 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 12897.81 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101953.843095 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116080.29 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 11608.03 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113561.872238 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104472.26 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 10447.23 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124009.098466 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94025.04 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 9402.50 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133411.602072 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84622.53 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 8462.25 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141873.855317 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76160.28 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 7616.03 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149489.883237 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68544.25 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 6854.43 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156344.308366 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61689.83 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 6168.98 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162513.290982 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55520.84 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 5552.08 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168065.375336 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49968.76 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 4996.88 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173062.251254 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44971.88 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 4497.19 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177559.439581 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40474.69 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 4047.47 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181606.909075 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36427.23 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3642.72 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185249.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32784.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188849.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29184.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192449.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25584.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196049.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21984.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199649.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18384.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203249.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14784.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206849.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11184.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210449.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7584.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214049.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3984.50 sec) 17:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217649.631620 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 384.50 sec) 17:16:00 [GPUGRID] [rr_sim] 2218034.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975444582.97G/439.78G) 17:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853235.05 17:16:00 [---] pick_jobs_to_run() start 17:16:00 [---] pick_jobs_to_run() end 17:16:00 [---] [rr_sim] end 17:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:16:00 [---] [cpu_sched_debug] preliminary job list: 17:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:16:00 [---] [cpu_sched_debug] final job list: 17:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:16:00 [---] [cpu_sched_debug] enforce_run_list: end 17:16:00 [---] choose_project(): 62160.000000 17:16:00 [WUProp@Home] doing RPC to get work 17:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373439.09 cpu 2121272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121272.90 17:16:00 [---] Got 0 tasks 17:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:17:00 [---] [rr_sim] doing sim: CPU sched 17:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975416992.62G/439.78G) 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217971.40 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 221797.14 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221797.139764 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996174.26 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 199617.43 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421414.565552 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796556.83 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 179655.68 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601070.248761 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616901.15 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 161690.11 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762760.363649 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455211.03 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 145521.10 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908281.467048 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309689.93 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 130968.99 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039250.460107 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178720.94 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 117872.09 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157122.553860 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060848.84 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 106084.88 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263207.438238 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954763.96 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 95476.40 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358683.834179 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859287.56 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 85928.76 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444612.590525 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773358.81 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 77335.88 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521948.471236 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696022.93 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 69602.29 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591550.763877 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626420.63 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 62642.06 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654192.827253 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563778.57 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 56377.86 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710570.684292 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507400.71 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 50740.07 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761310.755627 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456660.64 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 45666.06 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806976.819828 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410994.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 41099.46 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848076.277609 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369895.12 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 36989.51 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885065.789612 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332905.61 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 33290.56 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918356.350415 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299615.05 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 29961.50 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948317.855138 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269653.54 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 26965.35 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975283.209388 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242688.19 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 24268.82 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999552.028213 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218419.37 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 21841.94 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021393.965156 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196577.43 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 19657.74 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041051.708404 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176919.69 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 17691.97 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058743.677328 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159227.72 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 15922.77 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074666.449359 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143304.95 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 14330.49 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088996.944187 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128974.45 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 12897.45 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101894.389533 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116077.01 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 11607.70 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113502.090343 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104469.31 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 10446.93 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123949.021073 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94022.38 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 9402.24 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133351.258730 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84620.14 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 8462.01 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141813.272621 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76158.13 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 7615.81 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149429.085123 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68542.31 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 6854.23 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156283.316375 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61688.08 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 6168.81 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162452.124501 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55519.27 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 5551.93 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168004.051815 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49967.35 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 4996.73 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173000.786398 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44970.61 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 4497.06 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177497.847522 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40473.55 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 4047.36 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181545.202534 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36426.20 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3642.62 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185187.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32783.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188787.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29183.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192387.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25583.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195987.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21983.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199587.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18383.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203187.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14783.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206787.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11183.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210387.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7583.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213987.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3983.58 sec) 17:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217587.822044 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 383.58 sec) 17:17:00 [GPUGRID] [rr_sim] 2217971.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975416992.62G/439.78G) 17:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853232.31 17:17:00 [---] pick_jobs_to_run() start 17:17:00 [---] pick_jobs_to_run() end 17:17:00 [---] [rr_sim] end 17:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:17:00 [---] [cpu_sched_debug] preliminary job list: 17:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:17:00 [---] [cpu_sched_debug] final job list: 17:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:17:00 [---] [cpu_sched_debug] enforce_run_list: end 17:17:00 [---] choose_project(): 62220.000000 17:17:00 [WUProp@Home] doing RPC to get work 17:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373379.09 cpu 2121212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121212.90 17:17:00 [---] Got 0 tasks 17:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:18:00 [---] [rr_sim] doing sim: CPU sched 17:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975389402.26G/439.78G) 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217908.66 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 221790.87 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221790.866076 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996117.79 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 199611.78 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421402.645544 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796506.02 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 179650.60 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601053.247065 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616855.41 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 161685.54 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762738.788435 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455169.87 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 145516.99 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908255.775667 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309652.89 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 130965.29 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039221.064176 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178687.60 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 117868.76 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157089.823834 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060818.84 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 106081.88 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263171.707527 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954736.95 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 95473.70 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358645.402850 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859263.26 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 85926.33 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444571.728641 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773336.93 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 77333.69 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521905.421853 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 696003.24 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 69600.32 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591505.745743 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626402.92 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 62640.29 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654146.037245 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563762.62 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 56376.26 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710522.299596 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507386.36 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 50738.64 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761260.935712 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456647.73 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 45664.77 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806925.708217 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410982.95 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 41098.30 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1848024.003471 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369884.66 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 36988.47 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1885012.469200 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332896.19 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 33289.62 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918302.088356 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299606.57 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 29960.66 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948262.745596 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269645.92 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 26964.59 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975227.337112 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242681.32 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 24268.13 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999495.469477 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218413.19 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 21841.32 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021336.788605 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196571.87 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 19657.19 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040993.975820 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176914.68 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 17691.47 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058685.444314 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159223.22 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 15922.32 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074607.765958 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143300.89 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 14330.09 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088937.855438 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128970.81 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 12897.08 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101834.935970 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116073.72 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 11607.37 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113442.308449 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104466.35 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 10446.64 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123888.943680 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94019.72 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 9401.97 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133290.915388 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84617.75 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 8461.77 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141752.689925 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76155.97 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 7615.60 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149368.287008 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68540.37 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 6854.04 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156222.324383 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61686.34 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 6168.63 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162390.958021 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55517.70 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 5551.77 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167942.728295 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49965.93 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 4996.59 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172939.321541 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44969.34 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 4496.93 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177436.255463 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40472.41 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 4047.24 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181483.495992 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36425.16 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3642.52 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185126.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32782.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188726.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29182.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192326.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25582.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195926.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21982.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199526.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18382.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203126.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14782.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206726.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11182.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210326.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7582.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213926.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3982.65 sec) 17:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217526.012469 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 382.65 sec) 17:18:00 [GPUGRID] [rr_sim] 2217908.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975389402.26G/439.78G) 17:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853229.57 17:18:00 [---] pick_jobs_to_run() start 17:18:00 [---] pick_jobs_to_run() end 17:18:00 [---] [rr_sim] end 17:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:18:00 [---] [cpu_sched_debug] preliminary job list: 17:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:18:00 [---] [cpu_sched_debug] final job list: 17:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:18:00 [---] [cpu_sched_debug] enforce_run_list: end 17:18:00 [---] choose_project(): 62280.000000 17:18:00 [WUProp@Home] doing RPC to get work 17:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373319.09 cpu 2121152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121152.90 17:18:00 [---] Got 0 tasks 17:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:19:00 [---] [rr_sim] doing sim: CPU sched 17:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975361811.90G/439.78G) 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217845.92 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 221784.59 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221784.592388 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996061.33 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 199606.13 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421390.725536 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796455.20 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 179645.52 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601036.245370 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616809.68 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 161680.97 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762717.213221 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455128.71 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 145512.87 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908230.084286 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309615.84 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 130961.58 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039191.668245 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178654.26 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 117865.43 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157057.093809 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060788.83 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 106078.88 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263135.976815 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954709.95 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 95470.99 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358606.971521 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859238.95 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 85923.90 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444530.866757 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773315.06 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 77331.51 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521862.372469 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695983.55 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 69598.36 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591460.727609 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626385.20 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 62638.52 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654099.247236 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563746.68 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 56374.67 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710473.914900 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507372.01 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 50737.20 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761211.115798 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456634.81 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 45663.48 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806874.596606 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410971.33 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 41097.13 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847971.729333 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369874.19 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 36987.42 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884959.148787 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332886.78 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 33288.68 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918247.826296 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299598.10 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 29959.81 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948207.636054 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269638.29 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 26963.83 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975171.464836 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242674.46 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 24267.45 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999438.910740 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218407.01 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 21840.70 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021279.612054 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196566.31 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 19656.63 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040936.243236 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176909.68 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 17690.97 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058627.211300 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159218.71 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 15921.87 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074549.082558 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143296.84 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 14329.68 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088878.766689 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128967.16 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 12896.72 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101775.482408 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116070.44 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 11607.04 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113382.526555 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104463.40 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 10446.34 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123828.866287 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94017.06 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 9401.71 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133230.572046 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84615.35 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 8461.54 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141692.107229 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76153.82 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 7615.38 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149307.488894 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68538.43 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 6853.84 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156161.332392 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61684.59 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 6168.46 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162329.791540 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55516.13 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 5551.61 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167881.404774 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49964.52 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 4996.45 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172877.856684 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44968.07 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 4496.81 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177374.663403 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40471.26 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 4047.13 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181421.789451 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36424.13 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3642.41 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185064.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32781.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188664.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29181.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192264.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25581.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195864.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21981.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199464.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18381.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203064.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14781.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206664.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11181.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210264.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7581.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213864.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3981.72 sec) 17:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217464.202893 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 381.72 sec) 17:19:00 [GPUGRID] [rr_sim] 2217845.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975361811.90G/439.78G) 17:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853226.84 17:19:00 [---] pick_jobs_to_run() start 17:19:00 [---] pick_jobs_to_run() end 17:19:00 [---] [rr_sim] end 17:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:19:00 [---] [cpu_sched_debug] preliminary job list: 17:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:19:00 [---] [cpu_sched_debug] final job list: 17:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:19:00 [---] [cpu_sched_debug] enforce_run_list: end 17:19:00 [---] choose_project(): 62340.000000 17:19:00 [WUProp@Home] doing RPC to get work 17:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373259.09 cpu 2121092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121092.90 17:19:00 [---] Got 0 tasks 17:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:20:00 [---] [rr_sim] doing sim: CPU sched 17:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975334221.55G/439.78G) 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217783.19 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 221778.32 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221778.318699 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1996004.87 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 199600.49 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421378.805529 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796404.38 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 179640.44 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601019.243675 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616763.94 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 161676.39 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762695.638007 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455087.55 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 145508.75 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908204.392906 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309578.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 130957.88 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039162.272315 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178620.91 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 117862.09 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1157024.363783 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060758.82 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 106075.88 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263100.246104 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954682.94 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 95468.29 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358568.540193 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859214.65 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 85921.46 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444490.004873 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773293.18 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 77329.32 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521819.323085 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695963.86 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 69596.39 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591415.709476 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626367.48 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 62636.75 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654052.457228 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563730.73 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 56373.07 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710425.530204 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507357.66 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 50735.77 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761161.295883 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456621.89 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 45662.19 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806823.484994 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410959.70 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 41095.97 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847919.455194 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369863.73 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 36986.37 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884905.828374 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332877.36 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 33287.74 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918193.564236 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299589.62 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 29958.96 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948152.526512 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269630.66 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 26963.07 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975115.592560 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242667.59 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 24266.76 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999382.352004 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218400.83 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 21840.08 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021222.435503 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196560.75 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 19656.08 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040878.510652 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176904.68 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 17690.47 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058568.978286 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159214.21 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 15921.42 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074490.399157 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143292.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 14329.28 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088819.677940 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128963.51 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 12896.35 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101716.028846 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116067.16 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 11606.72 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113322.744661 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104460.44 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 10446.04 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123768.788894 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94014.40 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 9401.44 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133170.228704 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84612.96 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 8461.30 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141631.524533 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76151.66 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 7615.17 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149246.690779 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68536.50 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 6853.65 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156100.340401 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61682.85 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 6168.28 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162268.625060 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55514.56 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 5551.46 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167820.081253 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49963.11 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 4996.31 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172816.391827 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44966.80 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 4496.68 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177313.071344 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40470.12 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 4047.01 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181360.082909 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36423.10 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3642.31 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185002.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32780.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188602.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29180.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192202.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25580.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195802.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21980.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199402.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18380.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203002.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14780.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206602.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11180.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210202.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7580.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213802.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3980.79 sec) 17:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217402.393317 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 380.79 sec) 17:20:00 [GPUGRID] [rr_sim] 2217783.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975334221.55G/439.78G) 17:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853224.10 17:20:00 [---] pick_jobs_to_run() start 17:20:00 [---] pick_jobs_to_run() end 17:20:00 [---] [rr_sim] end 17:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:20:00 [---] [cpu_sched_debug] preliminary job list: 17:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:20:00 [---] [cpu_sched_debug] final job list: 17:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:20:00 [---] [cpu_sched_debug] enforce_run_list: end 17:20:00 [---] choose_project(): 62400.000000 17:20:00 [WUProp@Home] doing RPC to get work 17:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373199.09 cpu 2121032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2121032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2121032.90 17:20:00 [---] Got 0 tasks 17:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:21:00 [---] [rr_sim] doing sim: CPU sched 17:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975306631.19G/439.78G) 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217720.45 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 221772.05 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221772.045011 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995948.41 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 199594.84 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421366.885521 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796353.56 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 179635.36 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 601002.241980 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616718.21 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 161671.82 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762674.062793 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455046.39 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 145504.64 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908178.701525 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309541.75 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 130954.17 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039132.876384 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178587.57 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 117858.76 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156991.633757 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060728.82 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 106072.88 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263064.515392 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954655.93 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 95465.59 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358530.108864 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859190.34 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 85919.03 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444449.142989 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773271.31 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 77327.13 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521776.273701 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695944.18 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 69594.42 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591370.691342 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626349.76 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 62634.98 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1654005.667219 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563714.78 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 56371.48 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710377.145508 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507343.30 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 50734.33 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761111.475969 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456608.97 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 45660.90 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806772.373383 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410948.08 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 41094.81 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847867.181056 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369853.27 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 36985.33 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884852.507962 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332867.94 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 33286.79 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918139.302177 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299581.15 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 29958.11 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948097.416970 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269623.03 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 26962.30 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975059.720284 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242660.73 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 24266.07 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999325.793267 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218394.66 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 21839.47 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021165.258951 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196555.19 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 19655.52 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040820.778068 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176899.67 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 17689.97 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058510.745272 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159209.70 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 15920.97 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074431.715756 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143288.73 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 14328.87 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088760.589192 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128959.86 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 12895.99 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101656.575284 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116063.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 11606.39 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113262.962766 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104457.49 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 10445.75 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123708.711501 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94011.74 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 9401.17 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133109.885362 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84610.56 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 8461.06 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141570.941837 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76149.51 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 7614.95 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149185.892664 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68534.56 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 6853.46 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156039.348409 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61681.10 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 6168.11 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162207.458579 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55512.99 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 5551.30 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167758.757733 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49961.69 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 4996.17 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172754.926971 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44965.52 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 4496.55 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177251.479285 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40468.97 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 4046.90 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181298.376367 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36422.07 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3642.21 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184940.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32779.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188540.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29179.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192140.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25579.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195740.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21979.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199340.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18379.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202940.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14779.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206540.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11179.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210140.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7579.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213740.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3979.87 sec) 17:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217340.583742 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 379.87 sec) 17:21:00 [GPUGRID] [rr_sim] 2217720.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975306631.19G/439.78G) 17:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853221.36 17:21:00 [---] pick_jobs_to_run() start 17:21:00 [---] pick_jobs_to_run() end 17:21:00 [---] [rr_sim] end 17:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:21:00 [---] [cpu_sched_debug] preliminary job list: 17:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:21:00 [---] [cpu_sched_debug] final job list: 17:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:21:00 [---] [cpu_sched_debug] enforce_run_list: end 17:21:00 [---] choose_project(): 62460.000000 17:21:00 [WUProp@Home] doing RPC to get work 17:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373139.09 cpu 2120972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120972.90 17:21:00 [---] Got 0 tasks 17:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:22:00 [---] [rr_sim] doing sim: CPU sched 17:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975279040.83G/439.78G) 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217657.71 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 221765.77 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221765.771323 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995891.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 199589.19 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421354.965514 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796302.75 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 179630.27 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600985.240285 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616672.47 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 161667.25 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762652.487580 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1455005.23 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 145500.52 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908153.010145 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309504.70 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 130950.47 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039103.480453 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178554.23 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 117855.42 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156958.903731 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060698.81 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 106069.88 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1263028.784681 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954628.93 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 95462.89 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358491.677536 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859166.04 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 85916.60 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444408.281105 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773249.43 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 77324.94 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521733.224317 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695924.49 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 69592.45 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591325.673209 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626332.04 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 62633.20 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653958.877211 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563698.84 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 56369.88 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710328.760813 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507328.95 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 50732.90 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761061.656054 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456596.06 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 45659.61 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806721.261772 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410936.45 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 41093.65 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847814.906918 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369842.81 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 36984.28 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884799.187549 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332858.53 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 33285.85 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918085.040117 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299572.67 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 29957.27 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1948042.307428 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269615.41 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 26961.54 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975003.848008 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242653.87 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 24265.39 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999269.234530 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218388.48 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 21838.85 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021108.082400 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196549.63 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 19654.96 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040763.045483 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176894.67 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 17689.47 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058452.512258 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159205.20 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 15920.52 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074373.032355 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143284.68 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 14328.47 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088701.500443 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128956.21 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 12895.62 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101597.121721 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116060.59 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 11606.06 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113203.180872 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104454.53 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 10445.45 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123648.634108 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94009.08 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 9400.91 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133049.542020 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84608.17 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 8460.82 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141510.359141 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76147.35 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 7614.74 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149125.094550 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68532.62 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 6853.26 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155978.356418 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61679.36 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 6167.94 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162146.292099 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55511.42 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 5551.14 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167697.434212 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49960.28 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 4996.03 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172693.462114 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44964.25 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 4496.43 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177189.887225 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40467.83 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 4046.78 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181236.669826 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36421.04 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3642.10 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184878.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32778.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188478.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29178.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192078.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25578.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195678.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21978.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199278.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18378.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202878.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14778.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206478.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11178.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210078.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7578.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213678.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3978.94 sec) 17:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217278.774166 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378.94 sec) 17:22:00 [GPUGRID] [rr_sim] 2217657.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975279040.83G/439.78G) 17:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853218.63 17:22:00 [---] pick_jobs_to_run() start 17:22:00 [---] pick_jobs_to_run() end 17:22:00 [---] [rr_sim] end 17:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:22:00 [---] [cpu_sched_debug] preliminary job list: 17:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:22:00 [---] [cpu_sched_debug] final job list: 17:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:22:00 [---] [cpu_sched_debug] enforce_run_list: end 17:22:00 [---] choose_project(): 62520.000000 17:22:00 [WUProp@Home] doing RPC to get work 17:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373079.09 cpu 2120912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120912.90 17:22:00 [---] Got 0 tasks 17:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:23:00 [---] [rr_sim] doing sim: CPU sched 17:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975251450.48G/439.78G) 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217594.98 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 221759.50 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221759.497635 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995835.48 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 199583.55 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421343.045506 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796251.93 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 179625.19 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600968.238590 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616626.74 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 161662.67 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762630.912366 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454964.06 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 145496.41 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908127.318764 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309467.66 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 130946.77 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039074.084522 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178520.89 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 117852.09 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156926.173705 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060668.80 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 106066.88 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262993.053969 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954601.92 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 95460.19 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358453.246207 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859141.73 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 85914.17 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444367.419221 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773227.56 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 77322.76 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521690.174934 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695904.80 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 69590.48 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591280.655075 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626314.32 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 62631.43 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653912.087202 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563682.89 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 56368.29 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710280.376117 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507314.60 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 50731.46 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1761011.836140 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456583.14 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 45658.31 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806670.150161 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410924.83 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 41092.48 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847762.632779 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369832.34 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 36983.23 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884745.867136 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332849.11 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 33284.91 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1918030.778057 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299564.20 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 29956.42 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947987.197886 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269607.78 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 26960.78 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974947.975732 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242647.00 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 24264.70 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999212.675794 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218382.30 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 21838.23 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021050.905849 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196544.07 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 19654.41 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040705.312899 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176889.66 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 17688.97 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058394.279244 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159200.70 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 15920.07 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074314.348954 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143280.63 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 14328.06 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088642.411694 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128952.56 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 12895.26 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101537.668159 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116057.31 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 11605.73 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113143.398978 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104451.58 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 10445.16 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123588.556715 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94006.42 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 9400.64 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132989.198678 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84605.78 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 8460.58 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141449.776445 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76145.20 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 7614.52 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149064.296435 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68530.68 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 6853.07 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155917.364426 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61677.61 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 6167.76 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162085.125619 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55509.85 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 5550.99 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167636.110691 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49958.87 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 4995.89 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172631.997257 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44962.98 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 4496.30 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177128.295166 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40466.68 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 4046.67 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181174.963284 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36420.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3642.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184816.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32778.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188416.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29178.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192016.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25578.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195616.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21978.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199216.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18378.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202816.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14778.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206416.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11178.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210016.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7578.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213616.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3978.01 sec) 17:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217216.964591 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 378.01 sec) 17:23:00 [GPUGRID] [rr_sim] 2217594.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975251450.48G/439.78G) 17:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853215.89 17:23:00 [---] pick_jobs_to_run() start 17:23:00 [---] pick_jobs_to_run() end 17:23:00 [---] [rr_sim] end 17:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:23:00 [---] [cpu_sched_debug] preliminary job list: 17:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:23:00 [---] [cpu_sched_debug] final job list: 17:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:23:00 [---] [cpu_sched_debug] enforce_run_list: end 17:23:00 [---] choose_project(): 62580.000000 17:23:00 [WUProp@Home] doing RPC to get work 17:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 373019.09 cpu 2120852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120852.90 17:23:00 [---] Got 0 tasks 17:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:24:00 [---] [rr_sim] doing sim: CPU sched 17:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975223860.12G/439.78G) 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217532.24 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 221753.22 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221753.223947 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995779.02 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 199577.90 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421331.125498 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796201.11 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 179620.11 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600951.236895 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616581.00 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 161658.10 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762609.337152 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454922.90 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 145492.29 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908101.627383 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309430.61 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 130943.06 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039044.688592 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178487.55 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 117848.76 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156893.443679 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060638.80 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 106063.88 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262957.323258 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954574.92 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 95457.49 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358414.814878 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859117.42 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 85911.74 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444326.557337 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773205.68 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 77320.57 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521647.125550 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695885.11 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 69588.51 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591235.636941 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626296.60 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 62629.66 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653865.297194 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563666.94 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 56366.69 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710231.991421 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507300.25 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 50730.02 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760962.016225 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456570.22 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 45657.02 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806619.038549 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410913.20 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 41091.32 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847710.358641 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369821.88 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 36982.19 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884692.546723 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332839.69 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 33283.97 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917976.515998 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299555.72 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 29955.57 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947932.088344 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269600.15 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 26960.02 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974892.103456 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242640.14 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 24264.01 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999156.117057 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218376.12 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 21837.61 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020993.729298 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196538.51 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 19653.85 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040647.580315 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176884.66 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 17688.47 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058336.046230 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159196.19 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 15919.62 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074255.665553 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143276.57 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 14327.66 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088583.322945 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128948.92 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 12894.89 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101478.214597 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116054.02 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 11605.40 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113083.617084 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104448.62 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 10444.86 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123528.479322 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94003.76 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 9400.38 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132928.855336 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84603.38 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 8460.34 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141389.193749 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76143.05 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 7614.30 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149003.498321 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68528.74 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 6852.87 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155856.372435 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61675.87 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 6167.59 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162023.959138 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55508.28 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 5550.83 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167574.787171 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49957.45 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 4995.75 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172570.532400 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44961.71 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 4496.17 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177066.703107 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40465.54 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 4046.55 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181113.256743 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36418.98 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3641.90 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184755.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32777.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188355.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29177.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191955.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25577.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195555.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21977.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199155.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18377.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202755.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14777.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206355.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11177.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209955.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7577.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213555.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3977.08 sec) 17:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217155.155015 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 377.08 sec) 17:24:00 [GPUGRID] [rr_sim] 2217532.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975223860.12G/439.78G) 17:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853213.15 17:24:00 [---] pick_jobs_to_run() start 17:24:00 [---] pick_jobs_to_run() end 17:24:00 [---] [rr_sim] end 17:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:24:00 [---] [cpu_sched_debug] preliminary job list: 17:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:24:00 [---] [cpu_sched_debug] final job list: 17:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:24:00 [---] [cpu_sched_debug] enforce_run_list: end 17:24:00 [---] choose_project(): 62640.000000 17:24:00 [WUProp@Home] doing RPC to get work 17:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372959.09 cpu 2120792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120792.90 17:24:00 [---] Got 0 tasks 17:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:25:00 [---] [rr_sim] doing sim: CPU sched 17:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975196269.77G/439.78G) 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217469.50 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 221746.95 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221746.950258 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995722.55 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 199572.26 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421319.205491 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796150.30 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 179615.03 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600934.235200 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616535.27 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 161653.53 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762587.761938 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454881.74 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 145488.17 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908075.936003 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309393.57 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 130939.36 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1039015.292661 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178454.21 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 117845.42 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156860.713653 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060608.79 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 106060.88 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262921.592546 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954547.91 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 95454.79 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358376.383550 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859093.12 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 85909.31 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444285.695453 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773183.81 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 77318.38 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521604.076166 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695865.43 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 69586.54 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591190.618808 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626278.88 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 62627.89 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653818.507185 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563651.00 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 56365.10 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710183.606725 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507285.90 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 50728.59 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760912.196311 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456557.31 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 45655.73 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806567.926938 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410901.58 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 41090.16 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847658.084503 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369811.42 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 36981.14 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884639.226311 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332830.28 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 33283.03 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917922.253938 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299547.25 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 29954.72 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947876.978802 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269592.52 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 26959.25 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974836.231181 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242633.27 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 24263.33 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999099.558321 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218369.94 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 21836.99 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020936.552747 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196532.95 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 19653.29 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040589.847731 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176879.65 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 17687.97 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058277.813216 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159191.69 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 15919.17 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074196.982153 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143272.52 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 14327.25 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088524.234196 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128945.27 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 12894.53 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101418.761034 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116050.74 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 11605.07 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113023.835189 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104445.67 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 10444.57 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123468.401929 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94001.10 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 9400.11 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132868.511994 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84600.99 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 8460.10 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141328.611053 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76140.89 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 7614.09 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148942.700206 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68526.80 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 6852.68 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155795.380444 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61674.12 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 6167.41 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161962.792658 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55506.71 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 5550.67 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167513.463650 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49956.04 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 4995.60 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172509.067543 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44960.44 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 4496.04 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177005.111047 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40464.39 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 4046.44 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181051.550201 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36417.95 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3641.80 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184693.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32776.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188293.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29176.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191893.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25576.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195493.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21976.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199093.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18376.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202693.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14776.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206293.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11176.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209893.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7576.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213493.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3976.16 sec) 17:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217093.345439 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 376.16 sec) 17:25:00 [GPUGRID] [rr_sim] 2217469.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975196269.77G/439.78G) 17:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853210.42 17:25:00 [---] pick_jobs_to_run() start 17:25:00 [---] pick_jobs_to_run() end 17:25:00 [---] [rr_sim] end 17:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:25:00 [---] [cpu_sched_debug] preliminary job list: 17:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:25:00 [---] [cpu_sched_debug] final job list: 17:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:25:00 [---] [cpu_sched_debug] enforce_run_list: end 17:25:00 [---] choose_project(): 62700.000000 17:25:00 [WUProp@Home] doing RPC to get work 17:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372899.09 cpu 2120732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120732.90 17:25:00 [---] Got 0 tasks 17:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:26:00 [---] [rr_sim] doing sim: CPU sched 17:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975168679.41G/439.78G) 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217406.77 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 221740.68 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221740.676570 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995666.09 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 199566.61 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421307.285483 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796099.48 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 179609.95 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600917.233505 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616489.53 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 161648.95 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762566.186725 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454840.58 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 145484.06 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908050.244622 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309356.52 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 130935.65 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038985.896730 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178420.87 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 117842.09 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156827.983627 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060578.78 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 106057.88 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262885.861835 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954520.90 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 95452.09 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358337.952221 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859068.81 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 85906.88 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444244.833569 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773161.93 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 77316.19 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521561.026782 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695845.74 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 69584.57 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591145.600674 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626261.17 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 62626.12 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653771.717177 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563635.05 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 56363.50 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710135.222029 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507271.54 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 50727.15 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760862.376396 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456544.39 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 45654.44 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806516.815327 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410889.95 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 41089.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847605.810364 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369800.96 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 36980.10 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884585.905898 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332820.86 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 33282.09 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917867.991878 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299538.77 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 29953.88 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947821.869261 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269584.90 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 26958.49 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974780.358905 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242626.41 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 24262.64 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999042.999584 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218363.77 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 21836.38 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020879.376196 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196527.39 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 19652.74 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040532.115146 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176874.65 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 17687.47 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058219.580202 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159187.19 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 15918.72 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074138.298752 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143268.47 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 14326.85 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088465.145447 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128941.62 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 12894.16 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101359.307472 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116047.46 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 11604.75 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112964.053295 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104442.71 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 10444.27 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123408.324536 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93998.44 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 9399.84 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132808.168652 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84598.60 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 8459.86 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141268.028357 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76138.74 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 7613.87 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148881.902091 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68524.86 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 6852.49 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155734.388452 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61672.38 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 6167.24 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161901.626177 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55505.14 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 5550.51 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167452.140130 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49954.63 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 4995.46 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172447.602687 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44959.16 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 4495.92 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176943.518988 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40463.25 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 4046.32 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180989.843659 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36416.92 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3641.69 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184631.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32775.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188231.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29175.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191831.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25575.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195431.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21975.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199031.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18375.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202631.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14775.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206231.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11175.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209831.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7575.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213431.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3975.23 sec) 17:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2217031.535864 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 375.23 sec) 17:26:00 [GPUGRID] [rr_sim] 2217406.77: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975168679.41G/439.78G) 17:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853207.68 17:26:00 [---] pick_jobs_to_run() start 17:26:00 [---] pick_jobs_to_run() end 17:26:00 [---] [rr_sim] end 17:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:26:00 [---] [cpu_sched_debug] preliminary job list: 17:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:26:00 [---] [cpu_sched_debug] final job list: 17:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:26:00 [---] [cpu_sched_debug] enforce_run_list: end 17:26:00 [---] choose_project(): 62760.000000 17:26:00 [WUProp@Home] doing RPC to get work 17:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372839.09 cpu 2120672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120672.90 17:26:00 [---] Got 0 tasks 17:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:27:00 [---] [rr_sim] doing sim: CPU sched 17:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975141089.05G/439.78G) 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217344.03 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 221734.40 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221734.402882 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995609.63 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 199560.96 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421295.365476 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1796048.66 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 179604.87 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600900.231810 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616443.80 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 161644.38 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762544.611511 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454799.42 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 145479.94 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 908024.553242 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309319.48 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 130931.95 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038956.500799 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178387.53 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 117838.75 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156795.253601 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060548.78 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 106054.88 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262850.131123 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954493.90 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 95449.39 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358299.520893 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859044.51 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 85904.45 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444203.971685 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773140.06 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 77314.01 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521517.977399 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695826.05 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 69582.61 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591100.582541 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626243.45 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 62624.34 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653724.927168 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563619.10 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 56361.91 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710086.837333 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507257.19 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 50725.72 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760812.556482 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456531.47 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 45653.15 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806465.703716 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410878.33 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 41087.83 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847553.536226 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369790.49 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 36979.05 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884532.585485 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332811.44 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 33281.14 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917813.729819 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299530.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 29953.03 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947766.759719 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269577.27 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 26957.73 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974724.486629 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242619.54 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 24261.95 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998986.440848 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218357.59 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 21835.76 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020822.199645 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196521.83 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 19652.18 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040474.382562 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176869.65 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 17686.96 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058161.347188 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159182.68 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 15918.27 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074079.615351 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143264.41 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 14326.44 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088406.056698 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128937.97 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 12893.80 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101299.853910 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116044.17 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 11604.42 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112904.271401 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104439.76 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 10443.98 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123348.247143 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93995.78 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 9399.58 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132747.825310 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84596.20 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 8459.62 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141207.445661 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76136.58 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 7613.66 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148821.103977 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68522.92 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 6852.29 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155673.396461 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61670.63 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 6167.06 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161840.459697 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55503.57 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 5550.36 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167390.816609 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49953.21 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 4995.32 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172386.137830 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44957.89 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 4495.79 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176881.926929 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40462.10 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 4046.21 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180928.137118 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36415.89 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3641.59 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184569.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32774.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188169.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29174.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191769.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25574.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195369.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21974.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198969.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18374.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202569.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14774.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206169.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11174.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209769.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7574.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213369.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3974.30 sec) 17:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216969.726288 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 374.30 sec) 17:27:00 [GPUGRID] [rr_sim] 2217344.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975141089.05G/439.78G) 17:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853204.94 17:27:00 [---] pick_jobs_to_run() start 17:27:00 [---] pick_jobs_to_run() end 17:27:00 [---] [rr_sim] end 17:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:27:00 [---] [cpu_sched_debug] preliminary job list: 17:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:27:00 [---] [cpu_sched_debug] final job list: 17:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:27:00 [---] [cpu_sched_debug] enforce_run_list: end 17:27:00 [---] choose_project(): 62820.000000 17:27:00 [WUProp@Home] doing RPC to get work 17:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372779.09 cpu 2120612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120612.90 17:27:00 [---] Got 0 tasks 17:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:28:00 [---] [rr_sim] doing sim: CPU sched 17:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975113498.70G/439.78G) 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217281.29 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 221728.13 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221728.129194 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995553.16 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 199555.32 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421283.445468 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795997.85 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 179599.78 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600883.230115 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616398.06 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 161639.81 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762523.036297 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454758.26 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 145475.83 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907998.861861 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309282.43 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 130928.24 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038927.104869 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178354.19 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 117835.42 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156762.523575 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060518.77 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 106051.88 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262814.400411 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954466.89 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 95446.69 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358261.089564 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 859020.20 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 85902.02 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444163.109801 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773118.18 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 77311.82 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521474.928015 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695806.36 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 69580.64 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591055.564407 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626225.73 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 62622.57 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653678.137160 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563603.15 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 56360.32 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1710038.452638 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507242.84 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 50724.28 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760762.736567 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456518.56 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 45651.86 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806414.592104 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410866.70 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 41086.67 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847501.262088 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369780.03 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 36978.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884479.265073 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332802.03 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 33280.20 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917759.467759 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299521.82 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 29952.18 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947711.650177 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269569.64 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 26956.96 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974668.614353 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242612.68 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 24261.27 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998929.882111 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218351.41 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 21835.14 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020765.023094 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196516.27 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 19651.63 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040416.649978 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176864.64 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 17686.46 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058103.114174 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159178.18 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 15917.82 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074020.931950 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143260.36 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 14326.04 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088346.967949 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128934.32 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 12893.43 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101240.400348 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116040.89 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 11604.09 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112844.489506 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104436.80 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 10443.68 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123288.169749 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93993.12 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 9399.31 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132687.481968 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84593.81 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 8459.38 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141146.862965 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76134.43 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 7613.44 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148760.305862 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68520.99 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 6852.10 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155612.404470 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61668.89 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 6166.89 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161779.293216 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55502.00 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 5550.20 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167329.493088 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49951.80 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 4995.18 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172324.672973 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44956.62 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 4495.66 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176820.334870 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40460.96 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 4046.10 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180866.430576 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36414.86 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3641.49 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184507.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32773.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188107.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29173.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191707.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25573.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195307.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21973.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198907.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18373.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202507.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14773.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206107.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11173.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209707.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7573.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213307.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3973.38 sec) 17:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216907.916712 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 373.38 sec) 17:28:00 [GPUGRID] [rr_sim] 2217281.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975113498.70G/439.78G) 17:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853202.20 17:28:00 [---] pick_jobs_to_run() start 17:28:00 [---] pick_jobs_to_run() end 17:28:00 [---] [rr_sim] end 17:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:28:00 [---] [cpu_sched_debug] preliminary job list: 17:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:28:00 [---] [cpu_sched_debug] final job list: 17:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:28:00 [---] [cpu_sched_debug] enforce_run_list: end 17:28:00 [---] choose_project(): 62880.000000 17:28:00 [WUProp@Home] doing RPC to get work 17:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372719.09 cpu 2120552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120552.90 17:28:00 [---] Got 0 tasks 17:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:29:00 [---] [rr_sim] doing sim: CPU sched 17:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975085908.34G/439.78G) 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217218.56 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 221721.86 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221721.855505 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995496.70 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 199549.67 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421271.525460 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795947.03 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 179594.70 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600866.228420 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616352.33 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 161635.23 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762501.461083 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454717.09 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 145471.71 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907973.170480 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309245.38 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 130924.54 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038897.708938 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178320.85 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 117832.08 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156729.793549 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060488.76 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 106048.88 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262778.669700 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954439.89 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 95443.99 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358222.658235 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858995.90 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 85899.59 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444122.247917 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773096.31 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 77309.63 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521431.878631 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695786.68 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 69578.67 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1591010.546273 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626208.01 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 62620.80 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653631.347151 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563587.21 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 56358.72 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709990.067942 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507228.49 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 50722.85 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760712.916653 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456505.64 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 45650.56 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806363.480493 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410855.07 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 41085.51 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847448.987949 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369769.57 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 36976.96 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884425.944660 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332792.61 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 33279.26 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917705.205699 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299513.35 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 29951.33 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947656.540635 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269562.01 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 26956.20 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974612.742077 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242605.81 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 24260.58 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998873.323375 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218345.23 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 21834.52 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020707.846543 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196510.71 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 19651.07 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040358.917394 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176859.64 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 17685.96 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058044.881160 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159173.67 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 15917.37 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073962.248549 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143256.31 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 14325.63 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088287.879200 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128930.68 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 12893.07 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101180.946785 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116037.61 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 11603.76 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112784.707612 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104433.85 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 10443.38 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123228.092356 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93990.46 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 9399.05 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132627.138626 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84591.42 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 8459.14 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141086.280269 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76132.27 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 7613.23 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148699.507748 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68519.05 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 6851.90 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155551.412478 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61667.14 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 6166.71 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161718.126736 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55500.43 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 5550.04 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167268.169568 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49950.39 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 4995.04 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172263.208116 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44955.35 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 4495.53 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176758.742810 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40459.81 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 4045.98 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180804.724035 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36413.83 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3641.38 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184446.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32772.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188046.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29172.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191646.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25572.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195246.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21972.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198846.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18372.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202446.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14772.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206046.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11172.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209646.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7572.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213246.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3972.45 sec) 17:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216846.107137 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 372.45 sec) 17:29:00 [GPUGRID] [rr_sim] 2217218.56: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975085908.34G/439.78G) 17:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853199.47 17:29:00 [---] pick_jobs_to_run() start 17:29:00 [---] pick_jobs_to_run() end 17:29:00 [---] [rr_sim] end 17:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:29:00 [---] [cpu_sched_debug] preliminary job list: 17:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:29:00 [---] [cpu_sched_debug] final job list: 17:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:29:00 [---] [cpu_sched_debug] enforce_run_list: end 17:29:00 [---] choose_project(): 62940.000000 17:29:00 [WUProp@Home] doing RPC to get work 17:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372659.09 cpu 2120492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120492.90 17:29:00 [---] Got 0 tasks 17:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:30:00 [---] [rr_sim] doing sim: CPU sched 17:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975058317.99G/439.78G) 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217155.82 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 221715.58 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221715.581817 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995440.24 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 199544.02 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421259.605453 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795896.21 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 179589.62 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600849.226725 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616306.59 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 161630.66 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762479.885869 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454675.93 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 145467.59 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907947.479100 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309208.34 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 130920.83 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038868.313007 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178287.51 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 117828.75 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156697.063524 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060458.75 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 106045.88 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262742.938988 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954412.88 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 95441.29 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358184.226907 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858971.59 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 85897.16 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444081.386033 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773074.43 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 77307.44 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521388.829247 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695766.99 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 69576.70 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590965.528140 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626190.29 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 62619.03 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653584.557143 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563571.26 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 56357.13 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709941.683246 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507214.13 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 50721.41 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760663.096739 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456492.72 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 45649.27 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806312.368882 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410843.45 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 41084.34 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847396.713811 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369759.10 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 36975.91 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884372.624247 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332783.19 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 33278.32 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917650.943640 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299504.87 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 29950.49 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947601.431093 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269554.39 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 26955.44 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974556.869801 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242598.95 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 24259.89 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998816.764638 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218339.05 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 21833.91 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020650.669991 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196505.15 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 19650.51 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040301.184809 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176854.63 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 17685.46 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057986.648146 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159169.17 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 15916.92 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073903.565148 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143252.25 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 14325.23 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088228.790451 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128927.03 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 12892.70 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101121.493223 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116034.32 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 11603.43 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112724.925718 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104430.89 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 10443.09 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123168.014963 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93987.80 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 9398.78 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132566.795284 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84589.02 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 8458.90 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141025.697573 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76130.12 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 7613.01 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148638.709633 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68517.11 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 6851.71 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155490.420487 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61665.40 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 6166.54 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161656.960255 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55498.86 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 5549.89 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167206.846047 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49948.97 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 4994.90 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172201.743260 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44954.07 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 4495.41 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176697.150751 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40458.67 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 4045.87 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180743.017493 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36412.80 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3641.28 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184384.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32771.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187984.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29171.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191584.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25571.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195184.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21971.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198784.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18371.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202384.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14771.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205984.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11171.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209584.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7571.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213184.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3971.52 sec) 17:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216784.297561 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 371.52 sec) 17:30:00 [GPUGRID] [rr_sim] 2217155.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975058317.99G/439.78G) 17:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853196.73 17:30:00 [---] pick_jobs_to_run() start 17:30:00 [---] pick_jobs_to_run() end 17:30:00 [---] [rr_sim] end 17:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:30:00 [---] [cpu_sched_debug] preliminary job list: 17:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:30:00 [---] [cpu_sched_debug] final job list: 17:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:30:00 [---] [cpu_sched_debug] enforce_run_list: end 17:30:00 [---] choose_project(): 63000.000000 17:30:00 [WUProp@Home] doing RPC to get work 17:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372599.09 cpu 2120432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120432.90 17:30:00 [---] Got 0 tasks 17:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:31:00 [---] [rr_sim] doing sim: CPU sched 17:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975030727.63G/439.78G) 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217093.08 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 221709.31 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221709.308129 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995383.77 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 199538.38 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421247.685445 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795845.40 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 179584.54 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600832.225030 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616260.86 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 161626.09 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762458.310656 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454634.77 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 145463.48 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907921.787719 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309171.29 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 130917.13 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038838.917076 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178254.16 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 117825.42 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156664.333498 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060428.75 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 106042.87 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262707.208277 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954385.87 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 95438.59 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358145.795578 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858947.29 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 85894.73 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1444040.524149 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773052.56 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 77305.26 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521345.779863 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695747.30 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 69574.73 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590920.510006 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626172.57 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 62617.26 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653537.767135 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563555.31 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 56355.53 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709893.298550 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507199.78 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 50719.98 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760613.276824 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456479.80 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 45647.98 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806261.257271 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410831.82 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 41083.18 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847344.439673 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369748.64 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 36974.86 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884319.303834 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332773.78 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 33277.38 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917596.681580 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299496.40 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 29949.64 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947546.321551 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269546.76 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 26954.68 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974500.997525 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242592.08 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 24259.21 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998760.205901 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218332.88 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 21833.29 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020593.493440 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196499.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 19649.96 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040243.452225 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176849.63 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 17684.96 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057928.415132 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159164.67 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 15916.47 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073844.881748 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143248.20 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 14324.82 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088169.701702 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128923.38 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 12892.34 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101062.039661 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116031.04 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 11603.10 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112665.143824 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104427.94 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 10442.79 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123107.937570 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93985.14 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 9398.51 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132506.451942 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84586.63 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 8458.66 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140965.114877 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76127.97 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 7612.80 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148577.911518 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68515.17 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 6851.52 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155429.428496 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61663.65 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 6166.37 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161595.793775 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55497.29 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 5549.73 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167145.522526 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49947.56 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 4994.76 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172140.278403 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44952.80 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 4495.28 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176635.558692 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40457.52 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 4045.75 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180681.310951 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36411.77 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3641.18 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184322.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32770.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187922.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29170.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191522.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25570.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195122.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21970.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198722.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18370.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202322.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14770.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205922.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11170.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209522.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7570.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213122.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3970.59 sec) 17:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216722.487985 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 370.59 sec) 17:31:00 [GPUGRID] [rr_sim] 2217093.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975030727.63G/439.78G) 17:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853193.99 17:31:00 [---] pick_jobs_to_run() start 17:31:00 [---] pick_jobs_to_run() end 17:31:00 [---] [rr_sim] end 17:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:31:00 [---] [cpu_sched_debug] preliminary job list: 17:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:31:00 [---] [cpu_sched_debug] final job list: 17:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:31:00 [---] [cpu_sched_debug] enforce_run_list: end 17:31:00 [---] choose_project(): 63060.000000 17:31:00 [WUProp@Home] doing RPC to get work 17:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372539.09 cpu 2120372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120372.90 17:31:00 [---] Got 0 tasks 17:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:32:00 [---] [rr_sim] doing sim: CPU sched 17:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (975003137.27G/439.78G) 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2217030.34 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 221703.03 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221703.034441 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995327.31 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 199532.73 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421235.765438 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795794.58 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 179579.46 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600815.223335 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616215.12 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 161621.51 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762436.735442 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454593.61 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 145459.36 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907896.096339 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309134.25 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 130913.42 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038809.521145 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178220.82 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 117822.08 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156631.603472 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060398.74 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 106039.87 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262671.477565 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954358.87 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 95435.89 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358107.364250 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858922.98 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 85892.30 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443999.662265 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773030.68 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 77303.07 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521302.730480 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695727.61 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 69572.76 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590875.491873 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626154.85 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 62615.49 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653490.977126 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563539.37 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 56353.94 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709844.913854 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507185.43 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 50718.54 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760563.456910 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456466.89 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 45646.69 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806210.145659 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410820.20 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 41082.02 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847292.165534 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369738.18 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 36973.82 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884265.983422 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332764.36 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 33276.44 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917542.419520 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299487.92 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 29948.79 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947491.212009 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269539.13 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 26953.91 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974445.125249 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242585.22 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 24258.52 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998703.647165 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218326.70 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 21832.67 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020536.316889 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196494.03 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 19649.40 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040185.719641 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176844.62 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 17684.46 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057870.182118 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159160.16 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 15916.02 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073786.198347 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143244.15 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 14324.41 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088110.612953 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128919.73 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 12891.97 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101002.586098 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116027.76 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 11602.78 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112605.361929 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104424.98 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 10442.50 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123047.860177 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93982.48 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 9398.25 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132446.108600 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84584.24 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 8458.42 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140904.532181 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76125.81 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 7612.58 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148517.113404 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68513.23 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 6851.32 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155368.436504 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61661.91 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 6166.19 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161534.627295 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55495.72 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 5549.57 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167084.199006 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49946.15 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 4994.61 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172078.813546 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44951.53 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 4495.15 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176573.966632 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40456.38 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 4045.64 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180619.604410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36410.74 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3641.07 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184260.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32769.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187860.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29169.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191460.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25569.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195060.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21969.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198660.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18369.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202260.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14769.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205860.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11169.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209460.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7569.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213060.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3969.67 sec) 17:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216660.678410 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369.67 sec) 17:32:00 [GPUGRID] [rr_sim] 2217030.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (975003137.27G/439.78G) 17:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853191.26 17:32:00 [---] pick_jobs_to_run() start 17:32:00 [---] pick_jobs_to_run() end 17:32:00 [---] [rr_sim] end 17:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:32:00 [---] [cpu_sched_debug] preliminary job list: 17:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:32:00 [---] [cpu_sched_debug] final job list: 17:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:32:00 [---] [cpu_sched_debug] enforce_run_list: end 17:32:00 [---] choose_project(): 63120.000000 17:32:00 [WUProp@Home] doing RPC to get work 17:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372479.09 cpu 2120312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120312.90 17:32:00 [---] Got 0 tasks 17:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:33:00 [---] [rr_sim] doing sim: CPU sched 17:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974975546.92G/439.78G) 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216967.61 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 221696.76 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221696.760753 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995270.85 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 199527.08 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421223.845430 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795743.76 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 179574.38 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600798.221639 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616169.39 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 161616.94 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762415.160228 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454552.45 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 145455.24 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907870.404958 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309097.20 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 130909.72 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038780.125215 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178187.48 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 117818.75 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156598.873446 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060368.73 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 106036.87 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262635.746854 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954331.86 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 95433.19 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358068.932921 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858898.67 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 85889.87 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443958.800381 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 773008.81 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 77300.88 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521259.681096 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695707.93 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 69570.79 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590830.473739 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626137.13 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 62613.71 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653444.187118 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563523.42 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 56352.34 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709796.529158 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507171.08 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 50717.11 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760513.636995 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456453.97 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 45645.40 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806159.034048 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410808.57 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 41080.86 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847239.891396 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369727.72 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 36972.77 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884212.663009 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332754.94 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 33275.49 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917488.157461 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299479.45 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 29947.95 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947436.102467 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269531.51 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 26953.15 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974389.252973 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242578.35 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 24257.84 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998647.088428 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218320.52 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 21832.05 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020479.140338 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196488.47 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 19648.85 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040127.987057 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176839.62 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 17683.96 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057811.949104 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159155.66 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 15915.57 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073727.514946 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143240.09 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 14324.01 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088051.524204 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128916.08 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 12891.61 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100943.132536 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116024.47 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 11602.45 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112545.580035 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104422.03 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 10442.20 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122987.782784 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93979.82 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 9397.98 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132385.765258 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84581.84 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 8458.18 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140843.949485 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76123.66 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 7612.37 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148456.315289 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68511.29 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 6851.13 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155307.444513 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61660.16 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 6166.02 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161473.460814 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55494.15 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 5549.41 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167022.875485 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49944.73 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 4994.47 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172017.348689 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44950.26 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 4495.03 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176512.374573 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40455.23 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 4045.52 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180557.897868 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36409.71 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3640.97 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184198.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32768.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187798.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29168.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191398.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25568.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194998.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21968.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198598.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18368.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202198.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14768.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205798.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11168.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209398.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7568.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212998.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3968.74 sec) 17:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216598.868834 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368.74 sec) 17:33:00 [GPUGRID] [rr_sim] 2216967.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974975546.92G/439.78G) 17:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853188.52 17:33:00 [---] pick_jobs_to_run() start 17:33:00 [---] pick_jobs_to_run() end 17:33:00 [---] [rr_sim] end 17:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:33:00 [---] [cpu_sched_debug] preliminary job list: 17:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:33:00 [---] [cpu_sched_debug] final job list: 17:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:33:00 [---] [cpu_sched_debug] enforce_run_list: end 17:33:00 [---] choose_project(): 63180.000000 17:33:00 [WUProp@Home] doing RPC to get work 17:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372419.09 cpu 2120252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120252.90 17:33:00 [---] Got 0 tasks 17:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:34:00 [---] [rr_sim] doing sim: CPU sched 17:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974947956.56G/439.78G) 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216904.87 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 221690.49 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221690.487064 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995214.38 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 199521.44 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421211.925422 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795692.95 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 179569.29 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600781.219944 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616123.65 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 161612.37 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762393.585014 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454511.29 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 145451.13 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907844.713577 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309060.16 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 130906.02 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038750.729284 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178154.14 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 117815.41 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156566.143420 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060338.73 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 106033.87 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262600.016142 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954304.85 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 95430.49 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1358030.501592 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858874.37 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 85887.44 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443917.938498 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772986.93 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 77298.69 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521216.631712 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695688.24 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 69568.82 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590785.455605 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626119.42 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 62611.94 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653397.397109 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563507.47 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 56350.75 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709748.144463 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507156.73 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 50715.67 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760463.817081 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456441.05 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 45644.11 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806107.922437 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410796.95 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 41079.69 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847187.617258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369717.25 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 36971.73 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884159.342596 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332745.53 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 33274.55 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917433.895401 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299470.98 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 29947.10 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947380.992925 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269523.88 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 26952.39 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974333.380697 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242571.49 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 24257.15 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998590.529692 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218314.34 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 21831.43 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020421.963787 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196482.91 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 19648.29 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040070.254473 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176834.62 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 17683.46 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057753.716090 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159151.15 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 15915.12 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073668.831545 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143236.04 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 14323.60 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087992.435455 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128912.44 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 12891.24 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100883.678974 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116021.19 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 11602.12 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112485.798141 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104419.07 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 10441.91 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122927.705391 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93977.17 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 9397.72 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132325.421916 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84579.45 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 8457.94 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140783.366789 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76121.50 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 7612.15 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148395.517175 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68509.35 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 6850.94 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155246.452521 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61658.42 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 6165.84 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161412.294334 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55492.58 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 5549.26 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166961.551965 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49943.32 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 4994.33 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171955.883833 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44948.99 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 4494.90 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176450.782514 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40454.09 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 4045.41 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180496.191327 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36408.68 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3640.87 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184137.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32767.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187737.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29167.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191337.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25567.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194937.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21967.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198537.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18367.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202137.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14767.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205737.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11167.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209337.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7567.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212937.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3967.81 sec) 17:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216537.059258 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367.81 sec) 17:34:00 [GPUGRID] [rr_sim] 2216904.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974947956.56G/439.78G) 17:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853185.78 17:34:00 [---] pick_jobs_to_run() start 17:34:00 [---] pick_jobs_to_run() end 17:34:00 [---] [rr_sim] end 17:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:34:00 [---] [cpu_sched_debug] preliminary job list: 17:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:34:00 [---] [cpu_sched_debug] final job list: 17:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:34:00 [---] [cpu_sched_debug] enforce_run_list: end 17:34:00 [---] choose_project(): 63240.000000 17:34:00 [WUProp@Home] doing RPC to get work 17:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372359.09 cpu 2120192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120192.90 17:34:00 [---] Got 0 tasks 17:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:35:00 [---] [rr_sim] doing sim: CPU sched 17:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974920366.20G/439.78G) 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216842.13 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 221684.21 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221684.213376 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995157.92 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 199515.79 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421200.005415 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795642.13 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 179564.21 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600764.218249 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616077.92 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 161607.79 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762372.009801 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454470.12 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 145447.01 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907819.022197 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1309023.11 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 130902.31 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038721.333353 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178120.80 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 117812.08 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156533.413394 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060308.72 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 106030.87 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262564.285431 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954277.85 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 95427.78 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357992.070264 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858850.06 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 85885.01 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443877.076614 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772965.06 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 77296.51 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521173.582328 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695668.55 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 69566.86 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590740.437472 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626101.70 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 62610.17 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653350.607101 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563491.53 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 56349.15 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709699.759767 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507142.37 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 50714.24 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760413.997166 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456428.14 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 45642.81 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806056.810826 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410785.32 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 41078.53 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847135.343119 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369706.79 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 36970.68 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884106.022184 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332736.11 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 33273.61 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917379.633341 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299462.50 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 29946.25 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947325.883383 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269516.25 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 26951.63 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974277.508421 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242564.63 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 24256.46 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998533.970955 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218308.16 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 21830.82 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020364.787236 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196477.35 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 19647.73 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040012.521888 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176829.61 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 17682.96 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057695.483076 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159146.65 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 15914.67 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073610.148144 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143231.99 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 14323.20 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087933.346706 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128908.79 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 12890.88 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100824.225412 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116017.91 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 11601.79 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112426.016247 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104416.12 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 10441.61 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122867.627998 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93974.51 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 9397.45 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132265.078574 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84577.06 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 8457.71 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140722.784093 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76119.35 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 7611.93 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148334.719060 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68507.41 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 6850.74 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155185.460530 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61656.67 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 6165.67 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161351.127853 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55491.01 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 5549.10 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166900.228444 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49941.91 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 4994.19 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171894.418976 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44947.71 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 4494.77 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176389.190454 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40452.94 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 4045.29 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180434.484785 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36407.65 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3640.76 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184075.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32766.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187675.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29166.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191275.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25566.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194875.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21966.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198475.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18366.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202075.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14766.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205675.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11166.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209275.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7566.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212875.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3966.88 sec) 17:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216475.249683 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366.88 sec) 17:35:00 [GPUGRID] [rr_sim] 2216842.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974920366.20G/439.78G) 17:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853183.05 17:35:00 [---] pick_jobs_to_run() start 17:35:00 [---] pick_jobs_to_run() end 17:35:00 [---] [rr_sim] end 17:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:35:00 [---] [cpu_sched_debug] preliminary job list: 17:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:35:00 [---] [cpu_sched_debug] final job list: 17:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:35:00 [---] [cpu_sched_debug] enforce_run_list: end 17:35:00 [---] choose_project(): 63300.000000 17:35:00 [WUProp@Home] doing RPC to get work 17:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372299.09 cpu 2120132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120132.90 17:35:00 [---] Got 0 tasks 17:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:36:00 [---] [rr_sim] doing sim: CPU sched 17:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974892775.85G/439.78G) 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216779.40 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 221677.94 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221677.939688 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995101.46 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 199510.15 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421188.085407 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795591.31 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 179559.13 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600747.216554 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1616032.18 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 161603.22 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762350.434587 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454428.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 145442.90 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907793.330816 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308986.07 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 130898.61 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038691.937422 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178087.46 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 117808.75 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156500.683368 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060278.71 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 106027.87 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262528.554719 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954250.84 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 95425.08 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357953.638935 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858825.76 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 85882.58 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443836.214730 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772943.18 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 77294.32 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521130.532945 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695648.86 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 69564.89 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590695.419338 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626083.98 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 62608.40 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653303.817092 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563475.58 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 56347.56 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709651.375071 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507128.02 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 50712.80 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760364.177252 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456415.22 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 45641.52 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1806005.699214 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410773.70 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 41077.37 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847083.068981 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369696.33 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 36969.63 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1884052.701771 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332726.70 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 33272.67 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917325.371282 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299454.03 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 29945.40 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947270.773841 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269508.62 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 26950.86 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974221.636145 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242557.76 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 24255.78 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998477.412219 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218301.98 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 21830.20 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020307.610685 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196471.79 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 19647.18 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039954.789304 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176824.61 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 17682.46 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057637.250062 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159142.15 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 15914.21 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073551.464743 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143227.93 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 14322.79 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087874.257957 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128905.14 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 12890.51 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100764.771849 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116014.63 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 11601.46 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112366.234352 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104413.16 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 10441.32 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122807.550605 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93971.85 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 9397.18 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132204.735232 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84574.66 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 8457.47 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140662.201397 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76117.20 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 7611.72 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148273.920945 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68505.48 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 6850.55 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155124.468539 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61654.93 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 6165.49 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161289.961373 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55489.44 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 5548.94 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166838.904923 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49940.49 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 4994.05 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171832.954119 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44946.44 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 4494.64 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176327.598395 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40451.80 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 4045.18 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180372.778243 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36406.62 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3640.66 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184013.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32765.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187613.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29165.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191213.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25565.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194813.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21965.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198413.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18365.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202013.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14765.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205613.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11165.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209213.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7565.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212813.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3965.96 sec) 17:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216413.440107 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365.96 sec) 17:36:00 [GPUGRID] [rr_sim] 2216779.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974892775.85G/439.78G) 17:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853180.31 17:36:00 [---] pick_jobs_to_run() start 17:36:00 [---] pick_jobs_to_run() end 17:36:00 [---] [rr_sim] end 17:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:36:00 [---] [cpu_sched_debug] preliminary job list: 17:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:36:00 [---] [cpu_sched_debug] final job list: 17:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:36:00 [---] [cpu_sched_debug] enforce_run_list: end 17:36:00 [---] choose_project(): 63360.000000 17:36:00 [WUProp@Home] doing RPC to get work 17:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372239.09 cpu 2120072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120072.90 17:36:00 [---] Got 0 tasks 17:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:37:00 [---] [rr_sim] doing sim: CPU sched 17:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974865185.49G/439.78G) 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216716.66 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 221671.67 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221671.666000 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1995044.99 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 199504.50 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421176.165399 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795540.49 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 179554.05 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600730.214859 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615986.45 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 161598.64 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762328.859373 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454387.80 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 145438.78 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907767.639435 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308949.02 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 130894.90 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038662.541492 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178054.12 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 117805.41 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156467.953342 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060248.71 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 106024.87 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262492.824008 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954223.84 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 95422.38 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357915.207607 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858801.45 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 85880.15 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443795.352846 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772921.31 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 77292.13 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521087.483561 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695629.18 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 69562.92 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590650.401204 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626066.26 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 62606.63 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653257.027084 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563459.63 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 56345.96 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709602.990375 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507113.67 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 50711.37 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760314.357337 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456402.30 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 45640.23 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805954.587603 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410762.07 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 41076.21 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1847030.794843 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369685.87 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 36968.59 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883999.381358 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332717.28 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 33271.73 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917271.109222 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299445.55 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 29944.56 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947215.664299 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269501.00 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 26950.10 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974165.763869 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242550.90 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 24255.09 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998420.853482 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218295.81 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 21829.58 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020250.434134 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196466.23 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 19646.62 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039897.056720 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176819.60 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 17681.96 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057579.017048 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159137.64 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 15913.76 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073492.781343 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143223.88 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 14322.39 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087815.169208 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128901.49 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 12890.15 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100705.318287 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116011.34 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 11601.13 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112306.452458 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104410.21 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 10441.02 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122747.473212 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93969.19 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 9396.92 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132144.391890 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84572.27 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 8457.23 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140601.618701 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76115.04 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 7611.50 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148213.122831 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68503.54 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 6850.35 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155063.476547 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61653.18 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 6165.32 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161228.794892 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55487.87 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 5548.79 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166777.581403 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49939.08 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 4993.91 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171771.489262 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44945.17 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 4494.52 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176266.006336 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40450.65 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 4045.07 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180311.071702 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36405.59 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3640.56 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183951.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32765.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187551.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29165.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191151.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25565.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194751.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21965.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198351.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18365.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201951.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14765.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205551.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11165.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209151.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7565.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212751.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3965.03 sec) 17:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216351.630531 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365.03 sec) 17:37:00 [GPUGRID] [rr_sim] 2216716.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974865185.49G/439.78G) 17:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853177.57 17:37:00 [---] pick_jobs_to_run() start 17:37:00 [---] pick_jobs_to_run() end 17:37:00 [---] [rr_sim] end 17:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:37:00 [---] [cpu_sched_debug] preliminary job list: 17:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:37:00 [---] [cpu_sched_debug] final job list: 17:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:37:00 [---] [cpu_sched_debug] enforce_run_list: end 17:37:00 [---] choose_project(): 63420.000000 17:37:00 [WUProp@Home] doing RPC to get work 17:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372179.09 cpu 2120012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2120012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2120012.90 17:37:00 [---] Got 0 tasks 17:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:38:00 [---] [rr_sim] doing sim: CPU sched 17:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974837595.14G/439.78G) 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216653.92 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 221665.39 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221665.392311 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994988.53 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 199498.85 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421164.245392 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795489.68 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 179548.97 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600713.213164 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615940.71 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 161594.07 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762307.284159 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454346.64 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 145434.66 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907741.948055 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308911.98 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 130891.20 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038633.145561 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1178020.78 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 117802.08 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156435.223316 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060218.70 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 106021.87 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262457.093296 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954196.83 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 95419.68 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357876.776278 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858777.15 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 85877.71 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443754.490962 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772899.43 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 77289.94 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521044.434177 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695609.49 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 69560.95 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590605.383071 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626048.54 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 62604.85 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653210.237075 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563443.69 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 56344.37 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709554.605679 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507099.32 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 50709.93 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760264.537423 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456389.39 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 45638.94 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805903.475992 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410750.45 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 41075.04 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846978.520704 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369675.40 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 36967.54 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883946.060945 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332707.86 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 33270.79 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917216.847162 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299437.08 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 29943.71 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947160.554758 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269493.37 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 26949.34 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974109.891593 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242544.03 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 24254.40 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998364.294745 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218289.63 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 21828.96 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020193.257582 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196460.67 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 19646.07 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039839.324136 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176814.60 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 17681.46 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057520.784034 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159133.14 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 15913.31 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073434.097942 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143219.83 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 14321.98 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087756.080459 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128897.84 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 12889.78 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100645.864725 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116008.06 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 11600.81 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112246.670564 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104407.25 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 10440.73 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122687.395819 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93966.53 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 9396.65 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132084.048548 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84569.87 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 8456.99 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140541.036005 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76112.89 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 7611.29 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148152.324716 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68501.60 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 6850.16 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155002.484556 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61651.44 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 6165.14 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161167.628412 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55486.29 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 5548.63 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166716.257882 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49937.67 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 4993.77 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171710.024405 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44943.90 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 4494.39 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176204.414276 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40449.51 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 4044.95 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180249.365160 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36404.56 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3640.46 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183889.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32764.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187489.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29164.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191089.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25564.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194689.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21964.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198289.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18364.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201889.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14764.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205489.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11164.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209089.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7564.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212689.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3964.10 sec) 17:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216289.820956 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 364.10 sec) 17:38:00 [GPUGRID] [rr_sim] 2216653.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974837595.14G/439.78G) 17:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853174.84 17:38:00 [---] pick_jobs_to_run() start 17:38:00 [---] pick_jobs_to_run() end 17:38:00 [---] [rr_sim] end 17:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:38:00 [---] [cpu_sched_debug] preliminary job list: 17:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:38:00 [---] [cpu_sched_debug] final job list: 17:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:38:00 [---] [cpu_sched_debug] enforce_run_list: end 17:38:00 [---] choose_project(): 63480.000000 17:38:00 [WUProp@Home] doing RPC to get work 17:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372119.09 cpu 2119952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119952.90 17:38:00 [---] Got 0 tasks 17:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:39:00 [---] [rr_sim] doing sim: CPU sched 17:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974810004.78G/439.78G) 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216591.19 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 221659.12 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221659.118623 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994932.07 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 199493.21 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421152.325384 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795438.86 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 179543.89 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600696.211469 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615894.97 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 161589.50 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762285.708945 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454305.48 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 145430.55 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907716.256674 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308874.93 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 130887.49 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038603.749630 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177987.44 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 117798.74 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156402.493290 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060188.69 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 106018.87 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262421.362585 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954169.82 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 95416.98 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357838.344949 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858752.84 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 85875.28 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443713.629078 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772877.56 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 77287.76 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1521001.384793 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695589.80 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 69558.98 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590560.364937 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626030.82 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 62603.08 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653163.447067 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563427.74 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 56342.77 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709506.220983 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507084.97 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 50708.50 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760214.717508 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456376.47 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 45637.65 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805852.364381 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410738.82 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 41073.88 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846926.246566 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369664.94 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 36966.49 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883892.740533 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332698.45 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 33269.84 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917162.585103 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299428.60 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 29942.86 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947105.445216 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269485.74 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 26948.57 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1974054.019317 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242537.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 24253.72 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998307.736009 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218283.45 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 21828.35 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020136.081031 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196455.11 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 19645.51 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039781.591551 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176809.59 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 17680.96 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057462.551020 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159128.64 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 15912.86 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073375.414541 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143215.77 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 14321.58 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087696.991710 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128894.19 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 12889.42 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100586.411162 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116004.78 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 11600.48 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112186.888669 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104404.30 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 10440.43 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122627.318426 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93963.87 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 9396.39 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132023.705206 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84567.48 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 8456.75 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140480.453309 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76110.73 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 7611.07 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148091.526601 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68499.66 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 6849.97 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154941.492565 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61649.69 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 6164.97 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161106.461931 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55484.72 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 5548.47 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166654.934362 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49936.25 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 4993.63 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171648.559549 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44942.63 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 4494.26 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176142.822217 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40448.36 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 4044.84 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180187.658619 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36403.53 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3640.35 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183828.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32763.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187428.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29163.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191028.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25563.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194628.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21963.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198228.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18363.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201828.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14763.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205428.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11163.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209028.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7563.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212628.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3963.17 sec) 17:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216228.011380 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 363.17 sec) 17:39:00 [GPUGRID] [rr_sim] 2216591.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974810004.78G/439.78G) 17:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853172.10 17:39:00 [---] pick_jobs_to_run() start 17:39:00 [---] pick_jobs_to_run() end 17:39:00 [---] [rr_sim] end 17:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:39:00 [---] [cpu_sched_debug] preliminary job list: 17:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:39:00 [---] [cpu_sched_debug] final job list: 17:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:39:00 [---] [cpu_sched_debug] enforce_run_list: end 17:39:00 [---] choose_project(): 63540.000000 17:39:00 [WUProp@Home] doing RPC to get work 17:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 372059.09 cpu 2119892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119892.90 17:39:00 [---] Got 0 tasks 17:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:40:00 [---] [rr_sim] doing sim: CPU sched 17:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974782414.42G/439.78G) 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216528.45 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 221652.84 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221652.844935 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994875.60 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 199487.56 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421140.405377 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795388.04 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 179538.80 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600679.209774 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615849.24 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 161584.92 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762264.133732 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454264.32 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 145426.43 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907690.565294 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308837.88 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 130883.79 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038574.353699 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177954.10 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 117795.41 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156369.763264 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060158.69 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 106015.87 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262385.631873 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954142.82 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 95414.28 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357799.913621 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858728.54 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 85872.85 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443672.767194 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772855.68 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 77285.57 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520958.335409 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695570.11 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 69557.01 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590515.346804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 626013.10 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 62601.31 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653116.657058 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563411.79 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 56341.18 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709457.836288 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507070.61 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 50707.06 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760164.897594 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456363.55 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 45636.36 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805801.252770 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410727.20 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 41072.72 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846873.972428 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369654.48 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 36965.45 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883839.420120 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332689.03 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 33268.90 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917108.323043 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299420.13 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 29942.01 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1947050.335674 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269478.11 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 26947.81 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973998.147041 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242530.30 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 24253.03 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998251.177272 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218277.27 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 21827.73 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020078.904480 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196449.54 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 19644.95 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039723.858967 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176804.59 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 17680.46 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057404.318006 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159124.13 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 15912.41 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073316.731140 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143211.72 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 14321.17 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087637.902961 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128890.55 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 12889.05 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100526.957600 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116001.49 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 11600.15 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112127.106775 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104401.34 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 10440.13 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122567.241033 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93961.21 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 9396.12 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131963.361864 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84565.09 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 8456.51 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140419.870613 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76108.58 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 7610.86 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148030.728487 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68497.72 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 6849.77 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154880.500573 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61647.95 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 6164.79 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161045.295451 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55483.15 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 5548.32 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166593.610841 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49934.84 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 4993.48 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171587.094692 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44941.35 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 4494.14 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176081.230158 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40447.22 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 4044.72 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180125.952077 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36402.50 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3640.25 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183766.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32762.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187366.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29162.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190966.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25562.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194566.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21962.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198166.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18362.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201766.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14762.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205366.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11162.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208966.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7562.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212566.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3962.25 sec) 17:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216166.201804 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 362.25 sec) 17:40:00 [GPUGRID] [rr_sim] 2216528.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974782414.42G/439.78G) 17:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853169.36 17:40:00 [---] pick_jobs_to_run() start 17:40:00 [---] pick_jobs_to_run() end 17:40:00 [---] [rr_sim] end 17:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:40:00 [---] [cpu_sched_debug] preliminary job list: 17:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:40:00 [---] [cpu_sched_debug] final job list: 17:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:40:00 [---] [cpu_sched_debug] enforce_run_list: end 17:40:00 [---] choose_project(): 63600.000000 17:40:00 [WUProp@Home] doing RPC to get work 17:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371999.09 cpu 2119832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119832.90 17:40:00 [---] Got 0 tasks 17:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:41:00 [---] [rr_sim] doing sim: CPU sched 17:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974754824.07G/439.78G) 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216465.71 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 221646.57 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221646.571247 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994819.14 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 199481.91 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421128.485369 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795337.23 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 179533.72 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600662.208079 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615803.50 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 161580.35 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762242.558518 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454223.15 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 145422.32 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907664.873913 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308800.84 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 130880.08 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038544.957769 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177920.75 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 117792.08 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156337.033239 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060128.68 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 106012.87 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262349.901162 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954115.81 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 95411.58 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357761.482292 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858704.23 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 85870.42 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443631.905310 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772833.81 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 77283.38 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520915.286026 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695550.43 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 69555.04 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590470.328670 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625995.38 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 62599.54 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653069.867050 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563395.85 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 56339.58 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709409.451592 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507056.26 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 50705.63 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760115.077679 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456350.63 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 45635.06 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805750.141158 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410715.57 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 41071.56 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846821.698289 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369644.01 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 36964.40 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883786.099707 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332679.61 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 33267.96 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1917054.060983 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299411.65 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 29941.17 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946995.226132 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269470.49 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 26947.05 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973942.274766 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242523.44 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 24252.34 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998194.618536 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218271.09 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 21827.11 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020021.727929 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196443.98 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 19644.40 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039666.126383 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176799.59 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 17679.96 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057346.084992 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159119.63 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 15911.96 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073258.047739 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143207.66 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 14320.77 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087578.814212 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128886.90 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 12888.69 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100467.504038 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115998.21 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 11599.82 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112067.324881 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104398.39 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 10439.84 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122507.163640 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93958.55 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 9395.85 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131903.018523 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84562.69 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 8456.27 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140359.287917 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76106.42 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 7610.64 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147969.930372 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68495.78 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 6849.58 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154819.508582 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61646.20 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 6164.62 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160984.128971 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55481.58 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 5548.16 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166532.287320 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49933.43 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 4993.34 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171525.629835 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44940.08 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 4494.01 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176019.638098 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40446.07 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 4044.61 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180064.245535 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36401.47 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3640.15 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183704.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32761.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187304.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29161.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190904.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25561.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194504.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21961.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198104.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18361.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201704.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14761.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205304.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11161.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208904.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7561.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212504.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3961.32 sec) 17:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216104.392229 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 361.32 sec) 17:41:00 [GPUGRID] [rr_sim] 2216465.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974754824.07G/439.78G) 17:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853166.63 17:41:00 [---] pick_jobs_to_run() start 17:41:00 [---] pick_jobs_to_run() end 17:41:00 [---] [rr_sim] end 17:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:41:00 [---] [cpu_sched_debug] preliminary job list: 17:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:41:00 [---] [cpu_sched_debug] final job list: 17:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:41:00 [---] [cpu_sched_debug] enforce_run_list: end 17:41:00 [---] choose_project(): 63660.000000 17:41:00 [WUProp@Home] doing RPC to get work 17:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371939.09 cpu 2119772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119772.90 17:41:00 [---] Got 0 tasks 17:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:42:00 [---] [rr_sim] doing sim: CPU sched 17:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974727233.71G/439.78G) 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216402.98 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 221640.30 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221640.297559 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994762.68 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 199476.27 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421116.565361 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795286.41 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 179528.64 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600645.206384 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615757.77 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 161575.78 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762220.983304 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454181.99 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 145418.20 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907639.182532 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308763.79 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 130876.38 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038515.561838 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177887.41 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 117788.74 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156304.303213 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060098.67 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 106009.87 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262314.170450 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954088.81 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 95408.88 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357723.050964 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858679.92 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 85867.99 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443591.043426 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772811.93 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 77281.19 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520872.236642 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695530.74 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 69553.07 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590425.310536 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625977.67 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 62597.77 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1653023.077041 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563379.90 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 56337.99 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709361.066896 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507041.91 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 50704.19 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760065.257765 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456337.72 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 45633.77 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805699.029547 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410703.95 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 41070.39 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846769.424151 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369633.55 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 36963.36 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883732.779294 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332670.20 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 33267.02 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916999.798924 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299403.18 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 29940.32 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946940.116590 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269462.86 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 26946.29 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973886.402490 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242516.57 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 24251.66 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998138.059799 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218264.92 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 21826.49 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019964.551378 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196438.42 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 19643.84 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039608.393799 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176794.58 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 17679.46 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057287.851978 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159115.12 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 15911.51 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073199.364338 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143203.61 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 14320.36 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087519.725463 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128883.25 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 12888.33 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100408.050476 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115994.93 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 11599.49 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112007.542987 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104395.43 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 10439.54 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122447.086247 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93955.89 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 9395.59 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131842.675181 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84560.30 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 8456.03 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140298.705221 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76104.27 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 7610.43 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147909.132258 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68493.84 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 6849.38 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154758.516591 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61644.46 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 6164.45 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160922.962490 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55480.01 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 5548.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166470.963800 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49932.01 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 4993.20 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171464.164978 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44938.81 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 4493.88 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175958.046039 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40444.93 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 4044.49 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180002.538994 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36400.44 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3640.04 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183642.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32760.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187242.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29160.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190842.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25560.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194442.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21960.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198042.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18360.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201642.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14760.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205242.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11160.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208842.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7560.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212442.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3960.39 sec) 17:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2216042.582653 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 360.39 sec) 17:42:00 [GPUGRID] [rr_sim] 2216402.98: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974727233.71G/439.78G) 17:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853163.89 17:42:00 [---] pick_jobs_to_run() start 17:42:00 [---] pick_jobs_to_run() end 17:42:00 [---] [rr_sim] end 17:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:42:00 [---] [cpu_sched_debug] preliminary job list: 17:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:42:00 [---] [cpu_sched_debug] final job list: 17:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:42:00 [---] [cpu_sched_debug] enforce_run_list: end 17:42:00 [---] choose_project(): 63720.000000 17:42:00 [WUProp@Home] doing RPC to get work 17:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371879.09 cpu 2119712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119712.90 17:42:00 [---] Got 0 tasks 17:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:43:00 [---] [rr_sim] doing sim: CPU sched 17:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974699643.35G/439.78G) 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216340.24 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 221634.02 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221634.023870 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994706.21 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 199470.62 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421104.645354 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795235.59 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 179523.56 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600628.204689 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615712.03 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 161571.20 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762199.408090 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454140.83 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 145414.08 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907613.491152 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308726.75 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 130872.67 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038486.165907 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177854.07 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 117785.41 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156271.573187 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060068.67 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 106006.87 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262278.439738 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954061.80 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 95406.18 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357684.619635 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858655.62 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 85865.56 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443550.181542 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772790.06 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 77279.01 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520829.187258 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695511.05 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 69551.11 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590380.292403 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625959.95 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 62595.99 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652976.287033 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563363.95 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 56336.40 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709312.682200 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507027.56 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 50702.76 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1760015.437850 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456324.80 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 45632.48 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805647.917936 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410692.32 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 41069.23 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846717.150013 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369623.09 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 36962.31 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883679.458882 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332660.78 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 33266.08 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916945.536864 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299394.70 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 29939.47 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946885.007048 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269455.23 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 26945.52 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973830.530214 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242509.71 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 24250.97 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998081.501063 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218258.74 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 21825.87 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019907.374827 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196432.86 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 19643.29 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039550.661215 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176789.58 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 17678.96 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057229.618964 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159110.62 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 15911.06 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073140.680938 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143199.56 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 14319.96 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087460.636714 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128879.60 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 12887.96 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100348.596913 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115991.64 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 11599.16 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111947.761092 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104392.48 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 10439.25 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122387.008854 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93953.23 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 9395.32 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131782.331839 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84557.91 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 8455.79 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140238.122525 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76102.12 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 7610.21 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147848.334143 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68491.90 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 6849.19 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154697.524599 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61642.71 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 6164.27 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160861.796010 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55478.44 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 5547.84 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166409.640279 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49930.60 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 4993.06 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171402.700122 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44937.54 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 4493.75 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175896.453980 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40443.78 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 4044.38 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179940.832452 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36399.41 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3639.94 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183580.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32759.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187180.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29159.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190780.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25559.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194380.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21959.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197980.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18359.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201580.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14759.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205180.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11159.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208780.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7559.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212380.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3959.47 sec) 17:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215980.773077 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 359.47 sec) 17:43:00 [GPUGRID] [rr_sim] 2216340.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974699643.35G/439.78G) 17:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853161.15 17:43:00 [---] pick_jobs_to_run() start 17:43:00 [---] pick_jobs_to_run() end 17:43:00 [---] [rr_sim] end 17:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:43:00 [---] [cpu_sched_debug] preliminary job list: 17:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:43:00 [---] [cpu_sched_debug] final job list: 17:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:43:00 [---] [cpu_sched_debug] enforce_run_list: end 17:43:00 [---] choose_project(): 63780.000000 17:43:00 [WUProp@Home] doing RPC to get work 17:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371819.09 cpu 2119652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119652.90 17:43:00 [---] Got 0 tasks 17:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:44:00 [---] [rr_sim] doing sim: CPU sched 17:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974672053.00G/439.78G) 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216277.50 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 221627.75 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221627.750182 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994649.75 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 199464.98 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421092.725346 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795184.78 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 179518.48 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600611.202994 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615666.30 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 161566.63 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762177.832877 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454099.67 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 145409.97 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907587.799771 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308689.70 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 130868.97 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038456.769976 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177820.73 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 117782.07 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156238.843161 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060038.66 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 106003.87 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262242.709027 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954034.79 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 95403.48 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357646.188306 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858631.31 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 85863.13 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443509.319658 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772768.18 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 77276.82 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520786.137874 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695491.36 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 69549.14 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590335.274269 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625942.23 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 62594.22 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652929.497024 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563348.00 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 56334.80 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709264.297504 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 507013.20 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 50701.32 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759965.617936 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456311.88 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 45631.19 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805596.806325 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410680.70 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 41068.07 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846664.875874 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369612.63 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 36961.26 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883626.138469 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332651.36 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 33265.14 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916891.274804 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299386.23 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 29938.62 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946829.897506 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269447.60 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 26944.76 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973774.657938 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242502.84 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 24250.28 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998024.942326 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218252.56 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 21825.26 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019850.198276 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196427.30 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 19642.73 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039492.928630 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176784.57 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 17678.46 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057171.385950 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159106.12 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 15910.61 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073081.997537 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143195.50 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 14319.55 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087401.547965 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128875.95 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 12887.60 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100289.143351 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115988.36 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 11598.84 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111887.979198 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104389.52 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 10438.95 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122326.931460 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93950.57 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 9395.06 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131721.988497 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84555.51 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 8455.55 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140177.539829 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76099.96 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 7610.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147787.536028 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68489.97 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 6849.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154636.532608 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61640.97 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 6164.10 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160800.629529 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55476.87 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 5547.69 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166348.316758 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49929.19 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 4992.92 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171341.235265 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44936.27 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 4493.63 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175834.861921 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40442.64 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 4044.26 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179879.125911 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36398.38 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3639.84 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183518.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32758.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187118.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29158.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190718.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25558.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194318.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21958.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197918.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18358.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201518.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14758.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205118.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11158.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208718.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7558.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212318.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3958.54 sec) 17:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215918.963502 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 358.54 sec) 17:44:00 [GPUGRID] [rr_sim] 2216277.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974672053.00G/439.78G) 17:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853158.41 17:44:00 [---] pick_jobs_to_run() start 17:44:00 [---] pick_jobs_to_run() end 17:44:00 [---] [rr_sim] end 17:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:44:00 [---] [cpu_sched_debug] preliminary job list: 17:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:44:00 [---] [cpu_sched_debug] final job list: 17:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:44:00 [---] [cpu_sched_debug] enforce_run_list: end 17:44:00 [---] choose_project(): 63840.000000 17:44:00 [WUProp@Home] doing RPC to get work 17:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371759.09 cpu 2119592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119592.90 17:44:00 [---] Got 0 tasks 17:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:45:00 [---] [rr_sim] doing sim: CPU sched 17:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974644462.64G/439.78G) 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216214.76 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 221621.48 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221621.476494 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994593.29 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 199459.33 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421080.805339 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795133.96 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 179513.40 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600594.201299 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615620.56 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 161562.06 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762156.257663 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454058.51 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 145405.85 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907562.108391 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308652.66 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 130865.27 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038427.374045 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177787.39 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 117778.74 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156206.113135 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1060008.65 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 106000.87 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262206.978315 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 954007.79 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 95400.78 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357607.756978 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858607.01 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 85860.70 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443468.457774 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772746.31 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 77274.63 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520743.088491 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695471.68 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 69547.17 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590290.256136 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625924.51 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 62592.45 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652882.707016 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563332.06 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 56333.21 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709215.912808 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506998.85 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 50699.89 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759915.798021 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456298.97 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 45629.90 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805545.694713 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410669.07 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 41066.91 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846612.601736 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369602.16 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 36960.22 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883572.818056 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332641.95 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 33264.19 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916837.012745 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299377.75 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 29937.78 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946774.787964 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269439.98 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 26944.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973718.785662 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242495.98 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 24249.60 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997968.383590 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218246.38 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 21824.64 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019793.021725 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196421.74 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 19642.17 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039435.196046 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176779.57 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 17677.96 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057113.152935 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159101.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 15910.16 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073023.314136 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143191.45 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 14319.15 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087342.459216 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128872.31 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 12887.23 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100229.689789 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115985.08 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 11598.51 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111828.197304 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104386.57 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 10438.66 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122266.854067 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93947.91 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 9394.79 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131661.645155 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84553.12 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 8455.31 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140116.957133 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76097.81 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 7609.78 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147726.737914 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68488.03 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 6848.80 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154575.540616 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61639.22 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 6163.92 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160739.463049 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55475.30 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 5547.53 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166286.993238 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49927.77 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 4992.78 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171279.770408 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44934.99 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 4493.50 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175773.269861 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40441.50 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 4044.15 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179817.419369 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36397.35 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3639.73 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183457.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32757.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187057.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29157.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190657.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25557.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194257.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21957.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197857.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18357.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201457.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14757.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205057.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11157.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208657.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7557.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212257.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3957.61 sec) 17:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215857.153926 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 357.61 sec) 17:45:00 [GPUGRID] [rr_sim] 2216214.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974644462.64G/439.78G) 17:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853155.68 17:45:00 [---] pick_jobs_to_run() start 17:45:00 [---] pick_jobs_to_run() end 17:45:00 [---] [rr_sim] end 17:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:45:00 [---] [cpu_sched_debug] preliminary job list: 17:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:45:00 [---] [cpu_sched_debug] final job list: 17:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:45:00 [---] [cpu_sched_debug] enforce_run_list: end 17:45:00 [---] choose_project(): 63900.000000 17:45:00 [WUProp@Home] doing RPC to get work 17:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371699.09 cpu 2119532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119532.90 17:45:00 [---] Got 0 tasks 17:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:46:00 [---] [rr_sim] doing sim: CPU sched 17:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974616872.29G/439.78G) 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216152.03 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 221615.20 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221615.202806 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994536.83 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 199453.68 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421068.885331 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795083.14 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 179508.31 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600577.199604 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615574.83 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 161557.48 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762134.682449 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1454017.35 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 145401.73 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907536.417010 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308615.61 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 130861.56 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038397.978115 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177754.05 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 117775.40 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156173.383109 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059978.64 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 105997.86 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262171.247604 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953980.78 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 95398.08 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357569.325649 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858582.70 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 85858.27 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443427.595890 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772724.43 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 77272.44 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520700.039107 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695451.99 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 69545.20 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590245.238002 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625906.79 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 62590.68 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652835.917007 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563316.11 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 56331.61 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709167.528112 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506984.50 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 50698.45 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759865.978107 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456286.05 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 45628.60 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805494.583102 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410657.44 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 41065.74 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846560.327598 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369591.70 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 36959.17 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883519.497644 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332632.53 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 33263.25 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916782.750685 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299369.28 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 29936.93 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946719.678422 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269432.35 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 26943.23 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973662.913386 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242489.11 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 24248.91 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997911.824853 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218240.20 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 21824.02 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019735.845173 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196416.18 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 19641.62 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039377.463462 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176774.56 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 17677.46 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057054.919921 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159097.11 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 15909.71 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072964.630735 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143187.40 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 14318.74 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087283.370467 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128868.66 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 12886.87 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100170.236226 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115981.79 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 11598.18 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111768.415409 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104383.61 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 10438.36 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122206.776674 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93945.25 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 9394.53 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131601.301813 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84550.73 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 8455.07 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140056.374437 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76095.65 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 7609.57 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147665.939799 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68486.09 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 6848.61 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154514.548625 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61637.48 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 6163.75 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160678.296568 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55473.73 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 5547.37 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166225.669717 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49926.36 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 4992.64 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171218.305551 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44933.72 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 4493.37 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175711.677802 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40440.35 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 4044.04 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179755.712827 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36396.32 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3639.63 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183395.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32756.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186995.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29156.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190595.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25556.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194195.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21956.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197795.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18356.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201395.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14756.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204995.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11156.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208595.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7556.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212195.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3956.68 sec) 17:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215795.344351 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 356.68 sec) 17:46:00 [GPUGRID] [rr_sim] 2216152.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974616872.29G/439.78G) 17:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853152.94 17:46:00 [---] pick_jobs_to_run() start 17:46:00 [---] pick_jobs_to_run() end 17:46:00 [---] [rr_sim] end 17:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:46:00 [---] [cpu_sched_debug] preliminary job list: 17:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:46:00 [---] [cpu_sched_debug] final job list: 17:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:46:00 [---] [cpu_sched_debug] enforce_run_list: end 17:46:00 [---] choose_project(): 63960.000000 17:46:00 [WUProp@Home] doing RPC to get work 17:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371639.09 cpu 2119472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119472.90 17:46:00 [---] Got 0 tasks 17:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:47:00 [---] [rr_sim] doing sim: CPU sched 17:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974589281.93G/439.78G) 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216089.29 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 221608.93 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221608.929118 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994480.36 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 199448.04 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421056.965323 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1795032.33 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 179503.23 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600560.197909 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615529.09 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 161552.91 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762113.107235 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453976.18 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 145397.62 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907510.725629 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308578.57 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 130857.86 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038368.582184 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177720.71 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 117772.07 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156140.653083 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059948.64 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 105994.86 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262135.516892 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953953.77 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 95395.38 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357530.894321 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858558.40 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 85855.84 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443386.734006 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772702.56 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 77270.26 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520656.989723 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695432.30 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 69543.23 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590200.219868 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625889.07 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 62588.91 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652789.126999 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563300.16 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 56330.02 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709119.143417 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506970.15 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 50697.01 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759816.158193 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456273.13 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 45627.31 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805443.471491 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410645.82 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 41064.58 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846508.053459 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369581.24 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 36958.12 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883466.177231 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332623.11 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 33262.31 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916728.488625 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299360.80 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 29936.08 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946664.568880 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269424.72 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 26942.47 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973607.041110 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242482.25 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 24248.23 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997855.266116 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218234.03 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 21823.40 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019678.668622 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196410.62 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 19641.06 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039319.730878 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176769.56 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 17676.96 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056996.686907 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159092.60 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 15909.26 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072905.947334 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143183.34 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 14318.33 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087224.281718 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128865.01 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 12886.50 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100110.782664 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115978.51 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 11597.85 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111708.633515 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104380.66 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 10438.07 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122146.699281 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93942.59 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 9394.26 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131540.958471 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84548.33 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 8454.83 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139995.791741 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76093.50 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 7609.35 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147605.141685 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68484.15 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 6848.41 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154453.556634 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61635.73 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 6163.57 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160617.130088 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55472.16 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 5547.22 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166164.346197 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49924.94 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 4992.49 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171156.840695 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44932.45 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 4493.25 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175650.085743 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40439.21 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 4043.92 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179694.006286 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36395.28 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3639.53 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183333.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32755.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186933.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29155.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190533.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25555.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194133.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21955.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197733.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18355.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201333.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14755.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204933.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11155.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208533.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7555.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212133.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3955.76 sec) 17:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215733.534775 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 355.76 sec) 17:47:00 [GPUGRID] [rr_sim] 2216089.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974589281.93G/439.78G) 17:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853150.20 17:47:00 [---] pick_jobs_to_run() start 17:47:00 [---] pick_jobs_to_run() end 17:47:00 [---] [rr_sim] end 17:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:47:00 [---] [cpu_sched_debug] preliminary job list: 17:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:47:00 [---] [cpu_sched_debug] final job list: 17:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:47:00 [---] [cpu_sched_debug] enforce_run_list: end 17:47:00 [---] choose_project(): 64020.000000 17:47:00 [WUProp@Home] doing RPC to get work 17:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371579.09 cpu 2119412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119412.90 17:47:00 [---] Got 0 tasks 17:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:48:00 [---] [rr_sim] doing sim: CPU sched 17:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974561691.57G/439.78G) 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2216026.55 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 221602.66 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221602.655429 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994423.90 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 199442.39 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421045.045316 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794981.51 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 179498.15 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600543.196213 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615483.36 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 161548.34 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762091.532021 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453935.02 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 145393.50 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907485.034249 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308541.52 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 130854.15 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038339.186253 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177687.37 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 117768.74 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156107.923057 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059918.63 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 105991.86 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262099.786181 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953926.77 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 95392.68 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357492.462992 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858534.09 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 85853.41 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443345.872122 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772680.68 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 77268.07 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520613.940339 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695412.61 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 69541.26 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590155.201735 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625871.35 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 62587.14 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652742.336991 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563284.22 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 56328.42 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709070.758721 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506955.80 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 50695.58 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759766.338278 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456260.22 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 45626.02 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805392.359880 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410634.19 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 41063.42 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846455.779321 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369570.77 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 36957.08 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883412.856818 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332613.70 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 33261.37 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916674.226566 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299352.33 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 29935.23 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946609.459338 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269417.09 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 26941.71 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973551.168834 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242475.39 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 24247.54 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997798.707380 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218227.85 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 21822.78 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019621.492071 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196405.06 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 19640.51 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039261.998293 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176764.56 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 17676.46 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056938.453893 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159088.10 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 15908.81 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072847.263933 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143179.29 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 14317.93 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087165.192969 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128861.36 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 12886.14 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100051.329102 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115975.23 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 11597.52 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111648.851621 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104377.70 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 10437.77 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122086.621888 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93939.93 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 9393.99 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131480.615129 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84545.94 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 8454.59 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139935.209045 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76091.35 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 7609.13 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147544.343570 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68482.21 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 6848.22 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154392.564642 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61633.99 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 6163.40 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160555.963607 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55470.59 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 5547.06 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166103.022676 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49923.53 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 4992.35 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171095.375838 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44931.18 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 4493.12 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175588.493683 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40438.06 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 4043.81 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179632.299744 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36394.25 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3639.43 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183271.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32754.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186871.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29154.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190471.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25554.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194071.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21954.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197671.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18354.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201271.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14754.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204871.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11154.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208471.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7554.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212071.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3954.83 sec) 17:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215671.725199 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 354.83 sec) 17:48:00 [GPUGRID] [rr_sim] 2216026.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974561691.57G/439.78G) 17:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853147.47 17:48:00 [---] pick_jobs_to_run() start 17:48:00 [---] pick_jobs_to_run() end 17:48:00 [---] [rr_sim] end 17:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:48:00 [---] [cpu_sched_debug] preliminary job list: 17:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:48:00 [---] [cpu_sched_debug] final job list: 17:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:48:00 [---] [cpu_sched_debug] enforce_run_list: end 17:48:00 [---] choose_project(): 64080.000000 17:48:00 [WUProp@Home] doing RPC to get work 17:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371519.09 cpu 2119352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119352.90 17:48:00 [---] Got 0 tasks 17:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:49:00 [---] [rr_sim] doing sim: CPU sched 17:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974534101.22G/439.78G) 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215963.82 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 221596.38 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221596.381741 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994367.44 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 199436.74 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421033.125308 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794930.69 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 179493.07 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600526.194518 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615437.62 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 161543.76 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762069.956808 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453893.86 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 145389.39 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907459.342868 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308504.47 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 130850.45 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038309.790322 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177654.03 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 117765.40 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156075.193031 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059888.62 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 105988.86 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262064.055469 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953899.76 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 95389.98 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357454.031663 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858509.79 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 85850.98 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443305.010238 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772658.81 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 77265.88 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520570.890955 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695392.93 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 69539.29 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590110.183601 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625853.63 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 62585.36 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652695.546982 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563268.27 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 56326.83 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1709022.374025 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506941.44 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 50694.14 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759716.518364 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456247.30 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 45624.73 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805341.248268 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410622.57 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 41062.26 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846403.505183 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369560.31 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 36956.03 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883359.536405 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332604.28 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 33260.43 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916619.964506 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299343.85 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 29934.39 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946554.349797 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269409.47 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 26940.95 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973495.296558 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242468.52 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 24246.85 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997742.148643 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218221.67 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 21822.17 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019564.315520 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196399.50 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 19639.95 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039204.265709 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176759.55 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 17675.96 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056880.220879 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159083.60 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 15908.36 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072788.580533 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143175.24 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 14317.52 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087106.104220 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128857.71 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 12885.77 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099991.875540 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115971.94 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 11597.19 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111589.069727 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104374.75 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 10437.47 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122026.544495 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93937.27 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 9393.73 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131420.271787 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84543.55 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 8454.35 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139874.626349 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76089.19 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 7608.92 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147483.545455 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68480.27 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 6848.03 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154331.572651 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61632.24 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 6163.22 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160494.797127 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55469.02 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 5546.90 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166041.699155 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49922.12 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 4992.21 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171033.910981 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44929.91 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 4492.99 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175526.901624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40436.92 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 4043.69 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179570.593203 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36393.22 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3639.32 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183209.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32753.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186809.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29153.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190409.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25553.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194009.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21953.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197609.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18353.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201209.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14753.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204809.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11153.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208409.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7553.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212009.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3953.90 sec) 17:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215609.915624 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 353.90 sec) 17:49:00 [GPUGRID] [rr_sim] 2215963.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974534101.22G/439.78G) 17:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853144.73 17:49:00 [---] pick_jobs_to_run() start 17:49:00 [---] pick_jobs_to_run() end 17:49:00 [---] [rr_sim] end 17:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:49:00 [---] [cpu_sched_debug] preliminary job list: 17:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:49:00 [---] [cpu_sched_debug] final job list: 17:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:49:00 [---] [cpu_sched_debug] enforce_run_list: end 17:49:00 [---] choose_project(): 64140.000000 17:49:00 [WUProp@Home] doing RPC to get work 17:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371459.09 cpu 2119292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119292.90 17:49:00 [---] Got 0 tasks 17:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:50:00 [---] [rr_sim] doing sim: CPU sched 17:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974506510.86G/439.78G) 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215901.08 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 221590.11 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221590.108053 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994310.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 199431.10 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421021.205301 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794879.88 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 179487.99 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600509.192823 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615391.89 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 161539.19 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762048.381594 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453852.70 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 145385.27 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907433.651487 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308467.43 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 130846.74 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038280.394392 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177620.69 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 117762.07 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156042.463005 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059858.62 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 105985.86 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1262028.324758 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953872.76 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 95387.28 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357415.600335 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858485.48 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 85848.55 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443264.148354 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772636.93 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 77263.69 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520527.841572 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695373.24 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 69537.32 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590065.165467 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625835.92 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 62583.59 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652648.756974 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563252.32 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 56325.23 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708973.989329 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506927.09 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 50692.71 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759666.698449 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456234.38 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 45623.44 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805290.136657 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410610.94 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 41061.09 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846351.231044 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369549.85 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 36954.98 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883306.215993 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332594.86 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 33259.49 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916565.702446 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299335.38 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 29933.54 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946499.240255 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269401.84 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 26940.18 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973439.424282 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242461.66 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 24246.17 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997685.589907 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218215.49 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 21821.55 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019507.138969 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196393.94 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 19639.39 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039146.533125 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176754.55 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 17675.45 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056821.987865 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159079.09 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 15907.91 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072729.897132 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143171.18 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 14317.12 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087047.015471 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128854.07 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 12885.41 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099932.421977 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115968.66 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 11596.87 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111529.287832 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104371.79 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 10437.18 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121966.467102 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93934.61 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 9393.46 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131359.928445 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84541.15 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 8454.12 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139814.043653 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76087.04 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 7608.70 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147422.747341 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68478.33 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 6847.83 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154270.580660 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61630.50 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 6163.05 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160433.630647 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55467.45 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 5546.74 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165980.375635 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49920.70 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 4992.07 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170972.446124 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44928.63 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 4492.86 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175465.309565 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40435.77 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 4043.58 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179508.886661 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36392.19 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3639.22 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183148.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32752.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186748.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29152.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190348.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25552.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193948.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21952.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197548.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18352.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201148.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14752.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204748.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11152.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208348.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7552.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211948.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3952.97 sec) 17:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215548.106048 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 352.97 sec) 17:50:00 [GPUGRID] [rr_sim] 2215901.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974506510.86G/439.78G) 17:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853141.99 17:50:00 [---] pick_jobs_to_run() start 17:50:00 [---] pick_jobs_to_run() end 17:50:00 [---] [rr_sim] end 17:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:50:00 [---] [cpu_sched_debug] preliminary job list: 17:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:50:00 [---] [cpu_sched_debug] final job list: 17:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:50:00 [---] [cpu_sched_debug] enforce_run_list: end 17:50:00 [---] choose_project(): 64200.000000 17:50:00 [WUProp@Home] doing RPC to get work 17:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371399.09 cpu 2119232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119232.90 17:50:00 [---] Got 0 tasks 17:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:51:00 [---] [rr_sim] doing sim: CPU sched 17:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974478920.50G/439.78G) 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215838.34 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 221583.83 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221583.834365 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994254.51 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 199425.45 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 421009.285293 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794829.06 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 179482.91 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600492.191128 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615346.15 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 161534.62 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762026.806380 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453811.54 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 145381.15 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907407.960107 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308430.38 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 130843.04 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038250.998461 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177587.35 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 117758.73 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1156009.732979 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059828.61 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 105982.86 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261992.594046 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953845.75 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 95384.57 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357377.169006 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858461.17 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 85846.12 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443223.286470 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772615.06 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 77261.51 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520484.792188 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695353.55 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 69535.36 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1590020.147334 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625818.20 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 62581.82 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652601.966965 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563236.38 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 56323.64 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708925.604633 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506912.74 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 50691.27 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759616.878535 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456221.47 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 45622.15 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805239.025046 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410599.32 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 41059.93 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846298.956906 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369539.39 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 36953.94 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883252.895580 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332585.45 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 33258.54 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916511.440387 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299326.90 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 29932.69 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946444.130713 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269394.21 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 26939.42 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973383.552006 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242454.79 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 24245.48 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997629.031170 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218209.31 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 21820.93 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019449.962418 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196388.38 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 19638.84 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039088.800541 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176749.54 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 17674.95 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056763.754851 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159074.59 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 15907.46 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072671.213731 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143167.13 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 14316.71 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086987.926723 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128850.42 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 12885.04 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099872.968415 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115965.38 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 11596.54 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111469.505938 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104368.84 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 10436.88 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121906.389709 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93931.95 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 9393.20 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131299.585103 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84538.76 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 8453.88 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139753.460957 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76084.88 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 7608.49 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147361.949226 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68476.39 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 6847.64 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154209.588668 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61628.75 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 6162.88 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160372.464166 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55465.88 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 5546.59 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165919.052114 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49919.29 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 4991.93 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170910.981267 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44927.36 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 4492.74 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175403.717505 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40434.63 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 4043.46 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179447.180120 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36391.16 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3639.12 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183086.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32752.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186686.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29152.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190286.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25552.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193886.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21952.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197486.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18352.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201086.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14752.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204686.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11152.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208286.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7552.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211886.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3952.05 sec) 17:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215486.296472 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 352.05 sec) 17:51:00 [GPUGRID] [rr_sim] 2215838.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974478920.50G/439.78G) 17:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853139.26 17:51:00 [---] pick_jobs_to_run() start 17:51:00 [---] pick_jobs_to_run() end 17:51:00 [---] [rr_sim] end 17:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:51:00 [---] [cpu_sched_debug] preliminary job list: 17:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:51:00 [---] [cpu_sched_debug] final job list: 17:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:51:00 [---] [cpu_sched_debug] enforce_run_list: end 17:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:52:00 [---] [rr_sim] doing sim: CPU sched 17:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974451330.15G/439.78G) 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215775.61 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 221577.56 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221577.560676 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994198.05 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 199419.80 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420997.365285 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794778.24 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 179477.82 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600475.189433 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615300.42 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 161530.04 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 762005.231166 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453770.38 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 145377.04 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907382.268726 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308393.34 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 130839.33 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038221.602530 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177554.00 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 117755.40 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155977.002954 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059798.60 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 105979.86 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261956.863335 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953818.74 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 95381.87 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357338.737678 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858436.87 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 85843.69 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443182.424586 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772593.18 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 77259.32 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520441.742804 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695333.86 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 69533.39 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589975.129200 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625800.48 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 62580.05 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652555.176957 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563220.43 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 56322.04 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708877.219937 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506898.39 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 50689.84 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759567.058620 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456208.55 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 45620.85 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805187.913435 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410587.69 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 41058.77 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846246.682768 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369528.92 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 36952.89 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883199.575167 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332576.03 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 33257.60 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916457.178327 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299318.43 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 29931.84 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946389.021171 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269386.59 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 26938.66 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973327.679730 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242447.93 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 24244.79 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997572.472434 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218203.13 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 21820.31 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019392.785867 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196382.82 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 19638.28 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039031.067957 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176744.54 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 17674.45 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056705.521837 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159070.08 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 15907.01 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072612.530330 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143163.08 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 14316.31 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086928.837974 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128846.77 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 12884.68 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099813.514853 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115962.09 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 11596.21 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111409.724044 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104365.88 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 10436.59 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121846.312316 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93929.29 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 9392.93 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131239.241761 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84536.37 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 8453.64 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139692.878261 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76082.73 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 7608.27 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147301.151112 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68474.46 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 6847.45 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154148.596677 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61627.01 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 6162.70 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160311.297686 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55464.31 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 5546.43 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165857.728594 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49917.88 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 4991.79 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170849.516411 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44926.09 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 4492.61 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175342.125446 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40433.48 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 4043.35 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179385.473578 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36390.13 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3639.01 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183024.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32751.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186624.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29151.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190224.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25551.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193824.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21951.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197424.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18351.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201024.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14751.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204624.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11151.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208224.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7551.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211824.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3951.12 sec) 17:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215424.486897 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 351.12 sec) 17:52:00 [GPUGRID] [rr_sim] 2215775.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974451330.15G/439.78G) 17:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853136.52 17:52:00 [---] pick_jobs_to_run() start 17:52:00 [---] pick_jobs_to_run() end 17:52:00 [---] [rr_sim] end 17:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:52:00 [---] [cpu_sched_debug] preliminary job list: 17:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:52:00 [---] [cpu_sched_debug] final job list: 17:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:52:00 [---] [cpu_sched_debug] enforce_run_list: end 17:52:00 [---] choose_project(): 64320.000000 17:52:00 [WUProp@Home] doing RPC to get work 17:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371279.09 cpu 2119112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119112.90 17:52:00 [---] Got 0 tasks 17:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:53:00 [---] [rr_sim] doing sim: CPU sched 17:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974423739.79G/439.78G) 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215712.87 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 221571.29 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221571.286988 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994141.58 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 199414.16 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420985.445278 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794727.42 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 179472.74 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600458.187738 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615254.68 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 161525.47 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761983.655953 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453729.21 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 145372.92 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907356.577346 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308356.29 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 130835.63 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038192.206599 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177520.66 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 117752.07 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155944.272928 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059768.60 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 105976.86 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261921.132623 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953791.74 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 95379.17 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357300.306349 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858412.56 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 85841.26 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443141.562702 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772571.31 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 77257.13 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520398.693420 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695314.18 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 69531.42 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589930.111067 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625782.76 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 62578.28 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652508.386948 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563204.48 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 56320.45 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708828.835242 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506884.03 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 50688.40 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759517.238706 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456195.63 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 45619.56 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805136.801823 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410576.07 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 41057.61 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846194.408629 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369518.46 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 36951.85 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883146.254755 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332566.62 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 33256.66 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916402.916267 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299309.95 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 29931.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946333.911629 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269378.96 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 26937.90 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973271.807454 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242441.06 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 24244.11 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997515.913697 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218196.96 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 21819.70 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019335.609316 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196377.26 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 19637.73 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038973.335372 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176739.53 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 17673.95 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056647.288823 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159065.58 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 15906.56 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072553.846929 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143159.02 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 14315.90 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086869.749225 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128843.12 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 12884.31 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099754.061290 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115958.81 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 11595.88 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111349.942150 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104362.93 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 10436.29 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121786.234923 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93926.63 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 9392.66 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131178.898419 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84533.97 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 8453.40 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139632.295565 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76080.57 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 7608.06 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147240.352997 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68472.52 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 6847.25 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154087.604685 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61625.27 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 6162.53 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160250.131205 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55462.74 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 5546.27 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165796.405073 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49916.46 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 4991.65 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170788.051554 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44924.82 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 4492.48 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175280.533387 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40432.34 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 4043.23 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179323.767036 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36389.10 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3638.91 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182962.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32750.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186562.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29150.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190162.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25550.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193762.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21950.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197362.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18350.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200962.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14750.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204562.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11150.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208162.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7550.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211762.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3950.19 sec) 17:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215362.677321 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 350.19 sec) 17:53:00 [GPUGRID] [rr_sim] 2215712.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974423739.79G/439.78G) 17:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853133.78 17:53:00 [---] pick_jobs_to_run() start 17:53:00 [---] pick_jobs_to_run() end 17:53:00 [---] [rr_sim] end 17:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:53:00 [---] [cpu_sched_debug] preliminary job list: 17:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:53:00 [---] [cpu_sched_debug] final job list: 17:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:53:00 [---] [cpu_sched_debug] enforce_run_list: end 17:53:00 [---] choose_project(): 64380.000000 17:53:00 [WUProp@Home] doing RPC to get work 17:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371219.09 cpu 2119052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2119052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2119052.90 17:53:00 [---] Got 0 tasks 17:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:54:00 [---] [rr_sim] doing sim: CPU sched 17:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974396149.44G/439.78G) 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215650.13 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 221565.01 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221565.013300 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994085.12 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 199408.51 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420973.525270 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794676.61 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 179467.66 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600441.186043 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615208.95 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 161520.89 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761962.080739 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453688.05 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 145368.81 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907330.885965 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308319.25 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 130831.92 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038162.810669 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177487.32 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 117748.73 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155911.542902 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059738.59 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 105973.86 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261885.401912 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953764.73 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 95376.47 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357261.875020 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858388.26 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 85838.83 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443100.700818 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772549.43 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 77254.94 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520355.644037 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695294.49 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 69529.45 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589885.092933 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625765.04 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 62576.50 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652461.596940 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563188.54 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 56318.85 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708780.450546 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506869.68 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 50686.97 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759467.418791 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456182.71 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 45618.27 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805085.690212 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410564.44 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 41056.44 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846142.134491 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369508.00 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 36950.80 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883092.934342 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332557.20 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 33255.72 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916348.654208 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299301.48 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 29930.15 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946278.802087 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269371.33 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 26937.13 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973215.935178 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242434.20 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 24243.42 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997459.354961 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218190.78 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 21819.08 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019278.432765 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196371.70 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 19637.17 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038915.602788 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176734.53 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 17673.45 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056589.055809 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159061.08 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 15906.11 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072495.163528 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143154.97 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 14315.50 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086810.660476 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128839.47 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 12883.95 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099694.607728 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115955.53 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 11595.55 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111290.160255 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104359.97 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 10436.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121726.157530 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93923.98 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 9392.40 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131118.555077 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84531.58 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 8453.16 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139571.712869 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76078.42 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 7607.84 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147179.554882 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68470.58 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 6847.06 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154026.612694 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61623.52 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 6162.35 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160188.964725 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55461.17 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 5546.12 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165735.081552 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49915.05 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 4991.51 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170726.586697 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44923.55 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 4492.35 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175218.941327 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40431.19 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 4043.12 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179262.060495 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36388.07 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3638.81 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182900.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32749.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186500.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29149.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190100.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25549.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193700.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21949.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197300.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18349.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200900.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14749.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204500.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11149.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208100.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7549.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211700.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3949.27 sec) 17:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215300.867745 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 349.27 sec) 17:54:00 [GPUGRID] [rr_sim] 2215650.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974396149.44G/439.78G) 17:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853131.05 17:54:00 [---] pick_jobs_to_run() start 17:54:00 [---] pick_jobs_to_run() end 17:54:00 [---] [rr_sim] end 17:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:54:00 [---] [cpu_sched_debug] preliminary job list: 17:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:54:00 [---] [cpu_sched_debug] final job list: 17:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:54:00 [---] [cpu_sched_debug] enforce_run_list: end 17:54:00 [---] choose_project(): 64440.000000 17:54:00 [WUProp@Home] doing RPC to get work 17:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371159.09 cpu 2118992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118992.90 17:54:00 [---] Got 0 tasks 17:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:55:00 [---] [rr_sim] doing sim: CPU sched 17:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974396149.44G/439.78G) 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215650.13 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 221565.01 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221565.013300 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994085.12 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 199408.51 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420973.525270 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794676.61 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 179467.66 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600441.186043 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615208.95 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 161520.89 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761962.080739 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453688.05 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 145368.81 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907330.885965 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308319.25 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 130831.92 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038162.810669 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177487.32 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 117748.73 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155911.542902 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059738.59 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 105973.86 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261885.401912 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953764.73 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 95376.47 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357261.875020 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858388.26 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 85838.83 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443100.700818 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772549.43 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 77254.94 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520355.644037 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695294.49 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 69529.45 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589885.092933 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625765.04 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 62576.50 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652461.596940 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563188.54 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 56318.85 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708780.450546 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506869.68 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 50686.97 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759467.418791 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456182.71 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 45618.27 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805085.690212 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410564.44 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 41056.44 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846142.134491 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369508.00 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 36950.80 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883092.934342 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332557.20 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 33255.72 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916348.654208 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299301.48 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 29930.15 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946278.802087 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269371.33 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 26937.13 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973215.935178 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242434.20 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 24243.42 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997459.354961 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218190.78 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 21819.08 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019278.432765 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196371.70 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 19637.17 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038915.602788 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176734.53 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 17673.45 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056589.055809 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159061.08 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 15906.11 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072495.163528 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143154.97 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 14315.50 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086810.660476 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128839.47 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 12883.95 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099694.607728 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115955.53 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 11595.55 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111290.160255 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104359.97 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 10436.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121726.157530 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93923.98 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 9392.40 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131118.555077 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84531.58 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 8453.16 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139571.712869 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76078.42 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 7607.84 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147179.554882 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68470.58 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 6847.06 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154026.612694 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61623.52 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 6162.35 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160188.964725 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55461.17 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 5546.12 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165735.081552 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49915.05 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 4991.51 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170726.586697 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44923.55 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 4492.35 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175218.941327 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40431.19 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 4043.12 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179262.060495 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36388.07 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3638.81 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182900.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32749.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186500.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29149.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190100.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25549.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193700.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21949.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197300.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18349.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200900.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14749.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204500.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11149.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208100.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7549.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211700.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3949.27 sec) 17:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215300.867745 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 349.27 sec) 17:55:00 [GPUGRID] [rr_sim] 2215650.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974396149.44G/439.78G) 17:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853191.05 17:55:00 [---] pick_jobs_to_run() start 17:55:00 [---] pick_jobs_to_run() end 17:55:00 [---] [rr_sim] end 17:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:55:00 [---] [cpu_sched_debug] preliminary job list: 17:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:55:00 [---] [cpu_sched_debug] final job list: 17:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:55:00 [---] [cpu_sched_debug] enforce_run_list: end 17:55:00 [---] choose_project(): 64500.000000 17:55:00 [WUProp@Home] doing RPC to get work 17:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371099.09 cpu 2118992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118992.90 17:55:00 [---] Got 0 tasks 17:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:56:00 [---] [rr_sim] doing sim: CPU sched 17:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974368559.08G/439.78G) 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215587.40 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 221558.74 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221558.739612 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994028.66 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 199402.87 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420961.605262 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794625.79 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 179462.58 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600424.184348 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615163.21 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 161516.32 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761940.505525 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453646.89 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 145364.69 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907305.194584 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308282.20 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 130828.22 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038133.414738 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177453.98 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 117745.40 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155878.812876 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059708.58 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 105970.86 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261849.671200 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953737.72 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 95373.77 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357223.443692 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858363.95 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 85836.40 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443059.838934 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772527.56 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 77252.76 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520312.594653 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695274.80 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 69527.48 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589840.074799 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625747.32 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 62574.73 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652414.806931 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563172.59 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 56317.26 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708732.065850 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506855.33 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 50685.53 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759417.598877 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456169.80 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 45616.98 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805034.578601 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410552.82 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 41055.28 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846089.860353 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369497.54 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 36949.75 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883039.613929 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332547.78 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 33254.78 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916294.392148 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299293.00 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 29929.30 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946223.692545 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269363.70 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 26936.37 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973160.062902 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242427.33 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 24242.73 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997402.796224 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218184.60 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 21818.46 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019221.256213 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196366.14 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 19636.61 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038857.870204 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176729.53 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 17672.95 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056530.822795 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159056.57 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 15905.66 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072436.480128 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143150.92 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 14315.09 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086751.571727 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128835.82 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 12883.58 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099635.154166 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115952.24 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 11595.22 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111230.378361 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104357.02 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 10435.70 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121666.080137 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93921.32 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 9392.13 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131058.211735 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84529.18 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 8452.92 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139511.130173 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76076.27 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 7607.63 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147118.756768 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68468.64 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 6846.86 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153965.620703 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61621.78 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 6162.18 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160127.798244 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55459.60 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 5545.96 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165673.758032 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49913.64 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 4991.36 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170665.121840 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44922.27 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 4492.23 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175157.349268 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40430.05 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 4043.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179200.353953 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36387.04 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3638.70 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182839.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32748.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186439.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29148.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190039.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25548.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193639.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21948.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197239.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18348.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200839.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14748.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204439.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11148.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208039.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7548.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211639.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3948.34 sec) 17:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215239.058170 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 348.34 sec) 17:56:00 [GPUGRID] [rr_sim] 2215587.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974368559.08G/439.78G) 17:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853188.31 17:56:00 [---] pick_jobs_to_run() start 17:56:00 [---] pick_jobs_to_run() end 17:56:00 [---] [rr_sim] end 17:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:56:00 [---] [cpu_sched_debug] preliminary job list: 17:56:00 [---] [cpu_sched_debug] final job list: 17:56:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 17:56:00 [---] [work_fetch] Request work fetch: CPUs idle 17:56:00 [---] [cpu_sched_debug] enforce_run_list: end 17:56:00 [---] choose_project(): 64560.000000 17:56:00 [WUProp@Home] doing RPC to get work 17:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 371039.09 cpu 2118932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118932.90 17:56:00 [---] Got 0 tasks 17:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:57:00 [---] [rr_sim] doing sim: CPU sched 17:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974368559.08G/439.78G) 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215587.40 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 221558.74 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221558.739612 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1994028.66 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 199402.87 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420961.605262 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794625.79 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 179462.58 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600424.184348 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615163.21 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 161516.32 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761940.505525 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453646.89 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 145364.69 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907305.194584 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308282.20 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 130828.22 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038133.414738 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177453.98 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 117745.40 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155878.812876 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059708.58 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 105970.86 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261849.671200 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953737.72 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 95373.77 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357223.443692 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858363.95 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 85836.40 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443059.838934 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772527.56 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 77252.76 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520312.594653 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695274.80 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 69527.48 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589840.074799 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625747.32 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 62574.73 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652414.806931 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563172.59 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 56317.26 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708732.065850 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506855.33 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 50685.53 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759417.598877 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456169.80 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 45616.98 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1805034.578601 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410552.82 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 41055.28 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846089.860353 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369497.54 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 36949.75 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1883039.613929 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332547.78 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 33254.78 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916294.392148 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299293.00 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 29929.30 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946223.692545 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269363.70 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 26936.37 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973160.062902 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242427.33 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 24242.73 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997402.796224 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218184.60 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 21818.46 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019221.256213 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196366.14 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 19636.61 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038857.870204 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176729.53 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 17672.95 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056530.822795 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159056.57 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 15905.66 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072436.480128 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143150.92 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 14315.09 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086751.571727 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128835.82 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 12883.58 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099635.154166 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115952.24 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 11595.22 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111230.378361 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104357.02 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 10435.70 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121666.080137 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93921.32 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 9392.13 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131058.211735 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84529.18 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 8452.92 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139511.130173 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76076.27 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 7607.63 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147118.756768 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68468.64 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 6846.86 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153965.620703 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61621.78 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 6162.18 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160127.798244 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55459.60 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 5545.96 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165673.758032 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49913.64 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 4991.36 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170665.121840 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44922.27 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 4492.23 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175157.349268 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40430.05 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 4043.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179200.353953 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36387.04 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3638.70 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182839.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32748.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186439.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29148.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190039.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25548.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193639.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21948.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197239.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18348.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200839.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14748.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204439.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11148.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208039.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7548.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211639.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3948.34 sec) 17:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215239.058170 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 348.34 sec) 17:57:00 [GPUGRID] [rr_sim] 2215587.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974368559.08G/439.78G) 17:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853248.31 17:57:00 [---] pick_jobs_to_run() start 17:57:00 [---] pick_jobs_to_run() end 17:57:00 [---] [rr_sim] end 17:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:57:00 [---] [cpu_sched_debug] preliminary job list: 17:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:57:00 [---] [cpu_sched_debug] final job list: 17:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 17:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:57:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 974368559.08G 17:57:00 [---] [cpu_sched_debug] enforce_run_list: end 17:57:00 [---] choose_project(): 64620.000000 17:57:00 [WUProp@Home] doing RPC to get work 17:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370979.09 cpu 2118932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118932.90 17:57:00 [---] Got 0 tasks 17:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:58:00 [---] [rr_sim] doing sim: CPU sched 17:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974340968.72G/439.78G) 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215524.66 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 221552.47 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221552.465924 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993972.19 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 199397.22 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420949.685255 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794574.97 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 179457.50 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600407.182653 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615117.48 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 161511.75 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761918.930311 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453605.73 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 145360.57 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907279.503204 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308245.16 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 130824.52 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038104.018807 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177420.64 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 117742.06 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155846.082850 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059678.58 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 105967.86 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261813.940488 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953710.72 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 95371.07 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357185.012363 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858339.65 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 85833.96 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1443018.977051 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772505.68 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 77250.57 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520269.545269 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695255.11 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 69525.51 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589795.056666 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625729.60 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 62572.96 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652368.016923 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563156.64 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 56315.66 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708683.681154 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506840.98 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 50684.10 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759367.778962 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456156.88 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 45615.69 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804983.466990 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410541.19 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 41054.12 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1846037.586214 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369487.07 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 36948.71 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882986.293516 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332538.37 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 33253.84 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916240.130088 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299284.53 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 29928.45 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946168.583003 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269356.08 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 26935.61 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973104.190626 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242420.47 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 24242.05 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997346.237487 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218178.42 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 21817.84 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019164.079662 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196360.58 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 19636.06 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038800.137620 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176724.52 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 17672.45 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056472.589781 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159052.07 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 15905.21 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072377.796727 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143146.86 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 14314.69 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086692.482978 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128832.18 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 12883.22 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099575.700604 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115948.96 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 11594.90 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111170.596467 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104354.06 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 10435.41 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121606.002744 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93918.66 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 9391.87 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130997.868393 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84526.79 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 8452.68 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139450.547477 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76074.11 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 7607.41 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147057.958653 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68466.70 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 6846.67 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153904.628711 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61620.03 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 6162.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160066.631764 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55458.03 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 5545.80 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165612.434511 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49912.22 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 4991.22 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170603.656984 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44921.00 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 4492.10 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175095.757209 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40428.90 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 4042.89 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179138.647412 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36386.01 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3638.60 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182777.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32747.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186377.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29147.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189977.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25547.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193577.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21947.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197177.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18347.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200777.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14747.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204377.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11147.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207977.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7547.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211577.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3947.41 sec) 17:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215177.248594 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 347.41 sec) 17:58:00 [GPUGRID] [rr_sim] 2215524.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974340968.72G/439.78G) 17:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853245.57 17:58:00 [---] pick_jobs_to_run() start 17:58:00 [---] pick_jobs_to_run() end 17:58:00 [---] [rr_sim] end 17:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:58:00 [---] [cpu_sched_debug] preliminary job list: 17:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 17:58:00 [---] [cpu_sched_debug] final job list: 17:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:58:00 [---] [cpu_sched_debug] enforce_run_list: end 17:58:00 [---] choose_project(): 64680.000000 17:58:00 [WUProp@Home] doing RPC to get work 17:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370919.09 cpu 2118872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118872.90 17:58:00 [---] Got 0 tasks 17:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 17:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 17:59:00 [---] [rr_sim] doing sim: CPU sched 17:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974313378.37G/439.78G) 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215461.92 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 221546.19 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221546.192235 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993915.73 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 199391.57 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420937.765247 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794524.16 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 179452.42 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600390.180958 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615071.74 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 161507.17 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761897.355097 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453564.57 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 145356.46 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907253.811823 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308208.11 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 130820.81 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038074.622876 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177387.30 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 117738.73 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155813.352824 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059648.57 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 105964.86 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261778.209777 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953683.71 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 95368.37 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357146.581035 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858315.34 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 85831.53 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442978.115167 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772483.81 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 77248.38 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520226.495885 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695235.43 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 69523.54 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589750.038532 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625711.88 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 62571.19 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652321.226914 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563140.70 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 56314.07 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708635.296458 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506826.63 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 50682.66 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759317.959048 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456143.96 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 45614.40 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804932.355378 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410529.57 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 41052.96 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845985.312076 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369476.61 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 36947.66 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882932.973104 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332528.95 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 33252.89 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916185.868029 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299276.05 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 29927.61 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946113.473461 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269348.45 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 26934.84 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1973048.318351 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242413.60 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 24241.36 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997289.678751 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218172.24 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 21817.22 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019106.903111 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196355.02 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 19635.50 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038742.405035 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176719.52 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 17671.95 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056414.356767 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159047.57 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 15904.76 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072319.113326 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143142.81 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 14314.28 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086633.394229 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128828.53 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 12882.85 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099516.247041 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115945.68 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 11594.57 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111110.814572 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104351.11 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 10435.11 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121545.925351 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93916.00 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 9391.60 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130937.525051 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84524.40 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 8452.44 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139389.964781 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76071.96 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 7607.20 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146997.160538 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68464.76 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 6846.48 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153843.636720 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61618.29 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 6161.83 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160005.465283 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55456.46 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 5545.65 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165551.110990 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49910.81 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 4991.08 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170542.192127 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44919.73 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 4491.97 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175034.165149 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40427.76 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 4042.78 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179076.940870 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36384.98 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3638.50 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182715.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32746.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186315.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29146.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189915.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25546.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193515.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21946.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197115.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18346.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200715.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14746.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204315.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11146.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207915.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7546.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211515.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3946.48 sec) 17:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215115.439018 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 346.48 sec) 17:59:00 [GPUGRID] [rr_sim] 2215461.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974313378.37G/439.78G) 17:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853242.84 17:59:00 [---] pick_jobs_to_run() start 17:59:00 [---] pick_jobs_to_run() end 17:59:00 [---] [rr_sim] end 17:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 17:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 17:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 17:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 17:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 17:59:00 [---] [cpu_sched_debug] preliminary job list: 17:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:59:00 [---] [cpu_sched_debug] final job list: 17:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 17:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 17:59:00 [---] [cpu_sched_debug] enforce_run_list: end 17:59:00 [---] choose_project(): 64740.000000 17:59:00 [WUProp@Home] doing RPC to get work 17:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370859.09 cpu 2118812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118812.90 17:59:00 [---] Got 0 tasks 18:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:00:00 [---] [rr_sim] doing sim: CPU sched 18:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974285788.01G/439.78G) 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215399.19 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 221539.92 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221539.918547 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993859.27 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 199385.93 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420925.845240 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794473.34 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 179447.33 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600373.179263 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1615026.01 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 161502.60 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761875.779884 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453523.41 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 145352.34 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907228.120443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308171.07 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 130817.11 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038045.226945 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177353.96 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 117735.40 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155780.622798 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059618.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 105961.86 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261742.479065 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953656.71 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 95365.67 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357108.149706 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858291.04 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 85829.10 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442937.253283 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772461.93 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 77246.19 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520183.446502 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695215.74 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 69521.57 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589705.020399 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625694.17 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 62569.42 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652274.436906 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563124.75 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 56312.47 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708586.911762 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506812.27 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 50681.23 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759268.139133 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456131.05 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 45613.10 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804881.243767 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410517.94 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 41051.79 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845933.037938 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369466.15 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 36946.61 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882879.652691 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332519.53 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 33251.95 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916131.605969 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299267.58 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 29926.76 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946058.363919 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269340.82 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 26934.08 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972992.446075 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242406.74 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 24240.67 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997233.120014 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218166.07 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 21816.61 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019049.726560 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196349.46 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 19634.95 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038684.672451 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176714.51 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 17671.45 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056356.123753 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159043.06 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 15904.31 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072260.429925 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143138.76 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 14313.88 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086574.305480 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128824.88 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 12882.49 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099456.793479 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115942.39 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 11594.24 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111051.032678 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104348.15 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 10434.82 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121485.847958 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93913.34 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 9391.33 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130877.181709 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84522.00 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 8452.20 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139329.382085 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76069.80 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 7606.98 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146936.362424 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68462.82 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 6846.28 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153782.644729 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61616.54 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 6161.65 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159944.298803 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55454.89 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 5545.49 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165489.787470 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49909.40 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 4990.94 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170480.727270 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44918.46 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 4491.85 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174972.573090 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40426.61 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 4042.66 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179015.234328 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36383.95 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3638.40 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182653.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32745.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186253.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29145.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189853.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25545.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193453.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21945.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197053.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18345.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200653.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14745.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204253.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11145.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207853.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7545.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211453.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3945.56 sec) 18:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2215053.629443 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 345.56 sec) 18:00:00 [GPUGRID] [rr_sim] 2215399.19: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974285788.01G/439.78G) 18:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853240.10 18:00:00 [---] pick_jobs_to_run() start 18:00:00 [---] pick_jobs_to_run() end 18:00:00 [---] [rr_sim] end 18:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:00:00 [---] [cpu_sched_debug] preliminary job list: 18:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:00:00 [---] [cpu_sched_debug] final job list: 18:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:00:00 [---] [cpu_sched_debug] enforce_run_list: end 18:00:00 [---] choose_project(): 64800.000000 18:00:00 [WUProp@Home] doing RPC to get work 18:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370799.09 cpu 2118752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118752.90 18:00:00 [---] Got 0 tasks 18:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:01:00 [---] [rr_sim] doing sim: CPU sched 18:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974258197.65G/439.78G) 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215336.45 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 221533.64 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221533.644859 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993802.80 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 199380.28 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420913.925232 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794422.52 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 179442.25 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600356.177568 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614980.27 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 161498.03 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761854.204670 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453482.24 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 145348.22 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907202.429062 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308134.02 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 130813.40 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1038015.831015 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177320.62 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 117732.06 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155747.892772 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059588.56 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 105958.86 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261706.748354 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953629.70 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 95362.97 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357069.718377 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858266.73 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 85826.67 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442896.391399 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772440.06 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 77244.01 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520140.397118 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695196.05 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 69519.61 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589660.002265 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625676.45 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 62567.64 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652227.646897 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563108.80 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 56310.88 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708538.527067 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506797.92 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 50679.79 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759218.319219 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456118.13 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 45611.81 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804830.132156 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410506.32 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 41050.63 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845880.763799 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369455.68 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 36945.57 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882826.332278 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332510.12 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 33251.01 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916077.343909 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299259.10 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 29925.91 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1946003.254377 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269333.19 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 26933.32 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972936.573799 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242399.87 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 24239.99 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997176.561278 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218159.89 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 21815.99 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018992.550009 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196343.90 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 19634.39 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038626.939867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176709.51 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 17670.95 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056297.890739 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159038.56 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 15903.86 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072201.746524 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143134.70 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 14313.47 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086515.216731 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128821.23 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 12882.12 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099397.339917 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115939.11 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 11593.91 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110991.250784 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104345.20 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 10434.52 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121425.770564 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93910.68 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 9391.07 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130816.838367 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84519.61 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 8451.96 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139268.799389 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76067.65 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 7606.76 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146875.564309 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68460.88 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 6846.09 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153721.652737 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61614.80 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 6161.48 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159883.132323 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55453.32 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 5545.33 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165428.463949 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49907.98 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 4990.80 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170419.262413 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44917.19 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 4491.72 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174910.981031 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40425.47 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 4042.55 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178953.527787 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36382.92 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3638.29 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182591.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32744.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186191.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29144.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189791.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25544.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193391.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21944.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196991.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18344.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200591.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14744.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204191.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11144.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207791.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7544.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211391.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3944.63 sec) 18:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214991.819867 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 344.63 sec) 18:01:00 [GPUGRID] [rr_sim] 2215336.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974258197.65G/439.78G) 18:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853237.36 18:01:00 [---] pick_jobs_to_run() start 18:01:00 [---] pick_jobs_to_run() end 18:01:00 [---] [rr_sim] end 18:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:01:00 [---] [cpu_sched_debug] preliminary job list: 18:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:01:00 [---] [cpu_sched_debug] final job list: 18:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:01:00 [---] [cpu_sched_debug] enforce_run_list: end 18:01:00 [---] choose_project(): 64860.000000 18:01:00 [WUProp@Home] doing RPC to get work 18:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370739.09 cpu 2118692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118692.90 18:01:00 [---] Got 0 tasks 18:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:02:00 [---] [rr_sim] doing sim: CPU sched 18:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974230607.30G/439.78G) 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215273.71 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 221527.37 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221527.371171 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993746.34 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 199374.63 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420902.005224 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794371.71 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 179437.17 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600339.175873 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614934.54 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 161493.45 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761832.629456 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453441.08 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 145344.11 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907176.737681 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308096.97 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 130809.70 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037986.435084 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177287.28 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 117728.73 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155715.162746 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059558.55 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 105955.85 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261671.017642 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953602.69 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 95360.27 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1357031.287049 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858242.42 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 85824.24 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442855.529515 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772418.18 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 77241.82 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520097.347734 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695176.36 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 69517.64 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589614.984131 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625658.73 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 62565.87 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652180.856889 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563092.85 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 56309.29 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708490.142371 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506783.57 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 50678.36 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759168.499304 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456105.21 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 45610.52 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804779.020545 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410494.69 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 41049.47 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845828.489661 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369445.22 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 36944.52 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882773.011866 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332500.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 33250.07 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1916023.081850 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299250.63 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 29925.06 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945948.144836 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269325.57 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 26932.56 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972880.701523 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242393.01 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 24239.30 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997120.002541 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218153.71 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 21815.37 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018935.373458 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196338.34 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 19633.83 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038569.207283 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176704.50 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 17670.45 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056239.657725 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159034.05 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 15903.41 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072143.063123 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143130.65 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 14313.06 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086456.127982 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128817.58 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 12881.76 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099337.886354 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115935.83 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 11593.58 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110931.468890 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104342.24 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 10434.22 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121365.693171 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93908.02 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 9390.80 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130756.495025 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84517.22 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 8451.72 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139208.216693 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76065.50 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 7606.55 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146814.766195 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68458.95 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 6845.89 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153660.660746 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61613.05 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 6161.31 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159821.965842 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55451.75 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 5545.17 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165367.140429 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49906.57 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 4990.66 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170357.797556 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44915.91 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 4491.59 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174849.388972 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40424.32 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 4042.43 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178891.821245 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36381.89 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3638.19 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182530.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32743.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186130.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29143.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189730.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25543.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193330.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21943.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196930.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18343.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200530.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14743.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204130.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11143.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207730.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7543.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211330.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3943.70 sec) 18:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214930.010291 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 343.70 sec) 18:02:00 [GPUGRID] [rr_sim] 2215273.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974230607.30G/439.78G) 18:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853234.62 18:02:00 [---] pick_jobs_to_run() start 18:02:00 [---] pick_jobs_to_run() end 18:02:00 [---] [rr_sim] end 18:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:02:00 [---] [cpu_sched_debug] preliminary job list: 18:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:02:00 [---] [cpu_sched_debug] final job list: 18:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:02:00 [---] [cpu_sched_debug] enforce_run_list: end 18:02:00 [---] choose_project(): 64920.000000 18:02:00 [WUProp@Home] doing RPC to get work 18:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370679.09 cpu 2118632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118632.90 18:02:00 [---] Got 0 tasks 18:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:03:00 [---] [rr_sim] doing sim: CPU sched 18:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974203016.94G/439.78G) 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215210.97 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 221521.10 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221521.097483 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993689.88 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 199368.99 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420890.085217 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794320.89 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 179432.09 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600322.174178 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614888.80 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 161488.88 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761811.054242 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453399.92 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 145339.99 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907151.046301 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308059.93 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 130805.99 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037957.039153 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177253.94 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 117725.39 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155682.432720 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059528.54 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 105952.85 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261635.286931 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953575.69 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 95357.57 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356992.855720 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858218.12 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 85821.81 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442814.667631 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772396.31 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 77239.63 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520054.298350 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695156.68 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 69515.67 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589569.965998 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625641.01 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 62564.10 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652134.066880 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563076.91 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 56307.69 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708441.757675 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506769.22 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 50676.92 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759118.679390 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456092.30 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 45609.23 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804727.908933 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410483.07 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 41048.31 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845776.215523 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369434.76 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 36943.48 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882719.691453 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332491.28 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 33249.13 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915968.819790 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299242.16 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 29924.22 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945893.035294 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269317.94 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 26931.79 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972824.829247 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242386.15 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 24238.61 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997063.443805 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218147.53 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 21814.75 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018878.196907 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196332.78 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 19633.28 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038511.474699 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176699.50 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 17669.95 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056181.424711 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159029.55 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 15902.96 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072084.379723 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143126.60 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 14312.66 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086397.039233 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128813.94 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 12881.39 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099278.432792 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115932.54 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 11593.25 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110871.686995 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104339.29 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 10433.93 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121305.615778 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93905.36 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 9390.54 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130696.151683 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84514.82 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 8451.48 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139147.633997 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76063.34 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 7606.33 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146753.968080 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68457.01 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 6845.70 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153599.668755 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61611.31 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 6161.13 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159760.799362 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55450.18 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 5545.02 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165305.816908 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49905.16 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 4990.52 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170296.332700 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44914.64 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 4491.46 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174787.796912 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40423.18 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 4042.32 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178830.114704 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36380.86 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3638.09 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182468.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32742.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186068.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29142.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189668.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25542.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193268.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21942.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196868.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18342.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200468.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14742.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204068.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11142.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207668.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7542.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211268.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3942.77 sec) 18:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214868.200716 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 342.77 sec) 18:03:00 [GPUGRID] [rr_sim] 2215210.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974203016.94G/439.78G) 18:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853231.89 18:03:00 [---] pick_jobs_to_run() start 18:03:00 [---] pick_jobs_to_run() end 18:03:00 [---] [rr_sim] end 18:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:03:00 [---] [cpu_sched_debug] preliminary job list: 18:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:03:00 [---] [cpu_sched_debug] final job list: 18:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:03:00 [---] [cpu_sched_debug] enforce_run_list: end 18:03:00 [---] choose_project(): 64980.000000 18:03:00 [WUProp@Home] doing RPC to get work 18:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370619.09 cpu 2118572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118572.90 18:03:00 [---] Got 0 tasks 18:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:04:00 [---] [rr_sim] doing sim: CPU sched 18:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974175426.59G/439.78G) 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215148.24 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 221514.82 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221514.823794 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993633.41 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 199363.34 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420878.165209 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794270.07 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 179427.01 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600305.172483 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614843.07 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 161484.31 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761789.479029 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453358.76 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 145335.88 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907125.354920 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1308022.88 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 130802.29 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037927.643222 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177220.59 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 117722.06 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155649.702694 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059498.54 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 105949.85 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261599.556219 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953548.68 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 95354.87 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356954.424392 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858193.81 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 85819.38 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442773.805747 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772374.43 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 77237.44 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1520011.248966 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695136.99 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 69513.70 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589524.947864 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625623.29 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 62562.33 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652087.276872 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563060.96 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 56306.10 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708393.372979 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506754.86 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 50675.49 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759068.859475 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456079.38 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 45607.94 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804676.797322 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410471.44 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 41047.14 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845723.941384 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369424.30 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 36942.43 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882666.371040 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332481.87 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 33248.19 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915914.557730 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299233.68 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 29923.37 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945837.925752 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269310.31 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 26931.03 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972768.956971 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242379.28 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 24237.93 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997006.885068 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218141.35 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 21814.14 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018821.020356 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196327.22 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 19632.72 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038453.742114 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176694.50 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 17669.45 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056123.191697 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159025.05 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 15902.50 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072025.696322 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143122.54 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 14312.25 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086337.950484 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128810.29 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 12881.03 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099218.979230 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115929.26 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 11592.93 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110811.905101 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104336.33 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 10433.63 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121245.538385 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93902.70 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 9390.27 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130635.808341 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84512.43 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 8451.24 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139087.051301 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76061.19 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 7606.12 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146693.169965 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68455.07 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 6845.51 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153538.676763 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61609.56 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 6160.96 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159699.632881 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55448.61 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 5544.86 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165244.493387 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49903.74 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 4990.37 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170234.867843 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44913.37 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 4491.34 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174726.204853 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40422.03 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 4042.20 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178768.408162 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36379.83 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3637.98 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182406.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32741.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186006.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29141.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189606.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25541.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193206.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21941.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196806.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18341.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200406.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14741.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204006.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11141.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207606.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7541.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211206.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3941.85 sec) 18:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214806.391140 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 341.85 sec) 18:04:00 [GPUGRID] [rr_sim] 2215148.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974175426.59G/439.78G) 18:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853229.15 18:04:00 [---] pick_jobs_to_run() start 18:04:00 [---] pick_jobs_to_run() end 18:04:00 [---] [rr_sim] end 18:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:04:00 [---] [cpu_sched_debug] preliminary job list: 18:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:04:00 [---] [cpu_sched_debug] final job list: 18:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:04:00 [---] [cpu_sched_debug] enforce_run_list: end 18:04:00 [---] choose_project(): 65040.000000 18:04:00 [WUProp@Home] doing RPC to get work 18:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370559.09 cpu 2118512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118512.90 18:04:00 [---] Got 0 tasks 18:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:05:00 [---] [rr_sim] doing sim: CPU sched 18:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974147836.23G/439.78G) 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215085.50 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 221508.55 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221508.550106 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993576.95 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 199357.70 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420866.245202 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794219.26 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 179421.93 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600288.170787 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614797.33 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 161479.73 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761767.903815 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453317.60 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 145331.76 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907099.663539 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307985.84 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 130798.58 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037898.247292 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177187.25 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 117718.73 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155616.972668 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059468.53 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 105946.85 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261563.825508 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953521.68 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 95352.17 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356915.993063 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858169.51 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 85816.95 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442732.943863 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772352.56 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 77235.26 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519968.199583 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695117.30 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 69511.73 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589479.929730 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625605.57 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 62560.56 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1652040.486863 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563045.01 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 56304.50 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708344.988283 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506740.51 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 50674.05 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1759019.039561 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456066.46 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 45606.65 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804625.685711 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410459.82 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 41045.98 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845671.667246 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369413.83 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 36941.38 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882613.050627 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332472.45 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 33247.25 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915860.295671 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299225.21 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 29922.52 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945782.816210 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269302.68 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 26930.27 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972713.084695 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242372.42 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 24237.24 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996950.326331 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218135.17 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 21813.52 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018763.843804 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196321.66 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 19632.17 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038396.009530 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176689.49 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 17668.95 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056064.958683 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159020.54 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 15902.05 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071967.012921 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143118.49 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 14311.85 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086278.861735 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128806.64 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 12880.66 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099159.525668 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115925.98 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 11592.60 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110752.123207 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104333.38 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 10433.34 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121185.460992 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93900.04 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 9390.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130575.464999 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84510.04 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 8451.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139026.468605 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76059.03 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 7605.90 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146632.371851 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68453.13 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 6845.31 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153477.684772 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61607.82 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 6160.78 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159638.466401 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55447.03 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 5544.70 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165183.169867 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49902.33 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 4990.23 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170173.402986 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44912.10 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 4491.21 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174664.612794 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40420.89 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 4042.09 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178706.701620 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36378.80 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3637.88 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182344.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32740.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185944.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29140.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189544.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25540.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193144.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21940.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196744.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18340.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200344.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14740.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203944.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11140.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207544.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7540.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211144.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3940.92 sec) 18:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214744.581564 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 340.92 sec) 18:05:00 [GPUGRID] [rr_sim] 2215085.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974147836.23G/439.78G) 18:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853226.41 18:05:00 [---] pick_jobs_to_run() start 18:05:00 [---] pick_jobs_to_run() end 18:05:00 [---] [rr_sim] end 18:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:05:00 [---] [cpu_sched_debug] preliminary job list: 18:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:05:00 [---] [cpu_sched_debug] final job list: 18:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:05:00 [---] [cpu_sched_debug] enforce_run_list: end 18:05:00 [---] choose_project(): 65100.000000 18:05:00 [WUProp@Home] doing RPC to get work 18:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370499.09 cpu 2118452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118452.90 18:05:00 [---] Got 0 tasks 18:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:06:00 [---] [rr_sim] doing sim: CPU sched 18:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974120245.87G/439.78G) 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2215022.76 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 221502.28 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221502.276418 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993520.49 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 199352.05 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420854.325194 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794168.44 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 179416.84 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600271.169092 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614751.60 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 161475.16 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761746.328601 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453276.44 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 145327.64 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907073.972159 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307948.79 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 130794.88 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037868.851361 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177153.91 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 117715.39 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155584.242643 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059438.52 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 105943.85 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261528.094796 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953494.67 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 95349.47 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356877.561734 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858145.20 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 85814.52 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442692.081979 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772330.68 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 77233.07 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519925.150199 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695097.61 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 69509.76 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589434.911597 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625587.85 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 62558.79 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651993.696855 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563029.07 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 56302.91 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708296.603587 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506726.16 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 50672.62 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758969.219647 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456053.54 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 45605.35 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804574.574100 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410448.19 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 41044.82 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845619.393108 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369403.37 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 36940.34 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882559.730215 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332463.03 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 33246.30 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915806.033611 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299216.73 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 29921.67 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945727.706668 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269295.06 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 26929.51 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972657.212419 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242365.55 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 24236.56 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996893.767595 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218129.00 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 21812.90 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018706.667253 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196316.10 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 19631.61 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038338.276946 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176684.49 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 17668.45 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056006.725669 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159016.04 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 15901.60 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071908.329520 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143114.43 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 14311.44 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086219.772986 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128802.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 12880.30 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099100.072105 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115922.69 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 11592.27 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110692.341313 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104330.42 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 10433.04 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121125.383599 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93897.38 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 9389.74 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130515.121657 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84507.64 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 8450.76 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138965.885909 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76056.88 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 7605.69 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146571.573736 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68451.19 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 6845.12 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153416.692780 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61606.07 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 6160.61 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159577.299920 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55445.46 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 5544.55 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165121.846346 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49900.92 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 4990.09 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170111.938129 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44910.83 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 4491.08 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174603.020734 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40419.74 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 4041.97 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178644.995079 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36377.77 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3637.78 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182282.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32739.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185882.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29139.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189482.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25539.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193082.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21939.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196682.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18339.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200282.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14739.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203882.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11139.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207482.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7539.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211082.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3939.99 sec) 18:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214682.771989 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339.99 sec) 18:06:00 [GPUGRID] [rr_sim] 2215022.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974120245.87G/439.78G) 18:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853223.68 18:06:00 [---] pick_jobs_to_run() start 18:06:00 [---] pick_jobs_to_run() end 18:06:00 [---] [rr_sim] end 18:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:06:00 [---] [cpu_sched_debug] preliminary job list: 18:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:06:00 [---] [cpu_sched_debug] final job list: 18:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:06:00 [---] [cpu_sched_debug] enforce_run_list: end 18:06:00 [---] choose_project(): 65160.000000 18:06:00 [WUProp@Home] doing RPC to get work 18:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370439.09 cpu 2118392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118392.90 18:06:00 [---] Got 0 tasks 18:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:07:00 [---] [rr_sim] doing sim: CPU sched 18:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974092655.52G/439.78G) 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214960.03 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 221496.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221496.002730 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993464.02 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 199346.40 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420842.405186 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794117.62 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 179411.76 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600254.167397 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614705.86 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 161470.59 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761724.753387 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453235.27 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 145323.53 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907048.280778 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307911.75 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 130791.17 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037839.455430 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177120.57 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 117712.06 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155551.512617 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059408.51 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 105940.85 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261492.364085 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953467.66 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 95346.77 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356839.130406 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858120.90 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 85812.09 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442651.220095 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772308.81 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 77230.88 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519882.100815 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695077.93 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 69507.79 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589389.893463 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625570.13 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 62557.01 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651946.906847 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 563013.12 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 56301.31 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708248.218892 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506711.81 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 50671.18 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758919.399732 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456040.63 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 45604.06 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804523.462488 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410436.56 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 41043.66 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845567.118969 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369392.91 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 36939.29 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882506.409802 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332453.62 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 33245.36 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915751.771551 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299208.26 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 29920.83 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945672.597126 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269287.43 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 26928.74 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972601.340143 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242358.69 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 24235.87 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996837.208858 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218122.82 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 21812.28 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018649.490702 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196310.54 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 19631.05 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038280.544362 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176679.48 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 17667.95 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055948.492655 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159011.53 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 15901.15 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071849.646119 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143110.38 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 14311.04 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086160.684237 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128799.34 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 12879.93 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099040.618543 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115919.41 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 11591.94 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110632.559418 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104327.47 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 10432.75 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121065.306206 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93894.72 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 9389.47 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130454.778315 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84505.25 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 8450.52 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138905.303213 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76054.72 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 7605.47 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146510.775622 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68449.25 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 6844.93 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153355.700789 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61604.33 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 6160.43 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159516.133440 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55443.89 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 5544.39 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165060.522826 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49899.50 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 4989.95 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170050.473273 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44909.55 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 4490.96 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174541.428675 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40418.60 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 4041.86 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178583.288537 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36376.74 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3637.67 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182220.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32739.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185820.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29139.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189420.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25539.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193020.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21939.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196620.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18339.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200220.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14739.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203820.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11139.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207420.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7539.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211020.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3939.06 sec) 18:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214620.962413 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 339.06 sec) 18:07:00 [GPUGRID] [rr_sim] 2214960.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974092655.52G/439.78G) 18:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853220.94 18:07:00 [---] pick_jobs_to_run() start 18:07:00 [---] pick_jobs_to_run() end 18:07:00 [---] [rr_sim] end 18:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:07:00 [---] [cpu_sched_debug] preliminary job list: 18:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:07:00 [---] [cpu_sched_debug] final job list: 18:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:07:00 [---] [cpu_sched_debug] enforce_run_list: end 18:07:00 [---] choose_project(): 65220.000000 18:07:00 [WUProp@Home] doing RPC to get work 18:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370379.09 cpu 2118332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118332.90 18:07:00 [---] Got 0 tasks 18:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:08:00 [---] [rr_sim] doing sim: CPU sched 18:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974065065.16G/439.78G) 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214897.29 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 221489.73 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221489.729041 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993407.56 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 199340.76 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420830.485179 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794066.81 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 179406.68 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600237.165702 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614660.12 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 161466.01 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761703.178173 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453194.11 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 145319.41 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 907022.589398 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307874.70 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 130787.47 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037810.059499 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177087.23 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 117708.72 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155518.782591 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059378.51 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 105937.85 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261456.633373 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953440.66 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 95344.07 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356800.699077 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858096.59 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 85809.66 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442610.358211 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772286.93 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 77228.69 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519839.051431 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695058.24 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 69505.82 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589344.875330 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625552.42 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 62555.24 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651900.116838 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562997.17 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 56299.72 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708199.834196 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506697.46 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 50669.75 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758869.579818 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456027.71 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 45602.77 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804472.350877 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410424.94 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 41042.49 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845514.844831 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369382.45 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 36938.24 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882453.089389 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332444.20 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 33244.42 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915697.509492 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299199.78 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 29919.98 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945617.487584 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269279.80 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 26927.98 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972545.467867 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242351.82 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 24235.18 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996780.650122 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218116.64 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 21811.66 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018592.314151 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196304.98 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 19630.50 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038222.811777 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176674.48 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 17667.45 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055890.259641 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159007.03 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 15900.70 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071790.962718 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143106.33 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 14310.63 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086101.595488 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128795.69 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 12879.57 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098981.164981 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115916.13 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 11591.61 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110572.777524 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104324.51 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 10432.45 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121005.228813 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93892.06 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 9389.21 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130394.434973 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84502.86 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 8450.29 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138844.720517 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76052.57 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 7605.26 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146449.977507 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68447.31 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 6844.73 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153294.708798 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61602.58 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 6160.26 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159454.966959 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55442.32 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 5544.23 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164999.199305 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49898.09 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 4989.81 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169989.008416 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44908.28 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 4490.83 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174479.836616 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40417.45 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 4041.75 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178521.581996 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36375.71 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3637.57 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182159.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32738.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185759.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29138.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189359.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25538.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192959.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21938.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196559.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18338.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200159.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14738.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203759.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11138.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207359.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7538.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210959.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3938.14 sec) 18:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214559.152837 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 338.14 sec) 18:08:00 [GPUGRID] [rr_sim] 2214897.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974065065.16G/439.78G) 18:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853218.20 18:08:00 [---] pick_jobs_to_run() start 18:08:00 [---] pick_jobs_to_run() end 18:08:00 [---] [rr_sim] end 18:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:08:00 [---] [cpu_sched_debug] preliminary job list: 18:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:08:00 [---] [cpu_sched_debug] final job list: 18:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:08:00 [---] [cpu_sched_debug] enforce_run_list: end 18:08:00 [---] choose_project(): 65280.000000 18:08:00 [WUProp@Home] doing RPC to get work 18:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370319.09 cpu 2118272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118272.90 18:08:00 [---] Got 0 tasks 18:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:09:00 [---] [rr_sim] doing sim: CPU sched 18:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974037474.81G/439.78G) 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214834.55 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 221483.46 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221483.455353 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993351.10 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 199335.11 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420818.565171 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1794015.99 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 179401.60 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600220.164007 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614614.39 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 161461.44 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761681.602960 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453152.95 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 145315.30 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906996.898017 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307837.66 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 130783.77 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037780.663568 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177053.89 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 117705.39 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155486.052565 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059348.50 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 105934.85 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261420.902662 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953413.65 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 95341.37 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356762.267749 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858072.29 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 85807.23 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442569.496327 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772265.06 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 77226.51 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519796.002048 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695038.55 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 69503.86 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589299.857196 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625534.70 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 62553.47 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651853.326830 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562981.23 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 56298.12 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708151.449500 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506683.10 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 50668.31 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758819.759903 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456014.79 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 45601.48 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804421.239266 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410413.31 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 41041.33 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845462.570693 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369371.98 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 36937.20 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882399.768977 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332434.78 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 33243.48 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915643.247432 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299191.31 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 29919.13 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945562.378042 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269272.18 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 26927.22 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972489.595591 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242344.96 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 24234.50 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996724.091385 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218110.46 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 21811.05 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018535.137600 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196299.42 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 19629.94 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038165.079193 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176669.47 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 17666.95 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055832.026627 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159002.53 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 15900.25 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071732.279318 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143102.27 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 14310.23 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086042.506739 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128792.05 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 12879.20 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098921.711418 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115912.84 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 11591.28 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110512.995630 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104321.56 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 10432.16 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120945.151420 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93889.40 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 9388.94 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130334.091631 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84500.46 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 8450.05 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138784.137821 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76050.42 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 7605.04 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146389.179392 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68445.37 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 6844.54 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153233.716806 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61600.84 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 6160.08 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159393.800479 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55440.75 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 5544.08 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164937.875784 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49896.68 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 4989.67 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169927.543559 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44907.01 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 4490.70 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174418.244556 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40416.31 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 4041.63 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178459.875454 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36374.68 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3637.47 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182097.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32737.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185697.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29137.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189297.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25537.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192897.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21937.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196497.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18337.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200097.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14737.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203697.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11137.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207297.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7537.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210897.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3937.21 sec) 18:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214497.343262 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 337.21 sec) 18:09:00 [GPUGRID] [rr_sim] 2214834.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974037474.81G/439.78G) 18:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853215.47 18:09:00 [---] pick_jobs_to_run() start 18:09:00 [---] pick_jobs_to_run() end 18:09:00 [---] [rr_sim] end 18:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:09:00 [---] [cpu_sched_debug] preliminary job list: 18:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:09:00 [---] [cpu_sched_debug] final job list: 18:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:09:00 [---] [cpu_sched_debug] enforce_run_list: end 18:09:00 [---] choose_project(): 65340.000000 18:09:00 [WUProp@Home] doing RPC to get work 18:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370259.09 cpu 2118212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118212.90 18:09:00 [---] Got 0 tasks 18:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:10:00 [---] [rr_sim] doing sim: CPU sched 18:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (974009884.45G/439.78G) 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214771.82 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 221477.18 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221477.181665 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993294.63 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 199329.46 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420806.645164 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793965.17 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 179396.52 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600203.162312 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614568.65 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 161456.87 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761660.027746 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453111.79 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 145311.18 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906971.206636 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307800.61 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 130780.06 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037751.267638 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1177020.55 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 117702.05 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155453.322539 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059318.49 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 105931.85 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261385.171950 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953386.64 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 95338.66 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356723.836420 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858047.98 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 85804.80 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442528.634443 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772243.18 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 77224.32 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519752.952664 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 695018.86 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 69501.89 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589254.839062 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625516.98 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 62551.70 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651806.536821 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562965.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 56296.53 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708103.064804 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506668.75 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 50666.88 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758769.939989 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 456001.88 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 45600.19 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804370.127655 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410401.69 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 41040.17 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845410.296554 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369361.52 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 36936.15 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882346.448564 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332425.37 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 33242.54 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915588.985372 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299182.83 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 29918.28 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945507.268500 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269264.55 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 26926.45 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972433.723315 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242338.09 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 24233.81 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996667.532649 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218104.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 21810.43 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018477.961049 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196293.86 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 19629.39 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038107.346609 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176664.47 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 17666.45 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055773.793613 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158998.02 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 15899.80 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071673.595917 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143098.22 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 14309.82 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085983.417990 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128788.40 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 12878.84 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098862.257856 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115909.56 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 11590.96 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110453.213735 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104318.60 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 10431.86 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120885.074027 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93886.74 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 9388.67 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130273.748289 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84498.07 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 8449.81 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138723.555125 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76048.26 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 7604.83 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146328.381278 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68443.44 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 6844.34 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153172.724815 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61599.09 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 6159.91 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159332.633999 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55439.18 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 5543.92 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164876.552264 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49895.26 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 4989.53 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169866.078702 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44905.74 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 4490.57 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174356.652497 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40415.16 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 4041.52 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178398.168912 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36373.65 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3637.36 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182035.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32736.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185635.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29136.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189235.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25536.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192835.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21936.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196435.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18336.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200035.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14736.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203635.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11136.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207235.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7536.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210835.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3936.28 sec) 18:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214435.533686 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 336.28 sec) 18:10:00 [GPUGRID] [rr_sim] 2214771.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (974009884.45G/439.78G) 18:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853212.73 18:10:00 [---] pick_jobs_to_run() start 18:10:00 [---] pick_jobs_to_run() end 18:10:00 [---] [rr_sim] end 18:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:10:00 [---] [cpu_sched_debug] preliminary job list: 18:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:10:00 [---] [cpu_sched_debug] final job list: 18:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:10:00 [---] [cpu_sched_debug] enforce_run_list: end 18:10:00 [---] choose_project(): 65400.000000 18:10:00 [WUProp@Home] doing RPC to get work 18:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370199.09 cpu 2118152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118152.90 18:10:00 [---] Got 0 tasks 18:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:11:00 [---] [rr_sim] doing sim: CPU sched 18:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973982294.09G/439.78G) 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214709.08 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 221470.91 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221470.907977 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993238.17 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 199323.82 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420794.725156 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793914.35 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 179391.44 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600186.160617 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614522.92 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 161452.29 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761638.452532 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453070.63 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 145307.06 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906945.515256 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307763.56 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 130776.36 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037721.871707 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176987.21 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 117698.72 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155420.592513 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059288.49 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 105928.85 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261349.441239 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953359.64 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 95335.96 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356685.405091 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 858023.67 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 85802.37 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442487.772559 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772221.31 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 77222.13 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519709.903280 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694999.18 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 69499.92 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589209.820929 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625499.26 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 62549.93 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651759.746813 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562949.33 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 56294.93 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708054.680108 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506654.40 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 50665.44 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758720.120074 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455988.96 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 45598.90 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804319.016044 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410390.06 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 41039.01 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845358.022416 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369351.06 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 36935.11 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882293.128151 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332415.95 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 33241.60 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915534.723313 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299174.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 29917.44 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945452.158958 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269256.92 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 26925.69 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972377.851039 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242331.23 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 24233.12 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996610.973912 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218098.11 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 21809.81 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018420.784498 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196288.30 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 19628.83 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038049.614025 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176659.47 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 17665.95 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055715.560599 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158993.52 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 15899.35 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071614.912516 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143094.17 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 14309.42 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085924.329241 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128784.75 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 12878.48 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098802.804294 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115906.28 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 11590.63 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110393.431841 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104315.65 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 10431.56 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120824.996634 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93884.08 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 9388.41 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130213.404947 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84495.67 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 8449.57 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138662.972429 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76046.11 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 7604.61 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146267.583163 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68441.50 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 6844.15 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153111.732824 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61597.35 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 6159.73 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159271.467518 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55437.61 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 5543.76 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164815.228743 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49893.85 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 4989.39 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169804.613846 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44904.47 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 4490.45 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174295.060438 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40414.02 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 4041.40 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178336.462371 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36372.62 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3637.26 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181973.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32735.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185573.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29135.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189173.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25535.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192773.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21935.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196373.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18335.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199973.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14735.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203573.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11135.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207173.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7535.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210773.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3935.36 sec) 18:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214373.724110 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 335.36 sec) 18:11:00 [GPUGRID] [rr_sim] 2214709.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973982294.09G/439.78G) 18:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853209.99 18:11:00 [---] pick_jobs_to_run() start 18:11:00 [---] pick_jobs_to_run() end 18:11:00 [---] [rr_sim] end 18:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:11:00 [---] [cpu_sched_debug] preliminary job list: 18:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:11:00 [---] [cpu_sched_debug] final job list: 18:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:11:00 [---] [cpu_sched_debug] enforce_run_list: end 18:11:00 [---] choose_project(): 65460.000000 18:11:00 [WUProp@Home] doing RPC to get work 18:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370139.09 cpu 2118092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118092.90 18:11:00 [---] Got 0 tasks 18:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:12:00 [---] [rr_sim] doing sim: CPU sched 18:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973954703.74G/439.78G) 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214646.34 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 221464.63 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221464.634289 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993181.71 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 199318.17 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420782.805148 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793863.54 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 179386.35 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600169.158922 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614477.18 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 161447.72 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761616.877318 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1453029.47 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 145302.95 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906919.823875 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307726.52 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 130772.65 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037692.475776 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176953.87 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 117695.39 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155387.862487 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059258.48 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 105925.85 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261313.710527 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953332.63 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 95333.26 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356646.973763 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857999.37 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 85799.94 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442446.910675 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772199.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 77219.94 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519666.853896 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694979.49 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 69497.95 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589164.802795 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625481.54 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 62548.15 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651712.956804 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562933.39 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 56293.34 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1708006.295412 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506640.05 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 50664.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758670.300160 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455976.04 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 45597.60 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804267.904432 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410378.44 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 41037.84 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845305.748278 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369340.59 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 36934.06 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882239.807738 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332406.54 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 33240.65 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915480.461253 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299165.88 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 29916.59 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945397.049416 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269249.29 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 26924.93 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972321.978763 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242324.36 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 24232.44 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996554.415176 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218091.93 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 21809.19 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018363.607947 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196282.73 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 19628.27 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037991.881440 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176654.46 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 17665.45 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055657.327585 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158989.02 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 15898.90 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071556.229115 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143090.11 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 14309.01 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085865.240492 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128781.10 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 12878.11 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098743.350732 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115902.99 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 11590.30 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110333.649947 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104312.69 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 10431.27 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120764.919241 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93881.42 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 9388.14 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130153.061605 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84493.28 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 8449.33 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138602.389733 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76043.95 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 7604.40 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146206.785049 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68439.56 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 6843.96 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153050.740832 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61595.60 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 6159.56 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159210.301038 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55436.04 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 5543.60 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164753.905222 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49892.44 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 4989.24 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169743.148989 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44903.19 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 4490.32 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174233.468378 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40412.87 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 4041.29 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178274.755829 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36371.59 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3637.16 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181911.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32734.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185511.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29134.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189111.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25534.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192711.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21934.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196311.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18334.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199911.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14734.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203511.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11134.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207111.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7534.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210711.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3934.43 sec) 18:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214311.914535 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 334.43 sec) 18:12:00 [GPUGRID] [rr_sim] 2214646.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973954703.74G/439.78G) 18:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853207.26 18:12:00 [---] pick_jobs_to_run() start 18:12:00 [---] pick_jobs_to_run() end 18:12:00 [---] [rr_sim] end 18:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:12:00 [---] [cpu_sched_debug] preliminary job list: 18:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:12:00 [---] [cpu_sched_debug] final job list: 18:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:12:00 [---] [cpu_sched_debug] enforce_run_list: end 18:12:00 [---] choose_project(): 65520.000000 18:12:00 [WUProp@Home] doing RPC to get work 18:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370079.09 cpu 2118032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2118032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2118032.90 18:12:00 [---] Got 0 tasks 18:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:13:00 [---] [rr_sim] doing sim: CPU sched 18:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973927113.38G/439.78G) 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214583.61 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 221458.36 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221458.360600 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993125.25 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 199312.52 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420770.885141 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793812.72 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 179381.27 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600152.157227 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614431.45 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 161443.14 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761595.302105 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452988.30 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 145298.83 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906894.132495 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307689.47 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 130768.95 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037663.079845 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176920.53 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 117692.05 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155355.132461 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059228.47 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 105922.85 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261277.979815 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953305.63 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 95330.56 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356608.542434 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857975.06 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 85797.51 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442406.048791 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772177.56 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 77217.76 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519623.804512 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694959.80 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 69495.98 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589119.784662 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625463.82 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 62546.38 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651666.166796 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562917.44 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 56291.74 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707957.910717 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506625.70 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 50662.57 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758620.480245 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455963.13 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 45596.31 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804216.792821 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410366.81 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 41036.68 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845253.474139 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369330.13 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 36933.01 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882186.487326 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332397.12 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 33239.71 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915426.199193 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299157.41 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 29915.74 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945341.939875 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269241.67 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 26924.17 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972266.106487 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242317.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 24231.75 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996497.856439 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218085.75 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 21808.57 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018306.431395 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196277.17 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 19627.72 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037934.148856 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176649.46 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 17664.95 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055599.094571 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158984.51 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 15898.45 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071497.545714 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143086.06 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 14308.61 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085806.151743 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128777.45 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 12877.75 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098683.897169 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115899.71 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 11589.97 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110273.868053 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104309.74 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 10430.97 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120704.841848 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93878.76 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 9387.88 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130092.718263 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84490.89 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 8449.09 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138541.807037 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76041.80 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 7604.18 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146145.986934 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68437.62 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 6843.76 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152989.748841 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61593.86 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 6159.39 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159149.134557 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55434.47 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 5543.45 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164692.581702 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49891.02 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 4989.10 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169681.684132 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44901.92 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 4490.19 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174171.876319 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40411.73 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 4041.17 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178213.049288 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36370.56 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3637.06 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181850.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32733.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185450.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29133.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189050.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25533.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192650.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21933.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196250.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18333.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199850.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14733.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203450.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11133.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207050.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7533.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210650.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3933.50 sec) 18:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214250.104959 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 333.50 sec) 18:13:00 [GPUGRID] [rr_sim] 2214583.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973927113.38G/439.78G) 18:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853204.52 18:13:00 [---] pick_jobs_to_run() start 18:13:00 [---] pick_jobs_to_run() end 18:13:00 [---] [rr_sim] end 18:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:13:00 [---] [cpu_sched_debug] preliminary job list: 18:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:13:00 [---] [cpu_sched_debug] final job list: 18:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:13:00 [---] [cpu_sched_debug] enforce_run_list: end 18:13:00 [---] choose_project(): 65580.000000 18:13:00 [WUProp@Home] doing RPC to get work 18:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 370019.09 cpu 2117972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117972.90 18:13:00 [---] Got 0 tasks 18:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:14:00 [---] [rr_sim] doing sim: CPU sched 18:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973899523.02G/439.78G) 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214520.87 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 221452.09 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221452.086912 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993068.78 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 199306.88 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420758.965133 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793761.90 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 179376.19 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600135.155532 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614385.71 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 161438.57 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761573.726891 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452947.14 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 145294.71 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906868.441114 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307652.43 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 130765.24 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037633.683915 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176887.19 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 117688.72 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155322.402435 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059198.47 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 105919.85 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261242.249104 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953278.62 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 95327.86 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356570.111106 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857950.76 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 85795.08 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442365.186907 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772155.68 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 77215.57 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519580.755129 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694940.11 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 69494.01 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589074.766528 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625446.10 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 62544.61 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651619.376787 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562901.49 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 56290.15 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707909.526021 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506611.34 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 50661.13 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758570.660331 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455950.21 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 45595.02 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804165.681210 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410355.19 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 41035.52 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845201.200001 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369319.67 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 36931.97 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882133.166913 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332387.70 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 33238.77 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915371.937134 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299148.93 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 29914.89 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945286.830333 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269234.04 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 26923.40 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972210.234211 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242310.63 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 24231.06 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996441.297702 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218079.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 21807.96 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018249.254844 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196271.61 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 19627.16 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037876.416272 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176644.45 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 17664.45 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055540.861557 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158980.01 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 15898.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071438.862313 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143082.01 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 14308.20 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085747.062994 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128773.81 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 12877.38 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098624.443607 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115896.43 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 11589.64 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110214.086158 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104306.78 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 10430.68 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120644.764455 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93876.10 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 9387.61 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130032.374921 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84488.49 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 8448.85 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138481.224341 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76039.64 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 7603.96 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146085.188819 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68435.68 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 6843.57 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152928.756850 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61592.11 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 6159.21 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159087.968077 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55432.90 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 5543.29 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164631.258181 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49889.61 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 4988.96 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169620.219275 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44900.65 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 4490.06 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174110.284260 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40410.58 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 4041.06 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178151.342746 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36369.53 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3636.95 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181788.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32732.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185388.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29132.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188988.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25532.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192588.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21932.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196188.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18332.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199788.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14732.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203388.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11132.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206988.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7532.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210588.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3932.57 sec) 18:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214188.295384 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332.57 sec) 18:14:00 [GPUGRID] [rr_sim] 2214520.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973899523.02G/439.78G) 18:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853201.78 18:14:00 [---] pick_jobs_to_run() start 18:14:00 [---] pick_jobs_to_run() end 18:14:00 [---] [rr_sim] end 18:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:14:00 [---] [cpu_sched_debug] preliminary job list: 18:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:14:00 [---] [cpu_sched_debug] final job list: 18:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:14:00 [---] [cpu_sched_debug] enforce_run_list: end 18:14:00 [---] choose_project(): 65640.000000 18:14:00 [WUProp@Home] doing RPC to get work 18:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369959.09 cpu 2117912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117912.90 18:14:00 [---] Got 0 tasks 18:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:15:00 [---] [rr_sim] doing sim: CPU sched 18:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973899523.02G/439.78G) 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214520.87 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 221452.09 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221452.086912 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993068.78 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 199306.88 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420758.965133 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793761.90 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 179376.19 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600135.155532 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614385.71 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 161438.57 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761573.726891 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452947.14 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 145294.71 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906868.441114 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307652.43 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 130765.24 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037633.683915 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176887.19 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 117688.72 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155322.402435 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059198.47 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 105919.85 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261242.249104 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953278.62 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 95327.86 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356570.111106 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857950.76 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 85795.08 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442365.186907 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772155.68 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 77215.57 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519580.755129 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694940.11 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 69494.01 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589074.766528 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625446.10 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 62544.61 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651619.376787 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562901.49 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 56290.15 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707909.526021 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506611.34 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 50661.13 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758570.660331 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455950.21 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 45595.02 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804165.681210 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410355.19 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 41035.52 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845201.200001 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369319.67 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 36931.97 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882133.166913 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332387.70 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 33238.77 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915371.937134 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299148.93 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 29914.89 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945286.830333 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269234.04 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 26923.40 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972210.234211 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242310.63 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 24231.06 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996441.297702 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218079.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 21807.96 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018249.254844 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196271.61 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 19627.16 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037876.416272 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176644.45 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 17664.45 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055540.861557 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158980.01 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 15898.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071438.862313 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143082.01 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 14308.20 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085747.062994 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128773.81 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 12877.38 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098624.443607 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115896.43 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 11589.64 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110214.086158 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104306.78 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 10430.68 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120644.764455 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93876.10 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 9387.61 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130032.374921 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84488.49 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 8448.85 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138481.224341 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76039.64 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 7603.96 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146085.188819 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68435.68 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 6843.57 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152928.756850 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61592.11 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 6159.21 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159087.968077 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55432.90 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 5543.29 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164631.258181 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49889.61 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 4988.96 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169620.219275 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44900.65 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 4490.06 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174110.284260 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40410.58 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 4041.06 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178151.342746 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36369.53 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3636.95 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181788.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32732.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185388.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29132.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188988.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25532.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192588.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21932.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196188.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18332.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199788.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14732.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203388.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11132.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206988.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7532.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210588.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3932.57 sec) 18:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214188.295384 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332.57 sec) 18:15:00 [GPUGRID] [rr_sim] 2214520.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973899523.02G/439.78G) 18:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853261.78 18:15:00 [---] pick_jobs_to_run() start 18:15:00 [---] pick_jobs_to_run() end 18:15:00 [---] [rr_sim] end 18:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:15:00 [---] [cpu_sched_debug] preliminary job list: 18:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:15:00 [---] [cpu_sched_debug] final job list: 18:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:15:00 [---] [cpu_sched_debug] enforce_run_list: end 18:15:00 [---] choose_project(): 65700.000000 18:15:00 [WUProp@Home] doing RPC to get work 18:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369899.09 cpu 2117912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117912.90 18:15:00 [---] Got 0 tasks 18:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:16:00 [---] [rr_sim] doing sim: CPU sched 18:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973871932.67G/439.78G) 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214458.13 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 221445.81 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221445.813224 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993012.32 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 199301.23 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420747.045125 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793711.09 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 179371.11 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600118.153837 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614339.98 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 161434.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761552.151677 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452905.98 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 145290.60 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906842.749733 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307615.38 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 130761.54 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037604.287984 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176853.84 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 117685.38 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155289.672409 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059168.46 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 105916.85 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261206.518392 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953251.61 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 95325.16 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356531.679777 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857926.45 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 85792.65 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442324.325023 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772133.81 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 77213.38 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519537.705745 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694920.43 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 69492.04 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589029.748394 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625428.38 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 62542.84 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651572.586779 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562885.55 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 56288.55 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707861.141325 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506596.99 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 50659.70 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758520.840416 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455937.29 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 45593.73 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804114.569599 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410343.56 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 41034.36 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845148.925863 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369309.21 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 36930.92 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882079.846500 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332378.29 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 33237.83 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915317.675074 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299140.46 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 29914.05 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945231.720791 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269226.41 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 26922.64 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972154.361936 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242303.77 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 24230.38 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996384.738966 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218073.39 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 21807.34 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018192.078293 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196266.05 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 19626.61 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037818.683688 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176639.45 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 17663.94 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055482.628543 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158975.50 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 15897.55 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071380.178913 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143077.95 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 14307.80 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085687.974245 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128770.16 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 12877.02 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098564.990045 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115893.14 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 11589.31 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110154.304264 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104303.83 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 10430.38 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120584.687062 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93873.45 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 9387.34 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129972.031579 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84486.10 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 8448.61 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138420.641645 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76037.49 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 7603.75 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146024.390705 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68433.74 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 6843.37 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152867.764858 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61590.37 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 6159.04 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159026.801596 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55431.33 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 5543.13 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164569.934661 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49888.20 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 4988.82 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169558.754418 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44899.38 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 4489.94 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174048.692201 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40409.44 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 4040.94 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178089.636204 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36368.50 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3636.85 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181726.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32731.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185326.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29131.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188926.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25531.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192526.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21931.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196126.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18331.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199726.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14731.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203326.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11131.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206926.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7531.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210526.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3931.65 sec) 18:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214126.485808 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331.65 sec) 18:16:00 [GPUGRID] [rr_sim] 2214458.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973871932.67G/439.78G) 18:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853259.04 18:16:00 [---] pick_jobs_to_run() start 18:16:00 [---] pick_jobs_to_run() end 18:16:00 [---] [rr_sim] end 18:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:16:00 [---] [cpu_sched_debug] preliminary job list: 18:16:00 [---] [cpu_sched_debug] final job list: 18:16:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 18:16:00 [---] [work_fetch] Request work fetch: CPUs idle 18:16:00 [---] [cpu_sched_debug] enforce_run_list: end 18:16:00 [---] choose_project(): 65760.000000 18:16:00 [WUProp@Home] doing RPC to get work 18:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369839.09 cpu 2117852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117852.90 18:16:00 [---] Got 0 tasks 18:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:17:00 [---] [rr_sim] doing sim: CPU sched 18:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973871932.67G/439.78G) 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214458.13 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 221445.81 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221445.813224 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1993012.32 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 199301.23 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420747.045125 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793711.09 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 179371.11 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600118.153837 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614339.98 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 161434.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761552.151677 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452905.98 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 145290.60 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906842.749733 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307615.38 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 130761.54 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037604.287984 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176853.84 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 117685.38 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155289.672409 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059168.46 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 105916.85 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261206.518392 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953251.61 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 95325.16 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356531.679777 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857926.45 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 85792.65 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442324.325023 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772133.81 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 77213.38 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519537.705745 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694920.43 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 69492.04 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1589029.748394 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625428.38 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 62542.84 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651572.586779 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562885.55 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 56288.55 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707861.141325 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506596.99 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 50659.70 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758520.840416 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455937.29 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 45593.73 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804114.569599 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410343.56 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 41034.36 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845148.925863 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369309.21 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 36930.92 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882079.846500 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332378.29 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 33237.83 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915317.675074 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299140.46 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 29914.05 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945231.720791 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269226.41 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 26922.64 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972154.361936 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242303.77 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 24230.38 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996384.738966 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218073.39 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 21807.34 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018192.078293 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196266.05 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 19626.61 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037818.683688 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176639.45 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 17663.94 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055482.628543 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158975.50 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 15897.55 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071380.178913 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143077.95 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 14307.80 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085687.974245 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128770.16 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 12877.02 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098564.990045 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115893.14 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 11589.31 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110154.304264 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104303.83 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 10430.38 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120584.687062 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93873.45 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 9387.34 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129972.031579 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84486.10 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 8448.61 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138420.641645 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76037.49 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 7603.75 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146024.390705 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68433.74 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 6843.37 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152867.764858 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61590.37 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 6159.04 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159026.801596 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55431.33 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 5543.13 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164569.934661 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49888.20 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 4988.82 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169558.754418 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44899.38 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 4489.94 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174048.692201 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40409.44 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 4040.94 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178089.636204 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36368.50 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3636.85 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181726.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32731.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185326.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29131.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188926.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25531.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192526.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21931.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196126.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18331.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199726.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14731.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203326.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11131.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206926.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7531.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210526.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3931.65 sec) 18:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214126.485808 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331.65 sec) 18:17:00 [GPUGRID] [rr_sim] 2214458.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973871932.67G/439.78G) 18:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853319.04 18:17:00 [---] pick_jobs_to_run() start 18:17:00 [---] pick_jobs_to_run() end 18:17:00 [---] [rr_sim] end 18:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:17:00 [---] [cpu_sched_debug] preliminary job list: 18:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:17:00 [---] [cpu_sched_debug] final job list: 18:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 18:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:17:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 973871932.67G 18:17:00 [---] [cpu_sched_debug] enforce_run_list: end 18:17:00 [---] choose_project(): 65820.000000 18:17:00 [WUProp@Home] doing RPC to get work 18:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369779.09 cpu 2117852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117852.90 18:17:00 [---] Got 0 tasks 18:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:18:00 [---] [rr_sim] doing sim: CPU sched 18:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973844342.31G/439.78G) 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214395.40 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 221439.54 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221439.539536 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992955.86 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 199295.59 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420735.125118 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793660.27 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 179366.03 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600101.152142 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614294.24 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 161429.42 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761530.576463 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452864.82 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 145286.48 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906817.058353 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307578.34 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 130757.83 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037574.892053 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176820.50 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 117682.05 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155256.942383 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059138.45 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 105913.85 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261170.787681 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953224.61 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 95322.46 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356493.248448 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857902.15 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 85790.21 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442283.463139 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772111.93 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 77211.19 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519494.656361 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694900.74 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 69490.07 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588984.730261 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625410.67 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 62541.07 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651525.796770 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562869.60 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 56286.96 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707812.756629 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506582.64 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 50658.26 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758471.020502 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455924.37 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 45592.44 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804063.457987 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410331.94 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 41033.19 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845096.651724 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369298.74 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 36929.87 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1882026.526088 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332368.87 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 33236.89 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915263.413015 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299131.98 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 29913.20 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945176.611249 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269218.78 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 26921.88 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972098.489660 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242296.91 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 24229.69 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996328.180229 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218067.22 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 21806.72 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018134.901742 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196260.49 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 19626.05 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037760.951104 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176634.44 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 17663.44 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055424.395529 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158971.00 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 15897.10 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071321.495512 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143073.90 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 14307.39 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085628.885496 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128766.51 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 12876.65 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098505.536482 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115889.86 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 11588.99 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110094.522370 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104300.87 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 10430.09 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120524.609669 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93870.79 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 9387.08 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129911.688237 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84483.71 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 8448.37 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138360.058949 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76035.34 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 7603.53 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145963.592590 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68431.80 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 6843.18 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152806.772867 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61588.62 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 6158.86 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158965.635116 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55429.76 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 5542.98 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164508.611140 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49886.78 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 4988.68 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169497.289562 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44898.11 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 4489.81 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173987.100141 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40408.30 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 4040.83 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178027.929663 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36367.47 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3636.75 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181664.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32730.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185264.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29130.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188864.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25530.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192464.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21930.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196064.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18330.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199664.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14730.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203264.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11130.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206864.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7530.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210464.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3930.72 sec) 18:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214064.676232 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330.72 sec) 18:18:00 [GPUGRID] [rr_sim] 2214395.40: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973844342.31G/439.78G) 18:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853316.31 18:18:00 [---] pick_jobs_to_run() start 18:18:00 [---] pick_jobs_to_run() end 18:18:00 [---] [rr_sim] end 18:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:18:00 [---] [cpu_sched_debug] preliminary job list: 18:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 18:18:00 [---] [cpu_sched_debug] final job list: 18:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:18:00 [---] [cpu_sched_debug] enforce_run_list: end 18:18:00 [---] choose_project(): 65880.000000 18:18:00 [WUProp@Home] doing RPC to get work 18:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369719.09 cpu 2117792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117792.90 18:18:00 [---] Got 0 tasks 18:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:19:00 [---] [rr_sim] doing sim: CPU sched 18:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973816751.96G/439.78G) 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214332.66 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 221433.27 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221433.265847 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992899.39 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 199289.94 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420723.205110 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793609.45 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 179360.95 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600084.150447 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614248.51 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 161424.85 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761509.001250 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452823.66 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 145282.37 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906791.366972 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307541.29 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 130754.13 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037545.496122 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176787.16 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 117678.72 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155224.212358 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059108.45 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 105910.84 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261135.056969 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953197.60 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 95319.76 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356454.817120 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857877.84 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 85787.78 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442242.601255 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772090.06 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 77209.01 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519451.606977 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694881.05 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 69488.11 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588939.712127 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625392.95 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 62539.29 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651479.006762 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562853.65 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 56285.37 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707764.371933 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506568.29 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 50656.83 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758421.200587 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455911.46 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 45591.15 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1804012.346376 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410320.31 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 41032.03 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1845044.377586 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369288.28 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 36928.83 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881973.205675 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332359.45 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 33235.95 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915209.150955 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299123.51 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 29912.35 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945121.501707 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269211.16 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 26921.12 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1972042.617384 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242290.04 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 24229.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996271.621493 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218061.04 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 21806.10 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018077.725191 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196254.93 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 19625.49 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037703.218519 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176629.44 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 17662.94 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055366.162515 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158966.50 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 15896.65 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071262.812111 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143069.85 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 14306.98 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085569.796747 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128762.86 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 12876.29 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098446.082920 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115886.58 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 11588.66 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110034.740476 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104297.92 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 10429.79 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120464.532275 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93868.13 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 9386.81 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129851.344895 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84481.31 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 8448.13 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138299.476253 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76033.18 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 7603.32 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145902.794476 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68429.86 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 6842.99 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152745.780875 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61586.88 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 6158.69 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158904.468635 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55428.19 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 5542.82 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164447.287619 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49885.37 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 4988.54 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169435.824705 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44896.83 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 4489.68 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173925.508082 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40407.15 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 4040.72 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177966.223121 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36366.44 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3636.64 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181602.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32729.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185202.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29129.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188802.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25529.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192402.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21929.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196002.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18329.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199602.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14729.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203202.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11129.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206802.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7529.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210402.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3929.79 sec) 18:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2214002.866657 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329.79 sec) 18:19:00 [GPUGRID] [rr_sim] 2214332.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973816751.96G/439.78G) 18:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853313.57 18:19:00 [---] pick_jobs_to_run() start 18:19:00 [---] pick_jobs_to_run() end 18:19:00 [---] [rr_sim] end 18:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:19:00 [---] [cpu_sched_debug] preliminary job list: 18:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:19:00 [---] [cpu_sched_debug] final job list: 18:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:19:00 [---] [cpu_sched_debug] enforce_run_list: end 18:19:00 [---] choose_project(): 65940.000000 18:19:00 [WUProp@Home] doing RPC to get work 18:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369659.09 cpu 2117732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117732.90 18:19:00 [---] Got 0 tasks 18:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:20:00 [---] [rr_sim] doing sim: CPU sched 18:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973789161.60G/439.78G) 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214269.92 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 221426.99 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221426.992159 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992842.93 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 199284.29 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420711.285103 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793558.64 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 179355.86 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600067.148752 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614202.77 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 161420.28 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761487.426036 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452782.50 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 145278.25 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906765.675591 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307504.25 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 130750.42 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037516.100192 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176753.82 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 117675.38 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155191.482332 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059078.44 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 105907.84 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261099.326258 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953170.60 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 95317.06 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356416.385791 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857853.54 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 85785.35 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442201.739371 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772068.18 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 77206.82 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519408.557594 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694861.36 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 69486.14 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588894.693993 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625375.23 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 62537.52 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651432.216753 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562837.70 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 56283.77 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707715.987237 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506553.93 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 50655.39 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758371.380673 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455898.54 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 45589.85 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803961.234765 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410308.69 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 41030.87 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844992.103448 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369277.82 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 36927.78 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881919.885262 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332350.04 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 33235.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915154.888895 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299115.03 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 29911.50 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945066.392165 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269203.53 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 26920.35 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971986.745108 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242283.18 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 24228.32 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996215.062756 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218054.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 21805.49 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018020.548640 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196249.37 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 19624.94 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037645.485935 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176624.44 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 17662.44 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055307.929501 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158961.99 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 15896.20 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071204.128710 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143065.79 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 14306.58 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085510.707998 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128759.21 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 12875.92 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098386.629358 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115883.29 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 11588.33 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109974.958581 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104294.96 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 10429.50 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120404.454882 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93865.47 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 9386.55 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129791.001553 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84478.92 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 8447.89 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138238.893557 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76031.03 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 7603.10 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145841.996361 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68427.93 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 6842.79 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152684.788884 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61585.13 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 6158.51 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158843.302155 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55426.62 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 5542.66 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164385.964099 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49883.96 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 4988.40 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169374.359848 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44895.56 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 4489.56 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173863.916023 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40406.01 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 4040.60 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177904.516580 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36365.41 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3636.54 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181541.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32728.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185141.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29128.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188741.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25528.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192341.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21928.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195941.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18328.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199541.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14728.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203141.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11128.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206741.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7528.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210341.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3928.86 sec) 18:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213941.057081 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328.86 sec) 18:20:00 [GPUGRID] [rr_sim] 2214269.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973789161.60G/439.78G) 18:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853310.83 18:20:00 [---] pick_jobs_to_run() start 18:20:00 [---] pick_jobs_to_run() end 18:20:00 [---] [rr_sim] end 18:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:20:00 [---] [cpu_sched_debug] preliminary job list: 18:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:20:00 [---] [cpu_sched_debug] final job list: 18:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:20:00 [---] [cpu_sched_debug] enforce_run_list: end 18:20:00 [---] choose_project(): 66000.000000 18:20:00 [WUProp@Home] doing RPC to get work 18:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369599.09 cpu 2117672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117672.90 18:20:00 [---] Got 0 tasks 18:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:21:00 [---] [rr_sim] doing sim: CPU sched 18:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973761571.24G/439.78G) 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214207.18 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 221420.72 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221420.718471 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992786.47 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 199278.65 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420699.365095 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793507.82 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 179350.78 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600050.147057 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614157.04 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 161415.70 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761465.850822 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452741.33 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 145274.13 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906739.984211 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307467.20 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 130746.72 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037486.704261 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176720.48 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 117672.05 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155158.752306 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059048.43 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 105904.84 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261063.595546 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953143.59 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 95314.36 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356377.954463 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857829.23 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 85782.92 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442160.877487 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772046.31 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 77204.63 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519365.508210 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694841.68 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 69484.17 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588849.675860 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625357.51 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 62535.75 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651385.426745 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562821.76 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 56282.18 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707667.602541 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506539.58 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 50653.96 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758321.560758 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455885.62 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 45588.56 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803910.123154 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410297.06 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 41029.71 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844939.829309 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369267.36 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 36926.74 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881866.564849 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332340.62 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 33234.06 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915100.626836 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299106.56 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 29910.66 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1945011.282623 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269195.90 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 26919.59 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971930.872832 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242276.31 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 24227.63 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996158.504020 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218048.68 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 21804.87 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017963.372089 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196243.81 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 19624.38 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037587.753351 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176619.43 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 17661.94 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055249.696487 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158957.49 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 15895.75 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071145.445309 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143061.74 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 14306.17 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085451.619249 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128755.57 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 12875.56 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098327.175795 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115880.01 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 11588.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109915.176687 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104292.01 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 10429.20 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120344.377489 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93862.81 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 9386.28 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129730.658211 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84476.53 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 8447.65 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138178.310861 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76028.87 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 7602.89 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145781.198246 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68425.99 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 6842.60 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152623.796893 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61583.39 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 6158.34 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158782.135675 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55425.05 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 5542.50 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164324.640578 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49882.54 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 4988.25 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169312.894991 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44894.29 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 4489.43 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173802.323963 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40404.86 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 4040.49 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177842.810038 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36364.37 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3636.44 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181479.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32727.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185079.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29127.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188679.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25527.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192279.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21927.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195879.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18327.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199479.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14727.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203079.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11127.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206679.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7527.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210279.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3927.94 sec) 18:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213879.247505 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 327.94 sec) 18:21:00 [GPUGRID] [rr_sim] 2214207.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973761571.24G/439.78G) 18:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853308.10 18:21:00 [---] pick_jobs_to_run() start 18:21:00 [---] pick_jobs_to_run() end 18:21:00 [---] [rr_sim] end 18:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:21:00 [---] [cpu_sched_debug] preliminary job list: 18:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:21:00 [---] [cpu_sched_debug] final job list: 18:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:21:00 [---] [cpu_sched_debug] enforce_run_list: end 18:21:00 [---] choose_project(): 66060.000000 18:21:00 [WUProp@Home] doing RPC to get work 18:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369539.09 cpu 2117612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117612.90 18:21:00 [---] Got 0 tasks 18:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:22:00 [---] [rr_sim] doing sim: CPU sched 18:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973733980.89G/439.78G) 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214144.45 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 221414.44 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221414.444783 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992730.00 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 199273.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420687.445087 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793457.00 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 179345.70 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600033.145361 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614111.30 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 161411.13 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761444.275608 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452700.17 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 145270.02 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906714.292830 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307430.15 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 130743.02 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037457.308330 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176687.14 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 117668.71 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155126.022280 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1059018.43 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 105901.84 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1261027.864835 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953116.58 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 95311.66 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356339.523134 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857804.92 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 85780.49 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442120.015604 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772024.43 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 77202.44 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519322.458826 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694821.99 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 69482.20 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588804.657726 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625339.79 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 62533.98 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651338.636736 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562805.81 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 56280.58 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707619.217846 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506525.23 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 50652.52 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758271.740844 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455872.71 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 45587.27 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803859.011542 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410285.44 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 41028.54 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844887.555171 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369256.89 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 36925.69 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881813.244437 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332331.20 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 33233.12 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1915046.364776 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299098.08 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 29909.81 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944956.173081 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269188.27 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 26918.83 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971875.000556 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242269.45 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 24226.94 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996101.945283 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218042.50 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 21804.25 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017906.195538 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196238.25 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 19623.83 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037530.020767 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176614.43 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 17661.44 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055191.463473 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158952.98 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 15895.30 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071086.761908 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143057.69 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 14305.77 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085392.530500 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128751.92 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 12875.19 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098267.722233 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115876.73 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 11587.67 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109855.394793 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104289.05 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 10428.91 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120284.300096 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93860.15 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 9386.01 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129670.314870 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84474.13 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 8447.41 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138117.728165 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76026.72 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 7602.67 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145720.400132 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68424.05 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 6842.40 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152562.804901 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61581.64 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 6158.16 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158720.969194 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55423.48 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 5542.35 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164263.317057 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49881.13 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 4988.11 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169251.430135 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44893.02 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 4489.30 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173740.731904 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40403.72 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 4040.37 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177781.103496 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36363.34 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3636.33 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181417.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32727.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185017.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29127.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188617.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25527.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192217.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21927.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195817.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18327.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199417.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14727.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203017.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11127.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206617.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7527.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210217.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3927.01 sec) 18:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213817.437930 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 327.01 sec) 18:22:00 [GPUGRID] [rr_sim] 2214144.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973733980.89G/439.78G) 18:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853305.36 18:22:00 [---] pick_jobs_to_run() start 18:22:00 [---] pick_jobs_to_run() end 18:22:00 [---] [rr_sim] end 18:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:22:00 [---] [cpu_sched_debug] preliminary job list: 18:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:22:00 [---] [cpu_sched_debug] final job list: 18:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:22:00 [---] [cpu_sched_debug] enforce_run_list: end 18:22:00 [---] choose_project(): 66120.000000 18:22:00 [WUProp@Home] doing RPC to get work 18:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369479.09 cpu 2117552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117552.90 18:22:00 [---] Got 0 tasks 18:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:23:00 [---] [rr_sim] doing sim: CPU sched 18:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973706390.53G/439.78G) 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214081.71 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 221408.17 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221408.171095 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992673.54 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 199267.35 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420675.525080 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793406.19 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 179340.62 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 600016.143666 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614065.57 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 161406.56 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761422.700394 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452659.01 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 145265.90 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906688.601450 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307393.11 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 130739.31 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037427.912399 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176653.80 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 117665.38 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155093.292254 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058988.42 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 105898.84 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260992.134123 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953089.58 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 95308.96 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356301.091805 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857780.62 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 85778.06 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442079.153720 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 772002.56 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 77200.26 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519279.409442 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694802.30 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 69480.23 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588759.639593 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625322.07 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 62532.21 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651291.846728 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562789.86 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 56278.99 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707570.833150 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506510.88 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 50651.09 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758221.920929 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455859.79 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 45585.98 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803807.899931 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410273.81 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 41027.38 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844835.281033 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369246.43 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 36924.64 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881759.924024 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332321.79 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 33232.18 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914992.102716 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299089.61 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 29908.96 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944901.063539 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269180.65 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 26918.06 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971819.128280 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242262.58 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 24226.26 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996045.386547 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218036.32 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 21803.63 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017849.018987 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196232.69 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 19623.27 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037472.288182 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176609.42 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 17660.94 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055133.230459 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158948.48 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 15894.85 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071028.078508 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143053.63 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 14305.36 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085333.441751 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128748.27 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 12874.83 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098208.268671 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115873.44 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 11587.34 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109795.612898 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104286.10 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 10428.61 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120224.222703 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93857.49 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 9385.75 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129609.971528 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84471.74 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 8447.17 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138057.145469 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76024.57 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 7602.46 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145659.602017 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68422.11 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 6842.21 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152501.812910 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61579.90 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 6157.99 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158659.802714 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55421.91 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 5542.19 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164201.993537 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49879.72 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 4987.97 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169189.965278 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44891.75 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 4489.17 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173679.139845 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40402.57 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 4040.26 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177719.396955 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36362.31 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3636.23 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181355.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32726.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184955.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29126.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188555.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25526.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192155.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21926.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195755.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18326.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199355.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14726.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202955.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11126.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206555.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7526.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210155.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3926.08 sec) 18:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213755.628354 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 326.08 sec) 18:23:00 [GPUGRID] [rr_sim] 2214081.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973706390.53G/439.78G) 18:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853302.62 18:23:00 [---] pick_jobs_to_run() start 18:23:00 [---] pick_jobs_to_run() end 18:23:00 [---] [rr_sim] end 18:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:23:00 [---] [cpu_sched_debug] preliminary job list: 18:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:23:00 [---] [cpu_sched_debug] final job list: 18:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:23:00 [---] [cpu_sched_debug] enforce_run_list: end 18:23:00 [---] choose_project(): 66180.000000 18:23:00 [WUProp@Home] doing RPC to get work 18:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369419.09 cpu 2117492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117492.90 18:23:00 [---] Got 0 tasks 18:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:24:00 [---] [rr_sim] doing sim: CPU sched 18:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973678800.17G/439.78G) 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2214018.97 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 221401.90 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221401.897406 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992617.08 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 199261.71 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420663.605072 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793355.37 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 179335.54 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599999.141971 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1614019.83 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 161401.98 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761401.125181 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452617.85 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 145261.78 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906662.910069 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307356.06 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 130735.61 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037398.516468 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176620.46 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 117662.05 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155060.562228 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058958.41 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 105895.84 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260956.403412 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953062.57 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 95306.26 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356262.660477 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857756.31 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 85775.63 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1442038.291836 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771980.68 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 77198.07 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519236.360058 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694782.61 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 69478.26 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588714.621459 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625304.35 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 62530.44 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651245.056720 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562773.92 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 56277.39 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707522.448454 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506496.53 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 50649.65 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758172.101015 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455846.87 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 45584.69 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803756.788320 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410262.19 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 41026.22 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844783.006894 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369235.97 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 36923.60 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881706.603611 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332312.37 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 33231.24 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914937.840657 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299081.13 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 29908.11 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944845.953997 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269173.02 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 26917.30 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971763.256004 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242255.72 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 24225.57 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995988.827810 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218030.15 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 21803.01 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017791.842435 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196227.13 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 19622.71 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037414.555598 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176604.42 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 17660.44 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055074.997445 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158943.98 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 15894.40 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070969.395107 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143049.58 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 14304.96 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085274.353002 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128744.62 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 12874.46 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098148.815109 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115870.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 11587.02 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109735.831004 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104283.14 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 10428.31 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120164.145310 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93854.83 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 9385.48 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129549.628186 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84469.35 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 8446.93 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137996.562773 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76022.41 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 7602.24 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145598.803902 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68420.17 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 6842.02 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152440.820919 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61578.15 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 6157.82 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158598.636233 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55420.34 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 5542.03 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164140.670016 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49878.30 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 4987.83 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169128.500421 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44890.47 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 4489.05 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173617.547785 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40401.43 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 4040.14 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177657.690413 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36361.28 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3636.13 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181293.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32725.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184893.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29125.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188493.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25525.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192093.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21925.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195693.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18325.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199293.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14725.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202893.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11125.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206493.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7525.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210093.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3925.16 sec) 18:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213693.818778 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 325.16 sec) 18:24:00 [GPUGRID] [rr_sim] 2214018.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973678800.17G/439.78G) 18:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853299.89 18:24:00 [---] pick_jobs_to_run() start 18:24:00 [---] pick_jobs_to_run() end 18:24:00 [---] [rr_sim] end 18:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:24:00 [---] [cpu_sched_debug] preliminary job list: 18:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:24:00 [---] [cpu_sched_debug] final job list: 18:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:24:00 [---] [cpu_sched_debug] enforce_run_list: end 18:24:00 [---] choose_project(): 66240.000000 18:24:00 [WUProp@Home] doing RPC to get work 18:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369359.09 cpu 2117432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117432.90 18:24:00 [---] Got 0 tasks 18:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:25:00 [---] [rr_sim] doing sim: CPU sched 18:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973651209.82G/439.78G) 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213956.24 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 221395.62 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221395.623718 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992560.61 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 199256.06 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420651.685065 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793304.55 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 179330.46 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599982.140276 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613974.10 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 161397.41 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761379.549967 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452576.69 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 145257.67 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906637.218688 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307319.02 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 130731.90 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037369.120538 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176587.12 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 117658.71 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1155027.832202 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058928.40 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 105892.84 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260920.672700 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953035.56 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 95303.56 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356224.229148 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857732.01 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 85773.20 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441997.429952 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771958.81 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 77195.88 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519193.310675 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694762.93 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 69476.29 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588669.603325 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625286.63 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 62528.66 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651198.266711 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562757.97 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 56275.80 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707474.063758 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506482.17 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 50648.22 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758122.281100 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455833.96 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 45583.40 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803705.676709 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410250.56 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 41025.06 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844730.732756 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369225.50 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 36922.55 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881653.283199 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332302.95 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 33230.30 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914883.578597 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299072.66 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 29907.27 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944790.844455 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269165.39 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 26916.54 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971707.383728 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242248.85 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 24224.89 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995932.269073 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218023.97 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 21802.40 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017734.665884 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196221.57 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 19622.16 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037356.823014 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176599.41 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 17659.94 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055016.764431 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158939.47 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 15893.95 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070910.711706 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143045.53 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 14304.55 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085215.264254 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128740.97 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 12874.10 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098089.361546 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115866.88 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 11586.69 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109676.049110 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104280.19 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 10428.02 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120104.067917 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93852.17 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 9385.22 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129489.284844 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84466.95 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 8446.70 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137935.980077 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76020.26 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 7602.03 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145538.005788 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68418.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 6841.82 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152379.828927 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61576.41 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 6157.64 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158537.469753 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55418.77 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 5541.88 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164079.346496 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49876.89 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 4987.69 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169067.035564 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44889.20 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 4488.92 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173555.955726 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40400.28 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 4040.03 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177595.983872 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36360.25 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3636.03 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181232.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32724.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184832.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29124.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188432.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25524.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192032.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21924.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195632.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18324.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199232.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14724.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202832.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11124.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206432.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7524.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210032.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3924.23 sec) 18:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213632.009203 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 324.23 sec) 18:25:00 [GPUGRID] [rr_sim] 2213956.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973651209.82G/439.78G) 18:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853297.15 18:25:00 [---] pick_jobs_to_run() start 18:25:00 [---] pick_jobs_to_run() end 18:25:00 [---] [rr_sim] end 18:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:25:00 [---] [cpu_sched_debug] preliminary job list: 18:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:25:00 [---] [cpu_sched_debug] final job list: 18:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:25:00 [---] [cpu_sched_debug] enforce_run_list: end 18:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:26:00 [---] [rr_sim] doing sim: CPU sched 18:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973623619.46G/439.78G) 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213893.50 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 221389.35 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221389.350030 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992504.15 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 199250.42 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420639.765057 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793253.74 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 179325.37 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599965.138581 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613928.36 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 161392.84 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761357.974753 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452535.53 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 145253.55 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906611.527308 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307281.97 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 130728.20 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037339.724607 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176553.78 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 117655.38 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154995.102176 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058898.40 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 105889.84 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260884.941989 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 953008.56 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 95300.86 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356185.797820 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857707.70 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 85770.77 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441956.568068 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771936.93 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 77193.69 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519150.261291 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694743.24 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 69474.32 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588624.585192 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625268.92 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 62526.89 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651151.476703 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562742.02 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 56274.20 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707425.679062 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506467.82 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 50646.78 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758072.461186 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455821.04 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 45582.10 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803654.565097 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410238.94 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 41023.89 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844678.458618 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369215.04 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 36921.50 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881599.962786 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332293.54 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 33229.35 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914829.316537 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299064.18 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 29906.42 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944735.734913 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269157.77 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 26915.78 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971651.511452 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242241.99 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 24224.20 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995875.710337 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218017.79 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 21801.78 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017677.489333 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196216.01 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 19621.60 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037299.090430 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176594.41 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 17659.44 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054958.531417 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158934.97 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 15893.50 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070852.028305 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143041.47 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 14304.15 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085156.175505 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128737.32 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 12873.73 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098029.907984 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115863.59 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 11586.36 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109616.267216 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104277.23 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 10427.72 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120043.990524 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93849.51 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 9384.95 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129428.941502 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84464.56 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 8446.46 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137875.397381 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76018.10 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 7601.81 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145477.207673 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68416.29 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 6841.63 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152318.836936 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61574.66 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 6157.47 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158476.303272 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55417.20 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 5541.72 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164018.022975 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49875.48 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 4987.55 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169005.570707 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44887.93 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 4488.79 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173494.363667 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40399.14 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 4039.91 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177534.277330 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36359.22 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3635.92 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181170.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32723.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184770.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29123.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188370.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25523.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191970.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21923.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195570.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18323.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199170.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14723.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202770.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11123.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206370.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7523.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209970.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3923.30 sec) 18:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213570.199627 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 323.30 sec) 18:26:00 [GPUGRID] [rr_sim] 2213893.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973623619.46G/439.78G) 18:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853294.41 18:26:00 [---] pick_jobs_to_run() start 18:26:00 [---] pick_jobs_to_run() end 18:26:00 [---] [rr_sim] end 18:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:26:00 [---] [cpu_sched_debug] preliminary job list: 18:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:26:00 [---] [cpu_sched_debug] final job list: 18:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:26:00 [---] [cpu_sched_debug] enforce_run_list: end 18:26:00 [---] choose_project(): 66360.000000 18:26:00 [WUProp@Home] doing RPC to get work 18:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369239.09 cpu 2117312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117312.90 18:26:00 [---] Got 0 tasks 18:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:27:00 [---] [rr_sim] doing sim: CPU sched 18:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973596029.11G/439.78G) 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213830.76 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 221383.08 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221383.076342 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992447.69 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 199244.77 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420627.845049 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793202.92 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 179320.29 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599948.136886 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613882.63 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 161388.26 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761336.399539 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452494.36 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 145249.44 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906585.835927 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307244.93 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 130724.49 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037310.328676 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176520.43 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 117652.04 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154962.372150 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058868.39 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 105886.84 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260849.211277 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952981.55 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 95298.16 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356147.366491 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857683.40 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 85768.34 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441915.706184 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771915.06 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 77191.51 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519107.211907 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694723.55 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 69472.36 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588579.567058 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625251.20 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 62525.12 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651104.686694 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562726.08 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 56272.61 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707377.294366 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506453.47 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 50645.35 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1758022.641272 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455808.12 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 45580.81 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803603.453486 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410227.31 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 41022.73 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844626.184479 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369204.58 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 36920.46 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881546.642373 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332284.12 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 33228.41 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914775.054478 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299055.71 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 29905.57 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944680.625372 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269150.14 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 26915.01 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971595.639176 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242235.12 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 24223.51 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995819.151600 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218011.61 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 21801.16 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017620.312782 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196210.45 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 19621.05 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037241.357846 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176589.41 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 17658.94 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054900.298403 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158930.47 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 15893.05 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070793.344904 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143037.42 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 14303.74 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085097.086756 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128733.68 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 12873.37 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097970.454422 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115860.31 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 11586.03 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109556.485321 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104274.28 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 10427.43 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119983.913131 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93846.85 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 9384.69 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129368.598160 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84462.17 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 8446.22 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137814.814685 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76015.95 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 7601.59 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145416.409559 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68414.35 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 6841.44 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152257.844945 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61572.92 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 6157.29 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158415.136792 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55415.63 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 5541.56 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163956.699454 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49874.06 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 4987.41 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168944.105851 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44886.66 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 4488.67 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173432.771607 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40397.99 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 4039.80 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177472.570788 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36358.19 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3635.82 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181108.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32722.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184708.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29122.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188308.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25522.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191908.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21922.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195508.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18322.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199108.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14722.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202708.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11122.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206308.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7522.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209908.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3922.37 sec) 18:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213508.390051 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 322.37 sec) 18:27:00 [GPUGRID] [rr_sim] 2213830.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973596029.11G/439.78G) 18:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853291.68 18:27:00 [---] pick_jobs_to_run() start 18:27:00 [---] pick_jobs_to_run() end 18:27:00 [---] [rr_sim] end 18:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:27:00 [---] [cpu_sched_debug] preliminary job list: 18:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:27:00 [---] [cpu_sched_debug] final job list: 18:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:27:00 [---] [cpu_sched_debug] enforce_run_list: end 18:27:00 [---] choose_project(): 66420.000000 18:27:00 [WUProp@Home] doing RPC to get work 18:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369179.09 cpu 2117252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117252.90 18:27:00 [---] Got 0 tasks 18:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:28:00 [---] [rr_sim] doing sim: CPU sched 18:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973568438.75G/439.78G) 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213768.03 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 221376.80 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221376.802654 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992391.22 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 199239.12 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420615.925042 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793152.10 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 179315.21 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599931.135191 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613836.89 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 161383.69 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761314.824326 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452453.20 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 145245.32 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906560.144547 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307207.88 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 130720.79 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037280.932745 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176487.09 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 117648.71 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154929.642124 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058838.38 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 105883.84 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260813.480565 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952954.55 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 95295.45 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356108.935162 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857659.09 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 85765.91 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441874.844300 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771893.18 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 77189.32 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519064.162523 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694703.86 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 69470.39 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588534.548925 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625233.48 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 62523.35 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651057.896686 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562710.13 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 56271.01 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707328.909671 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506439.12 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 50643.91 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757972.821357 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455795.21 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 45579.52 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803552.341875 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410215.68 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 41021.57 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844573.910341 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369194.12 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 36919.41 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881493.321960 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332274.70 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 33227.47 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914720.792418 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299047.23 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 29904.72 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944625.515830 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269142.51 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 26914.25 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971539.766900 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242228.26 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 24222.83 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995762.592864 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218005.43 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 21800.54 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017563.136231 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196204.89 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 19620.49 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037183.625261 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176584.40 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 17658.44 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054842.065389 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158925.96 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 15892.60 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070734.661503 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143033.37 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 14303.34 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085037.998007 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128730.03 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 12873.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097911.000859 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115857.03 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 11585.70 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109496.703427 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104271.32 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 10427.13 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119923.835738 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93844.19 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 9384.42 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129308.254818 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84459.77 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 8445.98 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137754.231989 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76013.79 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 7601.38 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145355.611444 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68412.42 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 6841.24 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152196.852953 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61571.17 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 6157.12 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158353.970311 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55414.06 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 5541.41 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163895.375934 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49872.65 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 4987.27 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168882.640994 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44885.39 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 4488.54 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173371.179548 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40396.85 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 4039.68 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177410.864247 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36357.16 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3635.72 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181046.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32721.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184646.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29121.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188246.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25521.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191846.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21921.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195446.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18321.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199046.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14721.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202646.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11121.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206246.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7521.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209846.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3921.45 sec) 18:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213446.580476 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 321.45 sec) 18:28:00 [GPUGRID] [rr_sim] 2213768.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973568438.75G/439.78G) 18:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853288.94 18:28:00 [---] pick_jobs_to_run() start 18:28:00 [---] pick_jobs_to_run() end 18:28:00 [---] [rr_sim] end 18:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:28:00 [---] [cpu_sched_debug] preliminary job list: 18:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:28:00 [---] [cpu_sched_debug] final job list: 18:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:28:00 [---] [cpu_sched_debug] enforce_run_list: end 18:28:00 [---] choose_project(): 66480.000000 18:28:00 [WUProp@Home] doing RPC to get work 18:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369119.09 cpu 2117192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117192.90 18:28:00 [---] Got 0 tasks 18:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:29:00 [---] [rr_sim] doing sim: CPU sched 18:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973540848.39G/439.78G) 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213705.29 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 221370.53 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221370.528965 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992334.76 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 199233.48 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420604.005034 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793101.28 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 179310.13 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599914.133496 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613791.16 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 161379.12 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761293.249112 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452412.04 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 145241.20 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906534.453166 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307170.84 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 130717.08 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037251.536815 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176453.75 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 117645.38 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154896.912098 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058808.38 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 105880.84 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260777.749854 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952927.54 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 95292.75 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356070.503834 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857634.79 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 85763.48 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441833.982416 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771871.31 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 77187.13 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1519021.113140 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694684.18 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 69468.42 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588489.530791 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625215.76 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 62521.58 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1651011.106677 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562694.18 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 56269.42 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707280.524975 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506424.76 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 50642.48 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757923.001443 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455782.29 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 45578.23 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803501.230264 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410204.06 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 41020.41 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844521.636203 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369183.65 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 36918.37 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881440.001548 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332265.29 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 33226.53 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914666.530358 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299038.76 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 29903.88 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944570.406288 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269134.88 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 26913.49 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971483.894624 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242221.40 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 24222.14 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995706.034127 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217999.26 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 21799.93 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017505.959680 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196199.33 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 19619.93 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037125.892677 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176579.40 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 17657.94 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054783.832375 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158921.46 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 15892.15 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070675.978103 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143029.31 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 14302.93 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084978.909258 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128726.38 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 12872.64 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097851.547297 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115853.74 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 11585.37 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109436.921533 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104268.37 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 10426.84 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119863.758345 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93841.53 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 9384.15 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129247.911476 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84457.38 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 8445.74 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137693.649293 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76011.64 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 7601.16 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145294.813329 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68410.48 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 6841.05 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152135.860962 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61569.43 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 6156.94 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158292.803831 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55412.49 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 5541.25 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163834.052413 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49871.24 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 4987.12 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168821.176137 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44884.11 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 4488.41 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173309.587489 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40395.70 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 4039.57 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177349.157705 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36356.13 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3635.61 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180984.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32720.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184584.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29120.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188184.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25520.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191784.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21920.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195384.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18320.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198984.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14720.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202584.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11120.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206184.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7520.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209784.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3920.52 sec) 18:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213384.770900 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 320.52 sec) 18:29:00 [GPUGRID] [rr_sim] 2213705.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973540848.39G/439.78G) 18:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853286.20 18:29:00 [---] pick_jobs_to_run() start 18:29:00 [---] pick_jobs_to_run() end 18:29:00 [---] [rr_sim] end 18:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:29:00 [---] [cpu_sched_debug] preliminary job list: 18:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:29:00 [---] [cpu_sched_debug] final job list: 18:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:29:00 [---] [cpu_sched_debug] enforce_run_list: end 18:29:00 [---] choose_project(): 66540.000000 18:29:00 [WUProp@Home] doing RPC to get work 18:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 369059.09 cpu 2117132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117132.90 18:29:00 [---] Got 0 tasks 18:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:30:00 [---] [rr_sim] doing sim: CPU sched 18:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973513258.04G/439.78G) 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213642.55 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 221364.26 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221364.255277 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992278.30 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 199227.83 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420592.085027 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1793050.47 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 179305.05 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599897.131801 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613745.42 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 161374.54 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761271.673898 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452370.88 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 145237.09 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906508.761785 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307133.79 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 130713.38 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037222.140884 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176420.41 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 117642.04 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154864.182073 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058778.37 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 105877.84 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260742.019142 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952900.53 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 95290.05 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1356032.072505 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857610.48 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 85761.05 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441793.120532 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771849.43 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 77184.94 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518978.063756 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694664.49 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 69466.45 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588444.512657 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625198.04 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 62519.80 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650964.316669 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562678.24 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 56267.82 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707232.140279 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506410.41 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 50641.04 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757873.181528 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455769.37 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 45576.94 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803450.118652 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410192.43 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 41019.24 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844469.362064 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369173.19 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 36917.32 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881386.681135 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332255.87 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 33225.59 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914612.268299 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299030.28 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 29903.03 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944515.296746 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269127.26 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 26912.73 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971428.022348 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242214.53 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 24221.45 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995649.475391 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217993.08 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 21799.31 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017448.783129 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196193.77 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 19619.38 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037068.160093 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176574.39 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 17657.44 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054725.599361 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158916.95 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 15891.70 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070617.294702 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143025.26 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 14302.53 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084919.820509 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128722.73 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 12872.27 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097792.093735 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115850.46 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 11585.05 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109377.139639 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104265.41 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 10426.54 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119803.680952 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93838.87 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 9383.89 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129187.568134 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84454.98 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 8445.50 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137633.066597 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76009.49 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 7600.95 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145234.015215 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68408.54 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 6840.85 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152074.868970 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61567.68 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 6156.77 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158231.637350 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55410.92 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 5541.09 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163772.728893 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49869.82 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 4986.98 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168759.711280 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44882.84 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 4488.28 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173247.995429 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40394.56 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 4039.46 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177287.451164 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36355.10 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3635.51 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180922.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32719.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184522.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29119.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188122.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25519.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191722.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21919.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195322.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18319.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198922.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14719.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202522.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11119.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206122.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7519.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209722.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3919.59 sec) 18:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213322.961324 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 319.59 sec) 18:30:00 [GPUGRID] [rr_sim] 2213642.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973513258.04G/439.78G) 18:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853283.47 18:30:00 [---] pick_jobs_to_run() start 18:30:00 [---] pick_jobs_to_run() end 18:30:00 [---] [rr_sim] end 18:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:30:00 [---] [cpu_sched_debug] preliminary job list: 18:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:30:00 [---] [cpu_sched_debug] final job list: 18:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:30:00 [---] [cpu_sched_debug] enforce_run_list: end 18:30:00 [---] choose_project(): 66600.000000 18:30:00 [WUProp@Home] doing RPC to get work 18:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368999.09 cpu 2117072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117072.90 18:30:00 [---] Got 0 tasks 18:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:31:00 [---] [rr_sim] doing sim: CPU sched 18:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973485667.68G/439.78G) 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213579.82 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 221357.98 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221357.981589 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992221.83 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 199222.18 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420580.165019 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792999.65 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 179299.97 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599880.130106 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613699.69 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 161369.97 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761250.098684 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452329.72 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 145232.97 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906483.070405 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307096.75 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 130709.67 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037192.744953 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176387.07 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 117638.71 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154831.452047 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058748.36 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 105874.84 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260706.288431 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952873.53 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 95287.35 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355993.641177 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857586.17 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 85758.62 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441752.258648 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771827.56 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 77182.76 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518935.014372 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694644.80 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 69464.48 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588399.494524 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625180.32 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 62518.03 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650917.526660 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562662.29 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 56266.23 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707183.755583 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506396.06 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 50639.61 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757823.361614 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455756.45 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 45575.65 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803399.007041 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410180.81 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 41018.08 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844417.087926 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369162.73 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 36916.27 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881333.360722 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332246.46 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 33224.65 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914558.006239 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299021.81 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 29902.18 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944460.187204 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269119.63 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 26911.96 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971372.150072 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242207.67 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 24220.77 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995592.916654 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217986.90 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 21798.69 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017391.606578 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196188.21 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 19618.82 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037010.427509 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176569.39 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 17656.94 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054667.366347 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158912.45 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 15891.24 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070558.611301 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143021.20 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 14302.12 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084860.731760 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128719.08 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 12871.91 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097732.640173 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115847.18 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 11584.72 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109317.357744 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104262.46 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 10426.25 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119743.603559 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93836.21 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 9383.62 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129127.224792 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84452.59 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 8445.26 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137572.483901 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76007.33 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 7600.73 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145173.217100 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68406.60 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 6840.66 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152013.876979 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61565.94 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 6156.59 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158170.470870 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55409.35 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 5540.93 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163711.405372 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49868.41 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 4986.84 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168698.246424 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44881.57 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 4488.16 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173186.403370 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40393.41 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 4039.34 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177225.744622 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36354.07 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3635.41 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180861.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32718.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184461.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29118.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188061.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25518.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191661.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21918.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195261.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18318.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198861.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14718.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202461.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11118.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206061.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7518.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209661.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3918.66 sec) 18:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213261.151749 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 318.66 sec) 18:31:00 [GPUGRID] [rr_sim] 2213579.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973485667.68G/439.78G) 18:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853280.73 18:31:00 [---] pick_jobs_to_run() start 18:31:00 [---] pick_jobs_to_run() end 18:31:00 [---] [rr_sim] end 18:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:31:00 [---] [cpu_sched_debug] preliminary job list: 18:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:31:00 [---] [cpu_sched_debug] final job list: 18:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:31:00 [---] [cpu_sched_debug] enforce_run_list: end 18:31:00 [---] choose_project(): 66660.000000 18:31:00 [WUProp@Home] doing RPC to get work 18:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368939.09 cpu 2117012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2117012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2117012.90 18:31:00 [---] Got 0 tasks 18:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:32:00 [---] [rr_sim] doing sim: CPU sched 18:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973458077.32G/439.78G) 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213517.08 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 221351.71 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221351.707901 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992165.37 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 199216.54 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420568.245011 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792948.83 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 179294.88 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599863.128411 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613653.95 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 161365.40 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761228.523470 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452288.56 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 145228.86 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906457.379024 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307059.70 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 130705.97 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037163.349022 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176353.73 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 117635.37 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154798.722021 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058718.36 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 105871.84 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260670.557719 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952846.52 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 95284.65 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355955.209848 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857561.87 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 85756.19 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441711.396764 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771805.68 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 77180.57 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518891.964988 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694625.11 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 69462.51 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588354.476390 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625162.60 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 62516.26 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650870.736652 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562646.34 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 56264.63 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707135.370887 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506381.71 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 50638.17 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757773.541699 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455743.54 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 45574.35 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803347.895430 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410169.18 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 41016.92 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844364.813788 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369152.27 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 36915.23 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881280.040310 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332237.04 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 33223.70 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914503.744179 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299013.33 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 29901.33 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944405.077662 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269112.00 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 26911.20 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971316.277796 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242200.80 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 24220.08 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995536.357918 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217980.72 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 21798.07 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017334.430026 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196182.65 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 19618.26 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036952.694924 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176564.38 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 17656.44 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054609.133333 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158907.95 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 15890.79 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070499.927900 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143017.15 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 14301.72 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084801.643011 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128715.44 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 12871.54 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097673.186610 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115843.89 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 11584.39 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109257.575850 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104259.50 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 10425.95 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119683.526166 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93833.55 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 9383.36 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129066.881450 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84450.20 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 8445.02 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137511.901205 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76005.18 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 7600.52 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145112.418986 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68404.66 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 6840.47 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151952.884988 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61564.19 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 6156.42 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158109.304390 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55407.77 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 5540.78 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163650.081851 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49867.00 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 4986.70 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168636.781567 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44880.30 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 4488.03 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173124.811311 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40392.27 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 4039.23 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177164.038080 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36353.04 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3635.30 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180799.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32717.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184399.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29117.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187999.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25517.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191599.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21917.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195199.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18317.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198799.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14717.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202399.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11117.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205999.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7517.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209599.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3917.74 sec) 18:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213199.342173 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 317.74 sec) 18:32:00 [GPUGRID] [rr_sim] 2213517.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973458077.32G/439.78G) 18:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853277.99 18:32:00 [---] pick_jobs_to_run() start 18:32:00 [---] pick_jobs_to_run() end 18:32:00 [---] [rr_sim] end 18:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:32:00 [---] [cpu_sched_debug] preliminary job list: 18:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:32:00 [---] [cpu_sched_debug] final job list: 18:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:32:00 [---] [cpu_sched_debug] enforce_run_list: end 18:32:00 [---] choose_project(): 66720.000000 18:32:00 [WUProp@Home] doing RPC to get work 18:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368879.09 cpu 2116952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116952.90 18:32:00 [---] Got 0 tasks 18:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:33:00 [---] [rr_sim] doing sim: CPU sched 18:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973430486.97G/439.78G) 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213454.34 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 221345.43 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221345.434212 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992108.91 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 199210.89 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420556.325004 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792898.02 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 179289.80 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599846.126716 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613608.22 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 161360.82 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761206.948257 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452247.39 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 145224.74 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906431.687643 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1307022.65 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 130702.27 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037133.953092 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176320.39 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 117632.04 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154765.991995 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058688.35 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 105868.84 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260634.827008 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952819.52 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 95281.95 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355916.778519 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857537.56 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 85753.76 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441670.534880 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771783.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 77178.38 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518848.915604 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694605.43 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 69460.54 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588309.458256 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625144.88 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 62514.49 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650823.946643 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562630.40 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 56263.04 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707086.986191 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506367.36 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 50636.74 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757723.721785 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455730.62 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 45573.06 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803296.783819 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410157.56 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 41015.76 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844312.539649 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369141.80 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 36914.18 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881226.719897 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332227.62 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 33222.76 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914449.482120 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299004.86 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 29900.49 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944349.968120 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269104.37 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 26910.44 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971260.405521 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242193.94 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 24219.39 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995479.799181 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217974.54 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 21797.45 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017277.253475 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196177.09 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 19617.71 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036894.962340 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176559.38 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 17655.94 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054550.900319 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158903.44 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 15890.34 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070441.244499 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143013.10 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 14301.31 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084742.554262 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128711.79 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 12871.18 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097613.733048 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115840.61 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 11584.06 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109197.793956 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104256.55 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 10425.65 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119623.448773 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93830.89 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 9383.09 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129006.538108 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84447.80 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 8444.78 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137451.318509 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76003.02 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 7600.30 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145051.620871 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68402.72 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 6840.27 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151891.892996 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61562.45 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 6156.24 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158048.137909 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55406.20 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 5540.62 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163588.758331 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49865.58 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 4986.56 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168575.316710 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44879.03 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 4487.90 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173063.219252 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40391.12 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 4039.11 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177102.331539 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36352.01 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3635.20 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180737.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32716.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184337.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29116.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187937.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25516.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191537.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21916.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195137.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18316.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198737.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14716.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202337.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11116.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205937.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7516.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209537.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3916.81 sec) 18:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213137.532597 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 316.81 sec) 18:33:00 [GPUGRID] [rr_sim] 2213454.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973430486.97G/439.78G) 18:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853275.25 18:33:00 [---] pick_jobs_to_run() start 18:33:00 [---] pick_jobs_to_run() end 18:33:00 [---] [rr_sim] end 18:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:33:00 [---] [cpu_sched_debug] preliminary job list: 18:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:33:00 [---] [cpu_sched_debug] final job list: 18:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:33:00 [---] [cpu_sched_debug] enforce_run_list: end 18:33:00 [---] choose_project(): 66780.000000 18:33:00 [WUProp@Home] doing RPC to get work 18:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368819.09 cpu 2116892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116892.90 18:33:00 [---] Got 0 tasks 18:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:34:00 [---] [rr_sim] doing sim: CPU sched 18:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973402896.61G/439.78G) 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213391.61 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 221339.16 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221339.160524 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1992052.44 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 199205.24 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420544.404996 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792847.20 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 179284.72 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599829.125021 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613562.48 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 161356.25 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761185.373043 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452206.23 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 145220.62 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906405.996263 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306985.61 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 130698.56 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037104.557161 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176287.05 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 117628.70 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154733.261969 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058658.34 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 105865.83 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260599.096296 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952792.51 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 95279.25 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355878.347191 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857513.26 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 85751.33 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441629.672996 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771761.93 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 77176.19 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518805.866221 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694585.74 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 69458.57 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588264.440123 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625127.17 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 62512.72 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650777.156635 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562614.45 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 56261.44 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1707038.601496 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506353.00 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 50635.30 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757673.901870 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455717.70 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 45571.77 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803245.672207 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410145.93 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 41014.59 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844260.265511 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369131.34 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 36913.13 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881173.399484 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332218.21 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 33221.82 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914395.220060 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298996.39 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 29899.64 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944294.858578 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269096.75 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 26909.67 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971204.533245 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242187.07 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 24218.71 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995423.240444 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217968.36 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 21796.84 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017220.076924 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196171.53 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 19617.15 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036837.229756 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176554.38 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 17655.44 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054492.667305 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158898.94 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 15889.89 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070382.561098 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143009.04 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 14300.90 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084683.465513 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128708.14 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 12870.81 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097554.279486 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115837.33 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 11583.73 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109138.012061 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104253.59 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 10425.36 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119563.371380 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93828.23 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 9382.82 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128946.194766 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84445.41 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 8444.54 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137390.735813 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76000.87 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 7600.09 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144990.822756 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68400.78 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 6840.08 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151830.901005 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61560.70 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 6156.07 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157986.971429 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55404.63 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 5540.46 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163527.434810 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49864.17 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 4986.42 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168513.851853 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44877.75 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 4487.78 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173001.627192 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40389.98 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 4039.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177040.624997 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36350.98 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3635.10 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180675.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32715.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184275.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29115.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187875.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25515.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191475.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21915.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195075.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18315.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198675.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14715.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202275.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11115.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205875.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7515.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209475.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3915.88 sec) 18:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213075.723022 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 315.88 sec) 18:34:00 [GPUGRID] [rr_sim] 2213391.61: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973402896.61G/439.78G) 18:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853272.52 18:34:00 [---] pick_jobs_to_run() start 18:34:00 [---] pick_jobs_to_run() end 18:34:00 [---] [rr_sim] end 18:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:34:00 [---] [cpu_sched_debug] preliminary job list: 18:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:34:00 [---] [cpu_sched_debug] final job list: 18:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:34:00 [---] [cpu_sched_debug] enforce_run_list: end 18:34:00 [---] choose_project(): 66840.000000 18:34:00 [WUProp@Home] doing RPC to get work 18:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368759.09 cpu 2116832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116832.90 18:34:00 [---] Got 0 tasks 18:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:35:00 [---] [rr_sim] doing sim: CPU sched 18:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973375306.26G/439.78G) 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213328.87 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 221332.89 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221332.886836 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991995.98 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 199199.60 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420532.484988 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792796.38 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 179279.64 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599812.123326 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613516.75 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 161351.67 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761163.797829 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452165.07 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 145216.51 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906380.304882 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306948.56 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 130694.86 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037075.161230 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176253.71 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 117625.37 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154700.531943 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058628.34 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 105862.83 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260563.365585 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952765.50 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 95276.55 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355839.915862 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857488.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 85748.90 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441588.811112 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771740.06 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 77174.01 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518762.816837 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694566.05 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 69456.61 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588219.421989 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625109.45 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 62510.94 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650730.366626 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562598.50 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 56259.85 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706990.216800 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506338.65 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 50633.87 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757624.081956 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455704.79 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 45570.48 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803194.560596 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410134.31 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 41013.43 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844207.991373 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369120.88 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 36912.09 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881120.079071 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332208.79 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 33220.88 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914340.958000 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298987.91 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 29898.79 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944239.749036 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269089.12 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 26908.91 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971148.660969 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242180.21 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 24218.02 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995366.681708 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217962.19 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 21796.22 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017162.900373 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196165.97 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 19616.60 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036779.497172 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176549.37 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 17654.94 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054434.434291 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158894.43 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 15889.44 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070323.877698 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143004.99 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 14300.50 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084624.376764 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128704.49 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 12870.45 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097494.825923 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115834.04 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 11583.40 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109078.230167 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104250.64 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 10425.06 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119503.293986 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93825.57 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 9382.56 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128885.851424 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84443.02 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 8444.30 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137330.153117 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75998.72 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 7599.87 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144930.024642 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68398.84 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 6839.88 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151769.909014 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61558.96 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 6155.90 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157925.804948 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55403.06 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 5540.31 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163466.111289 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49862.76 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 4986.28 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168452.386997 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44876.48 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 4487.65 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172940.035133 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40388.83 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 4038.88 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176978.918456 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36349.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3634.99 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180613.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32714.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184213.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29114.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187813.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25514.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191413.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21914.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195013.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18314.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198613.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14714.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202213.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11114.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205813.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7514.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209413.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3914.95 sec) 18:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2213013.913446 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 314.95 sec) 18:35:00 [GPUGRID] [rr_sim] 2213328.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973375306.26G/439.78G) 18:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853269.78 18:35:00 [---] pick_jobs_to_run() start 18:35:00 [---] pick_jobs_to_run() end 18:35:00 [---] [rr_sim] end 18:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:35:00 [---] [cpu_sched_debug] preliminary job list: 18:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:35:00 [---] [cpu_sched_debug] final job list: 18:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:35:00 [---] [cpu_sched_debug] enforce_run_list: end 18:35:00 [---] choose_project(): 66900.000000 18:35:00 [WUProp@Home] doing RPC to get work 18:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368699.09 cpu 2116772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116772.90 18:35:00 [---] Got 0 tasks 18:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:36:00 [---] [rr_sim] doing sim: CPU sched 18:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973347715.90G/439.78G) 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213266.13 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 221326.61 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221326.613148 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991939.52 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 199193.95 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420520.564981 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792745.57 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 179274.56 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599795.121631 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613471.01 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 161347.10 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761142.222615 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452123.91 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 145212.39 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906354.613502 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306911.52 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 130691.15 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037045.765299 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176220.37 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 117622.04 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154667.801917 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058598.33 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 105859.83 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260527.634873 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952738.50 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 95273.85 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355801.484534 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857464.65 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 85746.46 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441547.949228 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771718.18 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 77171.82 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518719.767453 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694546.36 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 69454.64 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588174.403856 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625091.73 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 62509.17 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650683.576618 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562582.55 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 56258.26 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706941.832104 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506324.30 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 50632.43 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757574.262041 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455691.87 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 45569.19 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803143.448985 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410122.68 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 41012.27 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844155.717234 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369110.41 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 36911.04 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881066.758659 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332199.37 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 33219.94 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914286.695941 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298979.44 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 29897.94 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944184.639494 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269081.49 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 26908.15 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971092.788693 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242173.34 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 24217.33 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995310.122971 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217956.01 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 21795.60 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017105.723822 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196160.41 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 19616.04 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036721.764588 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176544.37 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 17654.44 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054376.201277 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158889.93 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 15888.99 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070265.194297 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143000.94 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 14300.09 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084565.288015 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128700.84 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 12870.08 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097435.372361 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115830.76 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 11583.08 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109018.448273 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104247.68 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 10424.77 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119443.216593 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93822.91 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 9382.29 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128825.508082 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84440.62 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 8444.06 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137269.570421 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75996.56 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 7599.66 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144869.226527 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68396.90 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 6839.69 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151708.917022 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61557.21 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 6155.72 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157864.638468 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55401.49 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 5540.15 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163404.787769 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49861.34 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 4986.13 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168390.922140 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44875.21 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 4487.52 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172878.443074 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40387.69 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 4038.77 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176917.211914 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36348.92 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3634.89 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180552.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32714.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184152.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29114.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187752.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25514.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191352.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21914.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194952.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18314.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198552.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14714.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202152.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11114.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205752.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7514.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209352.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3914.03 sec) 18:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212952.103870 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 314.03 sec) 18:36:00 [GPUGRID] [rr_sim] 2213266.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973347715.90G/439.78G) 18:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853267.04 18:36:00 [---] pick_jobs_to_run() start 18:36:00 [---] pick_jobs_to_run() end 18:36:00 [---] [rr_sim] end 18:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:36:00 [---] [cpu_sched_debug] preliminary job list: 18:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:36:00 [---] [cpu_sched_debug] final job list: 18:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:36:00 [---] [cpu_sched_debug] enforce_run_list: end 18:36:00 [---] choose_project(): 66960.000000 18:36:00 [WUProp@Home] doing RPC to get work 18:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368639.09 cpu 2116712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116712.90 18:36:00 [---] Got 0 tasks 18:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:37:00 [---] [rr_sim] doing sim: CPU sched 18:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973320125.54G/439.78G) 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213203.39 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 221320.34 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221320.339460 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991883.06 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 199188.31 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420508.644973 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792694.75 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 179269.47 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599778.119935 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613425.27 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 161342.53 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761120.647402 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452082.75 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 145208.27 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906328.922121 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306874.47 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 130687.45 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037016.369368 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176187.03 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 117618.70 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154635.071891 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058568.32 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 105856.83 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260491.904162 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952711.49 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 95271.15 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355763.053205 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857440.34 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 85744.03 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441507.087344 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771696.31 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 77169.63 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518676.718069 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694526.68 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 69452.67 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588129.385722 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625074.01 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 62507.40 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650636.786609 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562566.61 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 56256.66 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706893.447408 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506309.95 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 50630.99 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757524.442127 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455678.95 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 45567.90 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803092.337374 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410111.06 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 41011.11 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844103.443096 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369099.95 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 36910.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881013.438246 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332189.96 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 33219.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914232.433881 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298970.96 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 29897.10 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944129.529952 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269073.86 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 26907.39 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971036.916417 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242166.48 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 24216.65 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995253.564235 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217949.83 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 21794.98 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017048.547271 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196154.85 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 19615.48 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036664.032003 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176539.36 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 17653.94 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054317.968263 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158885.43 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 15888.54 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070206.510896 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142996.88 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 14299.69 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084506.199266 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128697.20 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 12869.72 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097375.918799 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115827.48 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 11582.75 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108958.666379 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104244.73 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 10424.47 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119383.139200 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93820.26 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 9382.03 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128765.164740 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84438.23 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 8443.82 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137208.987726 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75994.41 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 7599.44 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144808.428413 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68394.97 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 6839.50 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151647.925031 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61555.47 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 6155.55 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157803.471987 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55399.92 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 5539.99 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163343.464248 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49859.93 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 4985.99 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168329.457283 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44873.94 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 4487.39 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172816.851014 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40386.54 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 4038.65 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176855.505372 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36347.89 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3634.79 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180490.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32713.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184090.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29113.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187690.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25513.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191290.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21913.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194890.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18313.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198490.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14713.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202090.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11113.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205690.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7513.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209290.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3913.10 sec) 18:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212890.294295 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 313.10 sec) 18:37:00 [GPUGRID] [rr_sim] 2213203.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973320125.54G/439.78G) 18:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853264.31 18:37:00 [---] pick_jobs_to_run() start 18:37:00 [---] pick_jobs_to_run() end 18:37:00 [---] [rr_sim] end 18:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:37:00 [---] [cpu_sched_debug] preliminary job list: 18:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:37:00 [---] [cpu_sched_debug] final job list: 18:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:37:00 [---] [cpu_sched_debug] enforce_run_list: end 18:37:00 [---] choose_project(): 67020.000000 18:37:00 [WUProp@Home] doing RPC to get work 18:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368579.09 cpu 2116652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116652.90 18:37:00 [---] Got 0 tasks 18:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:38:00 [---] [rr_sim] doing sim: CPU sched 18:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973320125.54G/439.78G) 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213203.39 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 221320.34 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221320.339460 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991883.06 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 199188.31 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420508.644973 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792694.75 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 179269.47 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599778.119935 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613425.27 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 161342.53 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761120.647402 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452082.75 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 145208.27 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906328.922121 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306874.47 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 130687.45 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1037016.369368 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176187.03 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 117618.70 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154635.071891 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058568.32 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 105856.83 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260491.904162 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952711.49 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 95271.15 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355763.053205 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857440.34 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 85744.03 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441507.087344 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771696.31 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 77169.63 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518676.718069 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694526.68 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 69452.67 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588129.385722 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625074.01 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 62507.40 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650636.786609 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562566.61 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 56256.66 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706893.447408 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506309.95 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 50630.99 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757524.442127 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455678.95 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 45567.90 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803092.337374 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410111.06 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 41011.11 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844103.443096 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369099.95 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 36910.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1881013.438246 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332189.96 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 33219.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914232.433881 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298970.96 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 29897.10 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944129.529952 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269073.86 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 26907.39 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1971036.916417 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242166.48 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 24216.65 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995253.564235 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217949.83 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 21794.98 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017048.547271 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196154.85 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 19615.48 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036664.032003 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176539.36 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 17653.94 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054317.968263 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158885.43 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 15888.54 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070206.510896 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142996.88 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 14299.69 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084506.199266 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128697.20 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 12869.72 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097375.918799 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115827.48 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 11582.75 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108958.666379 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104244.73 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 10424.47 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119383.139200 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93820.26 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 9382.03 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128765.164740 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84438.23 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 8443.82 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137208.987726 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75994.41 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 7599.44 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144808.428413 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68394.97 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 6839.50 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151647.925031 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61555.47 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 6155.55 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157803.471987 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55399.92 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 5539.99 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163343.464248 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49859.93 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 4985.99 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168329.457283 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44873.94 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 4487.39 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172816.851014 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40386.54 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 4038.65 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176855.505372 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36347.89 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3634.79 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180490.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32713.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184090.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29113.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187690.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25513.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191290.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21913.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194890.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18313.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198490.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14713.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202090.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11113.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205690.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7513.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209290.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3913.10 sec) 18:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212890.294295 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 313.10 sec) 18:38:00 [GPUGRID] [rr_sim] 2213203.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973320125.54G/439.78G) 18:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853324.31 18:38:00 [---] pick_jobs_to_run() start 18:38:00 [---] pick_jobs_to_run() end 18:38:00 [---] [rr_sim] end 18:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:38:00 [---] [cpu_sched_debug] preliminary job list: 18:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:38:00 [---] [cpu_sched_debug] final job list: 18:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:38:00 [---] [cpu_sched_debug] enforce_run_list: end 18:38:00 [---] choose_project(): 67080.000000 18:38:00 [WUProp@Home] doing RPC to get work 18:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368519.09 cpu 2116652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116652.90 18:38:00 [---] Got 0 tasks 18:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:39:00 [---] [rr_sim] doing sim: CPU sched 18:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973292535.19G/439.78G) 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213140.66 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 221314.07 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221314.065771 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991826.59 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 199182.66 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420496.724966 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792643.93 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 179264.39 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599761.118240 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613379.54 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 161337.95 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761099.072188 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452041.59 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 145204.16 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906303.230740 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306837.43 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 130683.74 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036986.973438 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176153.68 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 117615.37 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154602.341865 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058538.32 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 105853.83 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260456.173450 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952684.48 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 95268.45 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355724.621877 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857416.04 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 85741.60 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441466.225460 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771674.43 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 77167.44 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518633.668686 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694506.99 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 69450.70 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588084.367588 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625056.29 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 62505.63 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650589.996601 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562550.66 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 56255.07 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706845.062712 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506295.60 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 50629.56 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757474.622212 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455666.04 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 45566.60 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803041.225763 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410099.43 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 41009.94 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844051.168958 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369089.49 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 36908.95 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880960.117833 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332180.54 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 33218.05 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914178.171821 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298962.49 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 29896.25 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944074.420411 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269066.24 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 26906.62 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970981.044141 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242159.61 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 24215.96 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995197.005498 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217943.65 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 21794.37 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016991.370720 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196149.29 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 19614.93 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036606.299419 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176534.36 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 17653.44 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054259.735249 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158880.92 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 15888.09 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070147.827495 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142992.83 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 14299.28 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084447.110517 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128693.55 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 12869.35 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097316.465237 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115824.19 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 11582.42 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108898.884484 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104241.77 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 10424.18 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119323.061807 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93817.60 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 9381.76 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128704.821398 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84435.84 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 8443.58 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137148.405030 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75992.25 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 7599.23 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144747.630298 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68393.03 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 6839.30 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151586.933040 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61553.72 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 6155.37 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157742.305507 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55398.35 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 5539.84 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163282.140728 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49858.52 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 4985.85 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168267.992426 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44872.67 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 4487.27 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172755.258955 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40385.40 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 4038.54 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176793.798831 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36346.86 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3634.69 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180428.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32712.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184028.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29112.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187628.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25512.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191228.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21912.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194828.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18312.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198428.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14712.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202028.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11112.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205628.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7512.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209228.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3912.17 sec) 18:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212828.484719 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 312.17 sec) 18:39:00 [GPUGRID] [rr_sim] 2213140.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973292535.19G/439.78G) 18:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853321.57 18:39:00 [---] pick_jobs_to_run() start 18:39:00 [---] pick_jobs_to_run() end 18:39:00 [---] [rr_sim] end 18:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:39:00 [---] [cpu_sched_debug] preliminary job list: 18:39:00 [---] [cpu_sched_debug] final job list: 18:39:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 18:39:00 [---] [work_fetch] Request work fetch: CPUs idle 18:39:00 [---] [cpu_sched_debug] enforce_run_list: end 18:39:00 [---] choose_project(): 67140.000000 18:39:00 [WUProp@Home] doing RPC to get work 18:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368459.09 cpu 2116592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116592.90 18:39:00 [---] Got 0 tasks 18:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:40:00 [---] [rr_sim] doing sim: CPU sched 18:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973292535.19G/439.78G) 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213140.66 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 221314.07 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221314.065771 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991826.59 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 199182.66 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420496.724966 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792643.93 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 179264.39 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599761.118240 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613379.54 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 161337.95 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761099.072188 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452041.59 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 145204.16 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906303.230740 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306837.43 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 130683.74 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036986.973438 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176153.68 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 117615.37 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154602.341865 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058538.32 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 105853.83 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260456.173450 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952684.48 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 95268.45 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355724.621877 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857416.04 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 85741.60 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441466.225460 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771674.43 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 77167.44 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518633.668686 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694506.99 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 69450.70 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588084.367588 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625056.29 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 62505.63 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650589.996601 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562550.66 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 56255.07 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706845.062712 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506295.60 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 50629.56 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757474.622212 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455666.04 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 45566.60 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1803041.225763 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410099.43 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 41009.94 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1844051.168958 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369089.49 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 36908.95 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880960.117833 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332180.54 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 33218.05 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914178.171821 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298962.49 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 29896.25 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944074.420411 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269066.24 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 26906.62 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970981.044141 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242159.61 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 24215.96 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995197.005498 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217943.65 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 21794.37 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016991.370720 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196149.29 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 19614.93 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036606.299419 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176534.36 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 17653.44 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054259.735249 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158880.92 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 15888.09 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070147.827495 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142992.83 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 14299.28 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084447.110517 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128693.55 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 12869.35 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097316.465237 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115824.19 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 11582.42 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108898.884484 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104241.77 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 10424.18 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119323.061807 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93817.60 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 9381.76 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128704.821398 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84435.84 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 8443.58 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137148.405030 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75992.25 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 7599.23 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144747.630298 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68393.03 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 6839.30 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151586.933040 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61553.72 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 6155.37 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157742.305507 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55398.35 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 5539.84 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163282.140728 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49858.52 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 4985.85 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168267.992426 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44872.67 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 4487.27 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172755.258955 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40385.40 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 4038.54 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176793.798831 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36346.86 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3634.69 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180428.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32712.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184028.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29112.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187628.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25512.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191228.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21912.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194828.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18312.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198428.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14712.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202028.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11112.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205628.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7512.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209228.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3912.17 sec) 18:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212828.484719 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 312.17 sec) 18:40:00 [GPUGRID] [rr_sim] 2213140.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973292535.19G/439.78G) 18:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853381.57 18:40:00 [---] pick_jobs_to_run() start 18:40:00 [---] pick_jobs_to_run() end 18:40:00 [---] [rr_sim] end 18:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:40:00 [---] [cpu_sched_debug] preliminary job list: 18:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:40:00 [---] [cpu_sched_debug] final job list: 18:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 18:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:40:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 973292535.19G 18:40:00 [---] [cpu_sched_debug] enforce_run_list: end 18:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:41:00 [---] [rr_sim] doing sim: CPU sched 18:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973264944.83G/439.78G) 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213077.92 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 221307.79 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221307.792083 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991770.13 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 199177.01 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420484.804958 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792593.12 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 179259.31 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599744.116545 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613333.80 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 161333.38 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761077.496974 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1452000.42 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 145200.04 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906277.539360 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306800.38 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 130680.04 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036957.577507 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176120.34 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 117612.03 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154569.611839 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058508.31 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 105850.83 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260420.442739 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952657.48 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 95265.75 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355686.190548 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857391.73 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 85739.17 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441425.363576 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771652.56 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 77165.26 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518590.619302 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694487.30 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 69448.73 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1588039.349455 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625038.57 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 62503.86 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650543.206592 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562534.71 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 56253.47 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706796.678016 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506281.24 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 50628.12 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757424.802298 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455653.12 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 45565.31 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802990.114151 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410087.81 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 41008.78 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843998.894819 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369079.03 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 36907.90 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880906.797421 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332171.12 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 33217.11 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914123.909762 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298954.01 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 29895.40 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1944019.310869 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269058.61 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 26905.86 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970925.171865 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242152.75 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 24215.27 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995140.446762 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217937.47 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 21793.75 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016934.194169 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196143.73 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 19614.37 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036548.566835 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176529.35 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 17652.94 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054201.502235 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158876.42 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 15887.64 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070089.144094 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142988.78 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 14298.88 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084388.021768 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128689.90 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 12868.99 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097257.011674 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115820.91 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 11582.09 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108839.102590 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104238.82 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 10423.88 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119262.984414 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93814.94 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 9381.49 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128644.478056 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84433.44 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 8443.34 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137087.822334 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75990.10 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 7599.01 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144686.832183 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68391.09 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 6839.11 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151525.941048 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61551.98 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 6155.20 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157681.139026 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55396.78 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 5539.68 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163220.817207 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49857.10 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 4985.71 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168206.527569 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44871.39 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 4487.14 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172693.666896 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40384.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 4038.43 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176732.092289 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36345.83 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3634.58 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180366.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32711.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183966.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29111.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187566.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25511.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191166.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21911.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194766.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18311.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198366.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14711.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201966.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11111.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205566.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7511.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209166.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3911.25 sec) 18:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212766.675143 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 311.25 sec) 18:41:00 [GPUGRID] [rr_sim] 2213077.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973264944.83G/439.78G) 18:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853378.83 18:41:00 [---] pick_jobs_to_run() start 18:41:00 [---] pick_jobs_to_run() end 18:41:00 [---] [rr_sim] end 18:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:41:00 [---] [cpu_sched_debug] preliminary job list: 18:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 18:41:00 [---] [cpu_sched_debug] final job list: 18:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:41:00 [---] [cpu_sched_debug] enforce_run_list: end 18:41:00 [---] choose_project(): 67260.000000 18:41:00 [WUProp@Home] doing RPC to get work 18:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368339.09 cpu 2116532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116532.90 18:41:00 [---] Got 0 tasks 18:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:42:00 [---] [rr_sim] doing sim: CPU sched 18:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973237354.48G/439.78G) 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2213015.18 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 221301.52 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221301.518395 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991713.67 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 199171.37 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420472.884950 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792542.30 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 179254.23 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599727.114850 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613288.07 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 161328.81 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761055.921760 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451959.26 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 145195.93 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906251.847979 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306763.34 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 130676.33 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036928.181576 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176087.00 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 117608.70 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154536.881813 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058478.30 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 105847.83 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260384.712027 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952630.47 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 95263.05 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355647.759219 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857367.42 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 85736.74 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441384.501692 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771630.68 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 77163.07 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518547.569918 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694467.61 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 69446.76 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587994.331321 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625020.85 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 62502.09 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650496.416584 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562518.77 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 56251.88 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706748.293321 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506266.89 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 50626.69 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757374.982383 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455640.20 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 45564.02 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802939.002540 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410076.18 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 41007.62 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843946.620681 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369068.56 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 36906.86 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880853.477008 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332161.71 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 33216.17 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914069.647702 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298945.54 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 29894.55 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943964.201327 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269050.98 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 26905.10 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970869.299589 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242145.88 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 24214.59 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995083.888025 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217931.30 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 21793.13 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016877.017617 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196138.17 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 19613.82 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036490.834251 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176524.35 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 17652.43 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054143.269221 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158871.91 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 15887.19 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070030.460693 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142984.72 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 14298.47 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084328.933019 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128686.25 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 12868.63 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097197.558112 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115817.63 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 11581.76 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108779.320696 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104235.86 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 10423.59 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119202.907021 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93812.28 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 9381.23 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128584.134714 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84431.05 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 8443.10 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137027.239638 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75987.94 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 7598.79 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144626.034069 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68389.15 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 6838.91 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151464.949057 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61550.23 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 6155.02 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157619.972546 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55395.21 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 5539.52 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163159.493686 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49855.69 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 4985.57 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168145.062713 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44870.12 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 4487.01 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172632.074836 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40383.11 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 4038.31 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176670.385748 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36344.80 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3634.48 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180304.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32710.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183904.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29110.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187504.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25510.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191104.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21910.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194704.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18310.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198304.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14710.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201904.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11110.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205504.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7510.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209104.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3910.32 sec) 18:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212704.865568 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 310.32 sec) 18:42:00 [GPUGRID] [rr_sim] 2213015.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973237354.48G/439.78G) 18:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853376.10 18:42:00 [---] pick_jobs_to_run() start 18:42:00 [---] pick_jobs_to_run() end 18:42:00 [---] [rr_sim] end 18:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:42:00 [---] [cpu_sched_debug] preliminary job list: 18:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:42:00 [---] [cpu_sched_debug] final job list: 18:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:42:00 [---] [cpu_sched_debug] enforce_run_list: end 18:42:00 [---] choose_project(): 67320.000000 18:42:00 [WUProp@Home] doing RPC to get work 18:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368279.09 cpu 2116472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116472.90 18:42:00 [---] Got 0 tasks 18:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:43:00 [---] [rr_sim] doing sim: CPU sched 18:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973209764.12G/439.78G) 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212952.45 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 221295.24 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221295.244707 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991657.20 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 199165.72 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420460.964943 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792491.48 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 179249.15 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599710.113155 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613242.33 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 161324.23 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761034.346546 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451918.10 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 145191.81 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906226.156599 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306726.29 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 130672.63 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036898.785645 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176053.66 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 117605.37 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154504.151788 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058448.30 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 105844.83 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260348.981316 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952603.47 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 95260.35 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355609.327891 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857343.12 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 85734.31 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441343.639808 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771608.81 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 77160.88 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518504.520534 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694447.93 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 69444.79 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587949.313188 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 625003.13 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 62500.31 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650449.626576 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562502.82 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 56250.28 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706699.908625 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506252.54 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 50625.25 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757325.162469 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455627.28 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 45562.73 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802887.890929 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410064.56 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 41006.46 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843894.346543 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369058.10 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 36905.81 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880800.156595 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332152.29 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 33215.23 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1914015.385642 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298937.06 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 29893.71 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943909.091785 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269043.36 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 26904.34 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970813.427313 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242139.02 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 24213.90 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1995027.329288 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217925.12 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 21792.51 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016819.841066 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196132.61 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 19613.26 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036433.101666 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176519.35 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 17651.93 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054085.036207 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158867.41 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 15886.74 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069971.777293 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142980.67 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 14298.07 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084269.844270 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128682.60 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 12868.26 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097138.104550 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115814.34 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 11581.43 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108719.538802 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104232.91 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 10423.29 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119142.829628 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93809.62 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 9380.96 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128523.791372 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84428.66 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 8442.87 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136966.656942 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75985.79 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 7598.58 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144565.235954 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68387.21 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 6838.72 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151403.957065 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61548.49 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 6154.85 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157558.806066 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55393.64 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 5539.36 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163098.170166 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49854.28 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 4985.43 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168083.597856 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44868.85 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 4486.88 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172570.482777 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40381.96 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 4038.20 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176608.679206 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36343.77 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3634.38 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180243.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32709.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183843.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29109.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187443.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25509.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191043.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21909.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194643.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18309.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198243.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14709.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201843.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11109.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205443.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7509.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209043.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3909.39 sec) 18:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212643.055992 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 309.39 sec) 18:43:00 [GPUGRID] [rr_sim] 2212952.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973209764.12G/439.78G) 18:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853373.36 18:43:00 [---] pick_jobs_to_run() start 18:43:00 [---] pick_jobs_to_run() end 18:43:00 [---] [rr_sim] end 18:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:43:00 [---] [cpu_sched_debug] preliminary job list: 18:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:43:00 [---] [cpu_sched_debug] final job list: 18:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:43:00 [---] [cpu_sched_debug] enforce_run_list: end 18:43:00 [---] choose_project(): 67380.000000 18:43:00 [WUProp@Home] doing RPC to get work 18:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368219.09 cpu 2116412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116412.90 18:43:00 [---] Got 0 tasks 18:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:44:00 [---] [rr_sim] doing sim: CPU sched 18:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973182173.76G/439.78G) 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212889.71 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 221288.97 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221288.971019 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991600.74 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 199160.07 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420449.044935 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792440.67 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 179244.07 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599693.111460 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613196.60 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 161319.66 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 761012.771333 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451876.94 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 145187.69 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906200.465218 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306689.24 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 130668.92 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036869.389715 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1176020.32 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 117602.03 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154471.421762 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058418.29 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 105841.83 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260313.250604 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952576.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 95257.65 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355570.896562 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857318.81 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 85731.88 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441302.777924 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771586.93 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 77158.69 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518461.471150 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694428.24 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 69442.82 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587904.295054 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624985.42 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 62498.54 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650402.836567 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562486.87 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 56248.69 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706651.523929 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506238.19 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 50623.82 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757275.342554 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455614.37 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 45561.44 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802836.779318 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410052.93 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 41005.29 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843842.072404 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369047.64 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 36904.76 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880746.836182 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332142.87 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 33214.29 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913961.123583 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298928.59 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 29892.86 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943853.982243 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269035.73 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 26903.57 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970757.555037 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242132.16 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 24213.22 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994970.770552 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217918.94 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 21791.89 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016762.664515 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196127.05 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 19612.70 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036375.369082 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176514.34 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 17651.43 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054026.803192 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158862.91 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 15886.29 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069913.093892 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142976.62 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 14297.66 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084210.755521 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128678.95 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 12867.90 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097078.650987 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115811.06 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 11581.11 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108659.756907 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104229.95 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 10423.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119082.752235 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93806.96 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 9380.70 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128463.448030 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84426.26 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 8442.63 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136906.074246 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75983.64 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 7598.36 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144504.437839 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68385.27 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 6838.53 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151342.965074 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61546.75 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 6154.67 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157497.639585 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55392.07 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 5539.21 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163036.846645 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49852.86 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 4985.29 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168022.132999 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44867.58 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 4486.76 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172508.890718 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40380.82 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 4038.08 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176546.972664 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36342.74 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3634.27 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180181.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32708.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183781.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29108.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187381.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25508.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190981.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21908.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194581.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18308.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198181.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14708.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201781.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11108.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205381.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7508.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208981.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3908.46 sec) 18:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212581.246417 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 308.46 sec) 18:44:00 [GPUGRID] [rr_sim] 2212889.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973182173.76G/439.78G) 18:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853370.62 18:44:00 [---] pick_jobs_to_run() start 18:44:00 [---] pick_jobs_to_run() end 18:44:00 [---] [rr_sim] end 18:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:44:00 [---] [cpu_sched_debug] preliminary job list: 18:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:44:00 [---] [cpu_sched_debug] final job list: 18:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:44:00 [---] [cpu_sched_debug] enforce_run_list: end 18:44:00 [---] choose_project(): 67440.000000 18:44:00 [WUProp@Home] doing RPC to get work 18:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368159.09 cpu 2116352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116352.90 18:44:00 [---] Got 0 tasks 18:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:45:00 [---] [rr_sim] doing sim: CPU sched 18:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973154583.41G/439.78G) 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212826.97 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 221282.70 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221282.697330 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991544.28 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 199154.43 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420437.124928 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792389.85 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 179238.98 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599676.109765 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613150.86 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 161315.09 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760991.196119 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451835.78 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 145183.58 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906174.773837 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306652.20 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 130665.22 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036839.993784 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175986.98 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 117598.70 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154438.691736 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058388.28 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 105838.83 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260277.519892 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952549.45 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 95254.95 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355532.465234 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857294.51 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 85729.45 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441261.916040 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771565.06 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 77156.51 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518418.421767 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694408.55 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 69440.86 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587859.276920 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624967.70 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 62496.77 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650356.046559 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562470.93 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 56247.09 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706603.139233 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506223.83 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 50622.38 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757225.522640 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455601.45 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 45560.15 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802785.667706 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410041.31 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 41004.13 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843789.798266 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369037.18 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 36903.72 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880693.515770 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332133.46 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 33213.35 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913906.861523 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298920.11 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 29892.01 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943798.872701 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269028.10 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 26902.81 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970701.682761 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242125.29 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 24212.53 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994914.211815 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217912.76 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 21791.28 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016705.487964 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196121.49 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 19612.15 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036317.636498 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176509.34 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 17650.93 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053968.570178 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158858.40 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 15885.84 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069854.410491 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142972.56 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 14297.26 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084151.666772 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128675.31 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 12867.53 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097019.197425 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115807.78 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 11580.78 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108599.975013 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104227.00 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 10422.70 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119022.674842 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93804.30 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 9380.43 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128403.104688 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84423.87 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 8442.39 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136845.491550 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75981.48 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 7598.15 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144443.639725 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68383.33 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 6838.33 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151281.973083 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61545.00 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 6154.50 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157436.473105 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55390.50 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 5539.05 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162975.523125 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49851.45 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 4985.15 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167960.668142 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44866.31 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 4486.63 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172447.298658 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40379.67 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 4037.97 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176485.266123 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36341.71 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3634.17 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180119.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32707.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183719.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29107.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187319.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25507.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190919.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21907.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194519.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18307.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198119.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14707.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201719.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11107.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205319.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7507.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208919.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3907.54 sec) 18:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212519.436841 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 307.54 sec) 18:45:00 [GPUGRID] [rr_sim] 2212826.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973154583.41G/439.78G) 18:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853367.89 18:45:00 [---] pick_jobs_to_run() start 18:45:00 [---] pick_jobs_to_run() end 18:45:00 [---] [rr_sim] end 18:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:45:00 [---] [cpu_sched_debug] preliminary job list: 18:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:45:00 [---] [cpu_sched_debug] final job list: 18:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:45:00 [---] [cpu_sched_debug] enforce_run_list: end 18:45:00 [---] choose_project(): 67500.000000 18:45:00 [WUProp@Home] doing RPC to get work 18:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368099.09 cpu 2116292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116292.90 18:45:00 [---] Got 0 tasks 18:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:46:00 [---] [rr_sim] doing sim: CPU sched 18:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973126993.05G/439.78G) 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212764.24 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 221276.42 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221276.423642 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991487.81 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 199148.78 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420425.204920 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792339.03 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 179233.90 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599659.108070 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613105.13 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 161310.51 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760969.620905 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451794.62 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 145179.46 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906149.082457 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306615.15 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 130661.52 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036810.597853 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175953.64 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 117595.36 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154405.961710 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058358.27 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 105835.83 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260241.789181 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952522.45 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 95252.24 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355494.033905 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857270.20 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 85727.02 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441221.054156 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771543.18 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 77154.32 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518375.372383 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694388.86 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 69438.89 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587814.258787 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624949.98 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 62495.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650309.256550 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562454.98 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 56245.50 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706554.754537 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506209.48 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 50620.95 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757175.702726 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455588.53 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 45558.85 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802734.556095 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410029.68 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 41002.97 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843737.524128 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369026.71 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 36902.67 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880640.195357 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332124.04 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 33212.40 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913852.599463 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298911.64 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 29891.16 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943743.763159 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269020.47 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 26902.05 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970645.810485 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242118.43 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 24211.84 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994857.653079 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217906.58 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 21790.66 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016648.311413 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196115.93 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 19611.59 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036259.903914 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176504.33 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 17650.43 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053910.337164 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158853.90 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 15885.39 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069795.727090 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142968.51 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 14296.85 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084092.578023 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128671.66 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 12867.17 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096959.743863 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115804.49 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 11580.45 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108540.193119 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104224.04 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 10422.40 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118962.597449 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93801.64 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 9380.16 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128342.761346 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84421.48 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 8442.15 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136784.908854 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75979.33 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 7597.93 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144382.841610 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68381.39 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 6838.14 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151220.981091 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61543.26 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 6154.33 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157375.306624 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55388.93 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 5538.89 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162914.199604 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49850.04 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 4985.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167899.203286 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44865.03 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 4486.50 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172385.706599 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40378.53 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 4037.85 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176423.559581 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36340.68 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3634.07 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180057.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32706.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183657.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29106.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187257.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25506.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190857.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21906.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194457.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18306.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198057.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14706.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201657.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11106.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205257.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7506.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208857.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3906.61 sec) 18:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212457.627265 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 306.61 sec) 18:46:00 [GPUGRID] [rr_sim] 2212764.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973126993.05G/439.78G) 18:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853365.15 18:46:00 [---] pick_jobs_to_run() start 18:46:00 [---] pick_jobs_to_run() end 18:46:00 [---] [rr_sim] end 18:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:46:00 [---] [cpu_sched_debug] preliminary job list: 18:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:46:00 [---] [cpu_sched_debug] final job list: 18:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:46:00 [---] [cpu_sched_debug] enforce_run_list: end 18:46:00 [---] choose_project(): 67560.000000 18:46:00 [WUProp@Home] doing RPC to get work 18:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 368039.09 cpu 2116232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116232.90 18:46:00 [---] Got 0 tasks 18:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:47:00 [---] [rr_sim] doing sim: CPU sched 18:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973099402.69G/439.78G) 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212701.50 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 221270.15 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221270.149954 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991431.35 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 199143.13 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420413.284912 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792288.21 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 179228.82 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599642.106375 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613059.39 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 161305.94 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760948.045691 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451753.45 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 145175.35 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906123.391076 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306578.11 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 130657.81 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036781.201922 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175920.30 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 117592.03 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154373.231684 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058328.27 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 105832.83 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260206.058469 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952495.44 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 95249.54 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355455.602576 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857245.90 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 85724.59 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441180.192273 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771521.31 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 77152.13 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518332.322999 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694369.18 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 69436.92 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587769.240653 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624932.26 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 62493.23 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650262.466542 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562439.03 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 56243.90 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706506.369841 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506195.13 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 50619.51 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757125.882811 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455575.62 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 45557.56 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802683.444484 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410018.06 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 41001.81 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843685.249989 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369016.25 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 36901.62 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880586.874944 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332114.62 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 33211.46 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913798.337404 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298903.16 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 29890.32 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943688.653617 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269012.85 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 26901.28 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970589.938209 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242111.56 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 24211.16 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994801.094342 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217900.41 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 21790.04 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016591.134862 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196110.36 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 19611.04 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036202.171330 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176499.33 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 17649.93 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053852.104150 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158849.40 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 15884.94 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069737.043689 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142964.46 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 14296.45 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084033.489274 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128668.01 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 12866.80 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096900.290301 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115801.21 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 11580.12 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108480.411224 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104221.09 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 10422.11 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118902.520056 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93798.98 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 9379.90 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128282.418004 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84419.08 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 8441.91 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136724.326158 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75977.17 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 7597.72 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144322.043496 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68379.46 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 6837.95 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151159.989100 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61541.51 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 6154.15 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157314.140144 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55387.36 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 5538.74 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162852.876083 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49848.62 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 4984.86 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167837.738429 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44863.76 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 4486.38 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172324.114540 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40377.38 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 4037.74 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176361.853040 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36339.65 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3633.96 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179995.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32705.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183595.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29105.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187195.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25505.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190795.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21905.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194395.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18305.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197995.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14705.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201595.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11105.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205195.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7505.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208795.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3905.68 sec) 18:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212395.817690 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 305.68 sec) 18:47:00 [GPUGRID] [rr_sim] 2212701.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973099402.69G/439.78G) 18:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853362.41 18:47:00 [---] pick_jobs_to_run() start 18:47:00 [---] pick_jobs_to_run() end 18:47:00 [---] [rr_sim] end 18:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:47:00 [---] [cpu_sched_debug] preliminary job list: 18:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:47:00 [---] [cpu_sched_debug] final job list: 18:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:47:00 [---] [cpu_sched_debug] enforce_run_list: end 18:47:00 [---] choose_project(): 67620.000000 18:47:00 [WUProp@Home] doing RPC to get work 18:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367979.09 cpu 2116172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116172.90 18:47:00 [---] Got 0 tasks 18:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:48:00 [---] [rr_sim] doing sim: CPU sched 18:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973071812.34G/439.78G) 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212638.76 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 221263.88 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221263.876266 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991374.89 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 199137.49 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420401.364905 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792237.40 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 179223.74 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599625.104680 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1613013.66 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 161301.37 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760926.470478 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451712.29 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 145171.23 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906097.699695 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306541.06 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 130654.11 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036751.805992 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175886.96 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 117588.70 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154340.501658 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058298.26 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 105829.83 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260170.327758 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952468.43 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 95246.84 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355417.171248 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857221.59 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 85722.16 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441139.330389 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771499.43 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 77149.94 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518289.273615 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694349.49 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 69434.95 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587724.222519 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624914.54 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 62491.45 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650215.676533 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562423.09 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 56242.31 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706457.985145 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506180.78 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 50618.08 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757076.062897 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455562.70 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 45556.27 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802632.332873 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 410006.43 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 41000.64 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843632.975851 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 369005.79 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 36900.58 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880533.554532 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332105.21 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 33210.52 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913744.075344 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298894.69 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 29889.47 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943633.544075 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269005.22 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 26900.52 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970534.065933 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242104.70 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 24210.47 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994744.535606 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217894.23 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 21789.42 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016533.958311 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196104.80 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 19610.48 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036144.438745 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176494.32 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 17649.43 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053793.871136 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158844.89 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 15884.49 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069678.360288 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142960.40 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 14296.04 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083974.400525 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128664.36 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 12866.44 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096840.836738 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115797.93 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 11579.79 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108420.629330 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104218.13 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 10421.81 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118842.442663 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93796.32 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 9379.63 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128222.074662 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84416.69 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 8441.67 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136663.743462 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75975.02 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 7597.50 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144261.245381 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68377.52 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 6837.75 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151098.997109 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61539.77 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 6153.98 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157252.973663 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55385.79 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 5538.58 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162791.552563 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49847.21 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 4984.72 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167776.273572 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44862.49 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 4486.25 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172262.522480 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40376.24 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 4037.62 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176300.146498 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36338.62 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3633.86 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179934.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32704.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183534.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29104.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187134.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25504.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190734.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21904.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194334.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18304.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197934.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14704.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201534.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11104.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205134.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7504.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208734.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3904.75 sec) 18:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212334.008114 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 304.75 sec) 18:48:00 [GPUGRID] [rr_sim] 2212638.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973071812.34G/439.78G) 18:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853359.68 18:48:00 [---] pick_jobs_to_run() start 18:48:00 [---] pick_jobs_to_run() end 18:48:00 [---] [rr_sim] end 18:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:48:00 [---] [cpu_sched_debug] preliminary job list: 18:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:48:00 [---] [cpu_sched_debug] final job list: 18:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:48:00 [---] [cpu_sched_debug] enforce_run_list: end 18:48:00 [---] choose_project(): 67680.000000 18:48:00 [WUProp@Home] doing RPC to get work 18:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367919.09 cpu 2116112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116112.90 18:48:00 [---] Got 0 tasks 18:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:49:00 [---] [rr_sim] doing sim: CPU sched 18:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973044221.98G/439.78G) 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212576.03 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 221257.60 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221257.602577 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991318.42 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 199131.84 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420389.444897 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792186.58 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 179218.66 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599608.102985 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612967.92 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 161296.79 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760904.895264 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451671.13 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 145167.11 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906072.008315 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306504.02 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 130650.40 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036722.410061 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175853.62 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 117585.36 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154307.771632 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058268.25 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 105826.83 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260134.597046 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952441.43 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 95244.14 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355378.739919 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857197.29 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 85719.73 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441098.468505 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771477.56 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 77147.76 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518246.224232 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694329.80 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 69432.98 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587679.204386 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624896.82 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 62489.68 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650168.886525 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562407.14 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 56240.71 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706409.600450 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506166.43 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 50616.64 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1757026.242982 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455549.78 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 45554.98 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802581.221261 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409994.80 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 40999.48 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843580.701713 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368995.32 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 36899.53 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880480.234119 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332095.79 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 33209.58 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913689.813284 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298886.21 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 29888.62 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943578.434533 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268997.59 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 26899.76 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970478.193657 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242097.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 24209.78 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994687.976869 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217888.05 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 21788.80 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016476.781760 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196099.24 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 19609.92 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036086.706161 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176489.32 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 17648.93 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053735.638122 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158840.39 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 15884.04 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069619.676888 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142956.35 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 14295.63 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083915.311776 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128660.71 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 12866.07 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096781.383176 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115794.64 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 11579.46 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108360.847436 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104215.18 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 10421.52 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118782.365270 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93793.66 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 9379.37 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128161.731320 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84414.29 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 8441.43 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136603.160766 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75972.87 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 7597.29 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144200.447266 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68375.58 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 6837.56 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151038.005117 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61538.02 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 6153.80 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157191.807183 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55384.22 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 5538.42 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162730.229042 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49845.80 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 4984.58 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167714.808715 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44861.22 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 4486.12 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172200.930421 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40375.10 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 4037.51 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176238.439956 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36337.59 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3633.76 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179872.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32703.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183472.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29103.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187072.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25503.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190672.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21903.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194272.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18303.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197872.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14703.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201472.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11103.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205072.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7503.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208672.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3903.83 sec) 18:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212272.198538 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 303.83 sec) 18:49:00 [GPUGRID] [rr_sim] 2212576.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973044221.98G/439.78G) 18:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853356.94 18:49:00 [---] pick_jobs_to_run() start 18:49:00 [---] pick_jobs_to_run() end 18:49:00 [---] [rr_sim] end 18:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:49:00 [---] [cpu_sched_debug] preliminary job list: 18:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:49:00 [---] [cpu_sched_debug] final job list: 18:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:49:00 [---] [cpu_sched_debug] enforce_run_list: end 18:49:00 [---] choose_project(): 67740.000000 18:49:00 [WUProp@Home] doing RPC to get work 18:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367859.09 cpu 2116052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2116052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2116052.90 18:49:00 [---] Got 0 tasks 18:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:50:00 [---] [rr_sim] doing sim: CPU sched 18:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (973016631.63G/439.78G) 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212513.29 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 221251.33 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221251.328889 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991261.96 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 199126.20 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420377.524889 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792135.76 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 179213.58 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599591.101290 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612922.19 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 161292.22 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760883.320050 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451629.97 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 145163.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906046.316934 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306466.97 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 130646.70 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036693.014130 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175820.27 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 117582.03 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154275.041606 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058238.25 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 105823.82 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260098.866335 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952414.42 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 95241.44 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355340.308591 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857172.98 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 85717.30 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441057.606621 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771455.68 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 77145.57 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518203.174848 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694310.11 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 69431.01 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587634.186252 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624879.10 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 62487.91 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650122.096516 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562391.19 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 56239.12 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706361.215754 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506152.07 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 50615.21 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756976.423068 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455536.87 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 45553.69 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802530.109650 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409983.18 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 40998.32 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843528.427574 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368984.86 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 36898.49 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880426.913706 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332086.38 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 33208.64 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913635.551225 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298877.74 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 29887.77 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943523.324991 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268989.96 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 26899.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970422.321381 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242090.97 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 24209.10 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994631.418133 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217881.87 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 21788.19 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016419.605209 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196093.68 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 19609.37 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036028.973577 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176484.32 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 17648.43 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053677.405108 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158835.88 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 15883.59 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069560.993487 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142952.30 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 14295.23 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083856.223027 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128657.07 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 12865.71 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096721.929614 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115791.36 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 11579.14 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108301.065542 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104212.22 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 10421.22 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118722.287877 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93791.00 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 9379.10 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128101.387978 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84411.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 8441.19 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136542.578070 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75970.71 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 7597.07 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144139.649152 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68373.64 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 6837.36 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150977.013126 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61536.28 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 6153.63 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157130.640702 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55382.65 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 5538.26 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162668.905521 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49844.38 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 4984.44 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167653.343859 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44859.95 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 4485.99 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172139.338362 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40373.95 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 4037.40 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176176.733415 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36336.56 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3633.66 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179810.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32702.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183410.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29102.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187010.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25502.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190610.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21902.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194210.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18302.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197810.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14702.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201410.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11102.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205010.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7502.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208610.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3902.90 sec) 18:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212210.388963 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 302.90 sec) 18:50:00 [GPUGRID] [rr_sim] 2212513.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (973016631.63G/439.78G) 18:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853354.20 18:50:00 [---] pick_jobs_to_run() start 18:50:00 [---] pick_jobs_to_run() end 18:50:00 [---] [rr_sim] end 18:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:50:00 [---] [cpu_sched_debug] preliminary job list: 18:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:50:00 [---] [cpu_sched_debug] final job list: 18:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:50:00 [---] [cpu_sched_debug] enforce_run_list: end 18:50:00 [---] choose_project(): 67800.000000 18:50:00 [WUProp@Home] doing RPC to get work 18:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367799.09 cpu 2115992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115992.90 18:50:00 [---] Got 0 tasks 18:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:51:00 [---] [rr_sim] doing sim: CPU sched 18:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972989041.27G/439.78G) 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212450.55 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 221245.06 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221245.055201 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991205.50 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 199120.55 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420365.604882 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792084.95 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 179208.49 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599574.099595 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612876.45 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 161287.65 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760861.744836 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451588.81 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 145158.88 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 906020.625554 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306429.93 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 130642.99 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036663.618199 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175786.93 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 117578.69 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154242.311580 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058208.24 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 105820.82 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260063.135623 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952387.42 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 95238.74 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355301.877262 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857148.67 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 85714.87 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1441016.744737 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771433.81 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 77143.38 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518160.125464 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694290.43 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 69429.04 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587589.168119 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624861.38 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 62486.14 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650075.306508 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562375.25 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 56237.52 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706312.831058 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506137.72 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 50613.77 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756926.603153 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455523.95 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 45552.39 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802478.998039 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409971.55 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 40997.16 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843476.153436 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368974.40 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 36897.44 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880373.593293 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332076.96 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 33207.70 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913581.289165 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298869.26 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 29886.93 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943468.215450 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268982.34 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 26898.23 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970366.449106 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242084.10 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 24208.41 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994574.859396 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217875.69 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 21787.57 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016362.428657 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196088.12 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 19608.81 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035971.240993 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176479.31 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 17647.93 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053619.172094 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158831.38 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 15883.14 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069502.310086 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142948.24 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 14294.82 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083797.134278 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128653.42 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 12865.34 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096662.476051 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115788.08 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 11578.81 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108241.283647 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104209.27 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 10420.93 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118662.210484 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93788.34 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 9378.83 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128041.044636 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84409.51 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 8440.95 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136481.995374 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75968.56 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 7596.86 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144078.851037 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68371.70 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 6837.17 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150916.021134 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61534.53 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 6153.45 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157069.474222 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55381.08 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 5538.11 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162607.582001 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49842.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 4984.30 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167591.879002 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44858.67 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 4485.87 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172077.746303 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40372.81 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 4037.28 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176115.026873 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36335.53 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3633.55 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179748.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32701.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183348.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29101.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186948.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25501.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190548.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21901.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194148.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18301.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197748.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14701.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201348.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11101.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204948.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7501.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208548.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3901.97 sec) 18:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212148.579387 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301.97 sec) 18:51:00 [GPUGRID] [rr_sim] 2212450.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972989041.27G/439.78G) 18:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853351.46 18:51:00 [---] pick_jobs_to_run() start 18:51:00 [---] pick_jobs_to_run() end 18:51:00 [---] [rr_sim] end 18:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:51:00 [---] [cpu_sched_debug] preliminary job list: 18:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:51:00 [---] [cpu_sched_debug] final job list: 18:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:51:00 [---] [cpu_sched_debug] enforce_run_list: end 18:51:00 [---] choose_project(): 67860.000000 18:51:00 [WUProp@Home] doing RPC to get work 18:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367739.09 cpu 2115932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115932.90 18:51:00 [---] Got 0 tasks 18:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:52:00 [---] [rr_sim] doing sim: CPU sched 18:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972961450.91G/439.78G) 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212387.82 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 221238.78 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221238.781513 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991149.03 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 199114.90 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420353.684874 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1792034.13 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 179203.41 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599557.097900 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612830.72 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 161283.07 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760840.169622 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451547.65 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 145154.76 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905994.934173 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306392.88 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 130639.29 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036634.222268 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175753.59 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 117575.36 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154209.581554 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058178.23 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 105817.82 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1260027.404912 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952360.41 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 95236.04 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355263.445933 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857124.37 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 85712.44 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440975.882853 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771411.93 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 77141.19 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518117.076080 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694270.74 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 69427.07 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587544.149985 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624843.67 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 62484.37 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1650028.516499 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562359.30 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 56235.93 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706264.446362 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506123.37 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 50612.34 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756876.783239 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455511.03 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 45551.10 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802427.886428 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409959.93 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 40995.99 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843423.879298 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368963.94 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 36896.39 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880320.272881 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332067.54 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 33206.75 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913527.027105 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298860.79 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 29886.08 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943413.105908 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268974.71 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 26897.47 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970310.576830 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242077.24 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 24207.72 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994518.300659 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217869.51 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 21786.95 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016305.252106 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196082.56 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 19608.26 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035913.508408 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176474.31 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 17647.43 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053560.939080 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158826.88 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 15882.69 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069443.626685 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142944.19 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 14294.42 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083738.045529 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128649.77 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 12864.98 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096603.022489 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115784.79 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 11578.48 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108181.501753 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104206.31 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 10420.63 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118602.133091 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93785.68 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 9378.57 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127980.701294 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84407.11 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 8440.71 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136421.412678 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75966.40 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 7596.64 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144018.052923 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68369.76 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 6836.98 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150855.029143 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61532.79 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 6153.28 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157008.307742 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55379.51 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 5537.95 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162546.258480 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49841.56 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 4984.16 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167530.414145 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44857.40 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 4485.74 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172016.154243 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40371.66 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 4037.17 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176053.320332 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36334.49 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3633.45 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179686.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32701.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183286.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29101.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186886.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25501.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190486.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21901.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194086.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18301.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197686.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14701.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201286.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11101.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204886.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7501.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208486.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3901.05 sec) 18:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212086.769811 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 301.05 sec) 18:52:00 [GPUGRID] [rr_sim] 2212387.82: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972961450.91G/439.78G) 18:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853348.73 18:52:00 [---] pick_jobs_to_run() start 18:52:00 [---] pick_jobs_to_run() end 18:52:00 [---] [rr_sim] end 18:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:52:00 [---] [cpu_sched_debug] preliminary job list: 18:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:52:00 [---] [cpu_sched_debug] final job list: 18:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:52:00 [---] [cpu_sched_debug] enforce_run_list: end 18:52:00 [---] choose_project(): 67920.000000 18:52:00 [WUProp@Home] doing RPC to get work 18:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367679.09 cpu 2115872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115872.90 18:52:00 [---] Got 0 tasks 18:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:53:00 [---] [rr_sim] doing sim: CPU sched 18:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972933860.56G/439.78G) 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212325.08 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 221232.51 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221232.507825 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991092.57 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 199109.26 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420341.764867 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791983.31 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 179198.33 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599540.096205 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612784.98 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 161278.50 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760818.594409 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451506.48 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 145150.65 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905969.242792 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306355.84 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 130635.58 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036604.826338 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175720.25 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 117572.03 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154176.851528 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058148.23 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 105814.82 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259991.674200 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952333.40 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 95233.34 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355225.014605 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857100.06 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 85710.01 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440935.020969 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771390.06 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 77139.01 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518074.026696 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694251.05 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 69425.11 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587499.131851 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624825.95 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 62482.59 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649981.726491 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562343.35 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 56234.34 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706216.061666 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506109.02 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 50610.90 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756826.963324 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455498.11 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 45549.81 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802376.774816 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409948.30 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 40994.83 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843371.605159 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368953.47 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 36895.35 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880266.952468 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332058.13 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 33205.81 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913472.765046 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298852.31 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 29885.23 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943357.996366 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268967.08 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 26896.71 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970254.704554 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242070.37 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 24207.04 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994461.741923 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217863.34 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 21786.33 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016248.075555 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196077.00 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 19607.70 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035855.775824 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176469.30 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 17646.93 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053502.706066 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158822.37 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 15882.24 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069384.943284 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142940.13 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 14294.01 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083678.956780 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128646.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 12864.61 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096543.568927 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115781.51 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 11578.15 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108121.719859 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104203.36 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 10420.34 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118542.055697 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93783.02 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 9378.30 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127920.357952 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84404.72 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 8440.47 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136360.829982 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75964.25 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 7596.42 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143957.254808 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68367.82 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 6836.78 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150794.037152 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61531.04 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 6153.10 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156947.141261 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55377.94 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 5537.79 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162484.934960 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49840.14 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 4984.01 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167468.949288 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44856.13 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 4485.61 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171954.562184 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40370.52 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 4037.05 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175991.613790 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36333.46 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3633.35 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179624.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32700.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183224.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29100.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186824.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25500.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190424.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21900.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194024.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18300.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197624.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14700.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201224.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11100.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204824.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7500.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208424.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3900.12 sec) 18:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2212024.960236 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 300.12 sec) 18:53:00 [GPUGRID] [rr_sim] 2212325.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972933860.56G/439.78G) 18:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853345.99 18:53:00 [---] pick_jobs_to_run() start 18:53:00 [---] pick_jobs_to_run() end 18:53:00 [---] [rr_sim] end 18:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:53:00 [---] [cpu_sched_debug] preliminary job list: 18:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:53:00 [---] [cpu_sched_debug] final job list: 18:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:53:00 [---] [cpu_sched_debug] enforce_run_list: end 18:53:00 [---] choose_project(): 67980.000000 18:53:00 [WUProp@Home] doing RPC to get work 18:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367619.09 cpu 2115812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115812.90 18:53:00 [---] Got 0 tasks 18:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:54:00 [---] [rr_sim] doing sim: CPU sched 18:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972906270.20G/439.78G) 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212262.34 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 221226.23 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221226.234136 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1991036.11 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 199103.61 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420329.844859 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791932.50 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 179193.25 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599523.094509 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612739.25 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 161273.92 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760797.019195 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451465.32 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 145146.53 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905943.551412 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306318.79 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 130631.88 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036575.430407 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175686.91 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 117568.69 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154144.121503 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058118.22 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 105811.82 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259955.943489 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952306.40 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 95230.64 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355186.583276 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857075.76 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 85707.58 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440894.159085 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771368.18 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 77136.82 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1518030.977313 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694231.36 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 69423.14 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587454.113718 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624808.23 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 62480.82 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649934.936482 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562327.40 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 56232.74 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706167.676970 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506094.66 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 50609.47 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756777.143410 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455485.20 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 45548.52 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802325.663205 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409936.68 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 40993.67 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843319.331021 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368943.01 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 36894.30 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880213.632055 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332048.71 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 33204.87 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913418.502986 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298843.84 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 29884.38 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943302.886824 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268959.45 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 26895.95 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970198.832278 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242063.51 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 24206.35 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994405.183186 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217857.16 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 21785.72 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016190.899004 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196071.44 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 19607.14 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035798.043240 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176464.30 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 17646.43 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053444.473052 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158817.87 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 15881.79 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069326.259883 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142936.08 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 14293.61 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083619.868031 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128642.47 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 12864.25 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096484.115365 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115778.23 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 11577.82 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108061.937965 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104200.40 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 10420.04 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118481.978304 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93780.36 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 9378.04 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127860.014610 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84402.33 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 8440.23 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136300.247286 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75962.09 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 7596.21 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143896.456693 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68365.88 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 6836.59 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150733.045160 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61529.30 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 6152.93 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156885.974781 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55376.37 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 5537.64 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162423.611439 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49838.73 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 4983.87 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167407.484431 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44854.86 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 4485.49 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171892.970125 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40369.37 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 4036.94 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175929.907249 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36332.43 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3633.24 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179563.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32699.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183163.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29099.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186763.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25499.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190363.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21899.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193963.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18299.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197563.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14699.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201163.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11099.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204763.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7499.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208363.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3899.19 sec) 18:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211963.150660 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 299.19 sec) 18:54:00 [GPUGRID] [rr_sim] 2212262.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972906270.20G/439.78G) 18:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853343.25 18:54:00 [---] pick_jobs_to_run() start 18:54:00 [---] pick_jobs_to_run() end 18:54:00 [---] [rr_sim] end 18:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:54:00 [---] [cpu_sched_debug] preliminary job list: 18:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:54:00 [---] [cpu_sched_debug] final job list: 18:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:54:00 [---] [cpu_sched_debug] enforce_run_list: end 18:54:00 [---] choose_project(): 68040.000000 18:54:00 [WUProp@Home] doing RPC to get work 18:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367559.09 cpu 2115752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115752.90 18:54:00 [---] Got 0 tasks 18:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:55:00 [---] [rr_sim] doing sim: CPU sched 18:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972878679.84G/439.78G) 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212199.60 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 221219.96 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221219.960448 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990979.64 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 199097.96 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420317.924851 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791881.68 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 179188.17 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599506.092814 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612693.51 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 161269.35 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760775.443981 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451424.16 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 145142.42 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905917.860031 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306281.74 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 130628.17 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036546.034476 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175653.57 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 117565.36 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154111.391477 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058088.21 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 105808.82 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259920.212777 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952279.39 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 95227.94 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355148.151948 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857051.45 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 85705.15 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440853.297201 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771346.31 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 77134.63 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517987.927929 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694211.68 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 69421.17 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587409.095584 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624790.51 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 62479.05 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649888.146474 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562311.46 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 56231.15 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706119.292275 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506080.31 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 50608.03 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756727.323495 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455472.28 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 45547.23 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802274.551594 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409925.05 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 40992.51 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843267.056883 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368932.55 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 36893.25 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880160.311642 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332039.29 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 33203.93 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913364.240926 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298835.36 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 29883.54 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943247.777282 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268951.83 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 26895.18 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970142.960002 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242056.64 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 24205.66 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994348.624450 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217850.98 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 21785.10 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016133.722453 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196065.88 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 19606.59 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035740.310656 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176459.29 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 17645.93 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053386.240038 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158813.36 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 15881.34 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069267.576483 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142932.03 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 14293.20 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083560.779282 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128638.83 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 12863.88 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096424.661802 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115774.94 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 11577.49 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108002.156070 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104197.45 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 10419.74 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118421.900911 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93777.70 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 9377.77 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127799.671268 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84399.93 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 8439.99 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136239.664590 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75959.94 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 7595.99 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143835.658579 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68363.95 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 6836.39 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150672.053169 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61527.55 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 6152.76 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156824.808300 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55374.80 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 5537.48 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162362.287918 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49837.32 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 4983.73 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167346.019575 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44853.58 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 4485.36 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171831.378065 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40368.23 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 4036.82 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175868.200707 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36331.40 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3633.14 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179501.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32698.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183101.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29098.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186701.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25498.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190301.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21898.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193901.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18298.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197501.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14698.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201101.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11098.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204701.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7498.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208301.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3898.26 sec) 18:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211901.341084 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298.26 sec) 18:55:00 [GPUGRID] [rr_sim] 2212199.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972878679.84G/439.78G) 18:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853340.52 18:55:00 [---] pick_jobs_to_run() start 18:55:00 [---] pick_jobs_to_run() end 18:55:00 [---] [rr_sim] end 18:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:55:00 [---] [cpu_sched_debug] preliminary job list: 18:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:55:00 [---] [cpu_sched_debug] final job list: 18:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:55:00 [---] [cpu_sched_debug] enforce_run_list: end 18:55:00 [---] choose_project(): 68100.000000 18:55:00 [WUProp@Home] doing RPC to get work 18:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367499.09 cpu 2115692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115692.90 18:55:00 [---] Got 0 tasks 18:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:56:00 [---] [rr_sim] doing sim: CPU sched 18:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972851089.49G/439.78G) 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212136.87 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 221213.69 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221213.686760 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990923.18 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 199092.32 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420306.004844 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791830.86 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 179183.09 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599489.091119 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612647.78 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 161264.78 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760753.868767 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451383.00 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 145138.30 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905892.168651 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306244.70 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 130624.47 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036516.638545 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175620.23 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 117562.02 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154078.661451 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058058.21 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 105805.82 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259884.482066 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952252.39 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 95225.24 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355109.720619 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857027.15 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 85702.71 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440812.435317 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771324.43 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 77132.44 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517944.878545 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694191.99 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 69419.20 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587364.077451 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624772.79 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 62477.28 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649841.356465 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562295.51 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 56229.55 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706070.907579 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506065.96 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 50606.60 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756677.503581 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455459.36 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 45545.94 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802223.439983 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409913.43 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 40991.34 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843214.782744 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368922.08 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 36892.21 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880106.991230 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332029.88 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 33202.99 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913309.978867 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298826.89 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 29882.69 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943192.667740 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268944.20 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 26894.42 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970087.087726 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242049.78 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 24204.98 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994292.065713 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217844.80 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 21784.48 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016076.545902 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196060.32 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 19606.03 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035682.578072 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176454.29 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 17645.43 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053328.007024 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158808.86 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 15880.89 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069208.893082 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142927.97 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 14292.80 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083501.690533 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128635.18 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 12863.52 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096365.208240 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115771.66 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 11577.17 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107942.374176 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104194.49 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 10419.45 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118361.823518 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93775.04 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 9377.50 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127739.327926 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84397.54 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 8439.75 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136179.081894 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75957.79 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 7595.78 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143774.860464 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68362.01 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 6836.20 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150611.061178 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61525.81 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 6152.58 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156763.641820 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55373.23 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 5537.32 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162300.964398 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49835.90 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 4983.59 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167284.554718 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44852.31 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 4485.23 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171769.786006 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40367.08 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 4036.71 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175806.494165 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36330.37 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3633.04 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179439.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32697.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183039.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29097.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186639.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25497.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190239.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21897.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193839.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18297.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197439.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14697.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201039.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11097.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204639.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7497.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208239.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3897.34 sec) 18:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211839.531509 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297.34 sec) 18:56:00 [GPUGRID] [rr_sim] 2212136.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972851089.49G/439.78G) 18:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853337.78 18:56:00 [---] pick_jobs_to_run() start 18:56:00 [---] pick_jobs_to_run() end 18:56:00 [---] [rr_sim] end 18:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:56:00 [---] [cpu_sched_debug] preliminary job list: 18:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:56:00 [---] [cpu_sched_debug] final job list: 18:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:56:00 [---] [cpu_sched_debug] enforce_run_list: end 18:56:00 [---] choose_project(): 68160.000000 18:56:00 [WUProp@Home] doing RPC to get work 18:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367439.09 cpu 2115632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115632.90 18:56:00 [---] Got 0 tasks 18:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:57:00 [---] [rr_sim] doing sim: CPU sched 18:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972851089.49G/439.78G) 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212136.87 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 221213.69 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221213.686760 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990923.18 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 199092.32 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420306.004844 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791830.86 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 179183.09 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599489.091119 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612647.78 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 161264.78 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760753.868767 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451383.00 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 145138.30 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905892.168651 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306244.70 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 130624.47 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036516.638545 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175620.23 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 117562.02 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154078.661451 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058058.21 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 105805.82 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259884.482066 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952252.39 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 95225.24 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355109.720619 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857027.15 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 85702.71 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440812.435317 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771324.43 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 77132.44 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517944.878545 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694191.99 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 69419.20 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587364.077451 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624772.79 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 62477.28 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649841.356465 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562295.51 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 56229.55 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706070.907579 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506065.96 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 50606.60 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756677.503581 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455459.36 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 45545.94 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802223.439983 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409913.43 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 40991.34 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843214.782744 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368922.08 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 36892.21 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880106.991230 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332029.88 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 33202.99 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913309.978867 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298826.89 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 29882.69 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943192.667740 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268944.20 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 26894.42 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970087.087726 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242049.78 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 24204.98 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994292.065713 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217844.80 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 21784.48 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016076.545902 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196060.32 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 19606.03 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035682.578072 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176454.29 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 17645.43 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053328.007024 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158808.86 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 15880.89 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069208.893082 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142927.97 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 14292.80 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083501.690533 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128635.18 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 12863.52 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096365.208240 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115771.66 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 11577.17 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107942.374176 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104194.49 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 10419.45 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118361.823518 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93775.04 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 9377.50 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127739.327926 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84397.54 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 8439.75 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136179.081894 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75957.79 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 7595.78 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143774.860464 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68362.01 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 6836.20 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150611.061178 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61525.81 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 6152.58 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156763.641820 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55373.23 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 5537.32 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162300.964398 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49835.90 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 4983.59 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167284.554718 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44852.31 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 4485.23 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171769.786006 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40367.08 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 4036.71 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175806.494165 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36330.37 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3633.04 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179439.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32697.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183039.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29097.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186639.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25497.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190239.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21897.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193839.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18297.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197439.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14697.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201039.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11097.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204639.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7497.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208239.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3897.34 sec) 18:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211839.531509 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297.34 sec) 18:57:00 [GPUGRID] [rr_sim] 2212136.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972851089.49G/439.78G) 18:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853397.78 18:57:00 [---] pick_jobs_to_run() start 18:57:00 [---] pick_jobs_to_run() end 18:57:00 [---] [rr_sim] end 18:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:57:00 [---] [cpu_sched_debug] preliminary job list: 18:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:57:00 [---] [cpu_sched_debug] final job list: 18:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:57:00 [---] [cpu_sched_debug] enforce_run_list: end 18:57:00 [---] choose_project(): 68220.000000 18:57:00 [WUProp@Home] doing RPC to get work 18:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367379.09 cpu 2115632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115632.90 18:57:00 [---] Got 0 tasks 18:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:58:00 [---] [rr_sim] doing sim: CPU sched 18:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972823499.13G/439.78G) 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212074.13 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 221207.41 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221207.413072 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990866.72 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 199086.67 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420294.084836 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791780.05 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 179178.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599472.089424 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612602.04 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 161260.20 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760732.293554 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451341.84 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 145134.18 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905866.477270 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306207.65 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 130620.77 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036487.242615 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175586.89 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 117558.69 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154045.931425 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058028.20 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 105802.82 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259848.751354 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952225.38 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 95222.54 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355071.289290 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857002.84 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 85700.28 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440771.573433 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771302.56 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 77130.26 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517901.829161 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694172.30 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 69417.23 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587319.059317 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624755.07 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 62475.51 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649794.566457 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562279.56 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 56227.96 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706022.522883 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506051.61 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 50605.16 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756627.683666 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455446.45 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 45544.64 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802172.328371 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409901.80 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 40990.18 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843162.508606 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368911.62 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 36891.16 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880053.670817 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332020.46 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 33202.05 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913255.716807 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298818.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 29881.84 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943137.558198 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268936.57 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 26893.66 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970031.215450 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242042.92 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 24204.29 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994235.506977 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217838.62 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 21783.86 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016019.369351 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196054.76 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 19605.48 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035624.845487 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176449.29 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 17644.93 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053269.774010 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158804.36 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 15880.44 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069150.209681 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142923.92 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 14292.39 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083442.601785 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128631.53 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 12863.15 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096305.754678 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115768.38 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 11576.84 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107882.592282 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104191.54 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 10419.15 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118301.746125 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93772.38 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 9377.24 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127678.984584 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84395.15 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 8439.51 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136118.499198 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75955.63 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 7595.56 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143714.062350 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68360.07 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 6836.01 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150550.069186 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61524.06 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 6152.41 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156702.475339 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55371.66 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 5537.17 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162239.640877 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49834.49 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 4983.45 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167223.089861 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44851.04 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 4485.10 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171708.193947 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40365.94 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 4036.59 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175744.787624 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36329.34 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3632.93 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179377.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32696.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182977.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29096.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186577.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25496.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190177.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21896.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193777.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18296.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197377.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14696.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200977.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11096.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204577.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7496.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208177.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3896.41 sec) 18:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211777.721933 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296.41 sec) 18:58:00 [GPUGRID] [rr_sim] 2212074.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972823499.13G/439.78G) 18:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853395.04 18:58:00 [---] pick_jobs_to_run() start 18:58:00 [---] pick_jobs_to_run() end 18:58:00 [---] [rr_sim] end 18:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:58:00 [---] [cpu_sched_debug] preliminary job list: 18:58:00 [---] [cpu_sched_debug] final job list: 18:58:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 18:58:00 [---] [work_fetch] Request work fetch: CPUs idle 18:58:00 [---] [cpu_sched_debug] enforce_run_list: end 18:58:00 [---] choose_project(): 68280.000000 18:58:00 [WUProp@Home] doing RPC to get work 18:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367319.09 cpu 2115572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115572.90 18:58:00 [---] Got 0 tasks 18:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 18:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 18:59:00 [---] [rr_sim] doing sim: CPU sched 18:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972823499.13G/439.78G) 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212074.13 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 221207.41 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221207.413072 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990866.72 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 199086.67 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420294.084836 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791780.05 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 179178.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599472.089424 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612602.04 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 161260.20 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760732.293554 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451341.84 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 145134.18 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905866.477270 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306207.65 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 130620.77 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036487.242615 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175586.89 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 117558.69 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154045.931425 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1058028.20 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 105802.82 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259848.751354 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952225.38 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 95222.54 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355071.289290 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 857002.84 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 85700.28 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440771.573433 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771302.56 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 77130.26 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517901.829161 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694172.30 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 69417.23 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587319.059317 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624755.07 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 62475.51 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649794.566457 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562279.56 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 56227.96 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1706022.522883 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506051.61 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 50605.16 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756627.683666 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455446.45 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 45544.64 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802172.328371 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409901.80 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 40990.18 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843162.508606 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368911.62 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 36891.16 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880053.670817 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332020.46 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 33202.05 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913255.716807 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298818.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 29881.84 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943137.558198 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268936.57 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 26893.66 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1970031.215450 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242042.92 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 24204.29 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994235.506977 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217838.62 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 21783.86 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016019.369351 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196054.76 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 19605.48 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035624.845487 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176449.29 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 17644.93 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053269.774010 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158804.36 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 15880.44 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069150.209681 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142923.92 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 14292.39 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083442.601785 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128631.53 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 12863.15 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096305.754678 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115768.38 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 11576.84 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107882.592282 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104191.54 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 10419.15 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118301.746125 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93772.38 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 9377.24 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127678.984584 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84395.15 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 8439.51 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136118.499198 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75955.63 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 7595.56 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143714.062350 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68360.07 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 6836.01 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150550.069186 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61524.06 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 6152.41 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156702.475339 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55371.66 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 5537.17 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162239.640877 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49834.49 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 4983.45 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167223.089861 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44851.04 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 4485.10 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171708.193947 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40365.94 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 4036.59 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175744.787624 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36329.34 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3632.93 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179377.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32696.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182977.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29096.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186577.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25496.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190177.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21896.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193777.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18296.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197377.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14696.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200977.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11096.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204577.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7496.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208177.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3896.41 sec) 18:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211777.721933 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296.41 sec) 18:59:00 [GPUGRID] [rr_sim] 2212074.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972823499.13G/439.78G) 18:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853455.04 18:59:00 [---] pick_jobs_to_run() start 18:59:00 [---] pick_jobs_to_run() end 18:59:00 [---] [rr_sim] end 18:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 18:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 18:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 18:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 18:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 18:59:00 [---] [cpu_sched_debug] preliminary job list: 18:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 18:59:00 [---] [cpu_sched_debug] final job list: 18:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 18:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 18:59:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 972823499.13G 18:59:00 [---] [cpu_sched_debug] enforce_run_list: end 18:59:00 [---] choose_project(): 68340.000000 18:59:00 [WUProp@Home] doing RPC to get work 18:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367259.09 cpu 2115572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115572.90 18:59:00 [---] Got 0 tasks 19:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:00:00 [---] [rr_sim] doing sim: CPU sched 19:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972795908.78G/439.78G) 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2212011.39 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 221201.14 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221201.139383 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990810.25 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 199081.03 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420282.164829 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791729.23 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 179172.92 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599455.087729 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612556.31 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 161255.63 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760710.718340 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451300.68 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 145130.07 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905840.785889 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306170.61 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 130617.06 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036457.846684 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175553.55 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 117555.35 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1154013.201399 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057998.19 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 105799.82 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259813.020643 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952198.37 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 95219.84 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1355032.857962 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856978.54 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 85697.85 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440730.711549 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771280.68 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 77128.07 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517858.779778 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694152.61 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 69415.26 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587274.041183 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624737.35 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 62473.74 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649747.776448 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562263.62 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 56226.36 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705974.138187 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506037.26 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 50603.73 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756577.863752 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455433.53 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 45543.35 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802121.216760 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409890.18 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 40989.02 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843110.234468 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368901.16 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 36890.12 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1880000.350404 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332011.04 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 33201.10 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913201.454747 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298809.94 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 29880.99 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943082.448656 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268928.95 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 26892.89 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969975.343174 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242036.05 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 24203.61 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994178.948240 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217832.45 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 21783.24 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015962.192800 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196049.20 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 19604.92 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035567.112903 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176444.28 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 17644.43 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053211.540996 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158799.85 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 15879.99 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069091.526280 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142919.87 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 14291.99 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083383.513036 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128627.88 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 12862.79 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096246.301115 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115765.09 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 11576.51 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107822.810387 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104188.58 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 10418.86 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118241.668732 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93769.73 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 9376.97 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127618.641242 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84392.75 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 8439.28 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136057.916502 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75953.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 7595.35 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143653.264235 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68358.13 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 6835.81 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150489.077195 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61522.32 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 6152.23 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156641.308859 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55370.08 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 5537.01 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162178.317357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49833.08 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 4983.31 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167161.625004 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44849.77 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 4484.98 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171646.601887 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40364.79 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 4036.48 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175683.081082 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36328.31 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3632.83 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179315.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32695.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182915.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29095.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186515.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25495.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190115.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21895.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193715.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18295.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197315.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14695.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200915.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11095.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204515.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7495.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208115.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3895.48 sec) 19:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211715.912357 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 295.48 sec) 19:00:00 [GPUGRID] [rr_sim] 2212011.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972795908.78G/439.78G) 19:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853452.31 19:00:00 [---] pick_jobs_to_run() start 19:00:00 [---] pick_jobs_to_run() end 19:00:00 [---] [rr_sim] end 19:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:00:00 [---] [cpu_sched_debug] preliminary job list: 19:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 19:00:00 [---] [cpu_sched_debug] final job list: 19:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:00:00 [---] [cpu_sched_debug] enforce_run_list: end 19:00:00 [---] choose_project(): 68400.000000 19:00:00 [WUProp@Home] doing RPC to get work 19:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367199.09 cpu 2115512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115512.90 19:00:00 [---] Got 0 tasks 19:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:01:00 [---] [rr_sim] doing sim: CPU sched 19:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972768318.42G/439.78G) 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211948.66 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 221194.87 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221194.865695 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990753.79 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 199075.38 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420270.244821 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791678.41 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 179167.84 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599438.086034 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612510.57 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 161251.06 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760689.143126 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451259.51 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 145125.95 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905815.094509 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306133.56 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 130613.36 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036428.450753 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175520.21 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 117552.02 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153980.471373 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057968.19 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 105796.82 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259777.289931 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952171.37 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 95217.14 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354994.426633 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856954.23 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 85695.42 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440689.849665 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771258.81 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 77125.88 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517815.730394 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694132.93 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 69413.29 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587229.023050 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624719.63 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 62471.96 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649700.986440 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562247.67 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 56224.77 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705925.753491 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506022.90 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 50602.29 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756528.043837 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455420.61 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 45542.06 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802070.105149 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409878.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 40987.86 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843057.960329 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368890.70 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 36889.07 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879947.029992 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 332001.63 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 33200.16 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913147.192688 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298801.46 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 29880.15 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1943027.339114 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268921.32 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 26892.13 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969919.470898 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242029.19 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 24202.92 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994122.389504 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217826.27 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 21782.63 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015905.016248 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196043.64 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 19604.36 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035509.380319 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176439.28 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 17643.93 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053153.307982 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158795.35 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 15879.53 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069032.842879 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142915.81 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 14291.58 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083324.424287 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128624.23 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 12862.42 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096186.847553 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115761.81 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 11576.18 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107763.028493 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104185.63 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 10418.56 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118181.591339 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93767.07 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 9376.71 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127558.297900 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84390.36 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 8439.04 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135997.333806 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75951.32 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 7595.13 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143592.466120 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68356.19 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 6835.62 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150428.085204 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61520.57 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 6152.06 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156580.142378 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55368.51 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 5536.85 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162116.993836 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49831.66 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 4983.17 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167100.160148 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44848.50 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 4484.85 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171585.009828 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40363.65 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 4036.36 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175621.374541 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36327.28 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3632.73 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179254.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32694.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182854.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29094.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186454.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25494.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190054.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21894.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193654.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18294.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197254.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14694.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200854.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11094.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204454.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7494.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208054.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3894.55 sec) 19:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211654.102782 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 294.55 sec) 19:01:00 [GPUGRID] [rr_sim] 2211948.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972768318.42G/439.78G) 19:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853449.57 19:01:00 [---] pick_jobs_to_run() start 19:01:00 [---] pick_jobs_to_run() end 19:01:00 [---] [rr_sim] end 19:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:01:00 [---] [cpu_sched_debug] preliminary job list: 19:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:01:00 [---] [cpu_sched_debug] final job list: 19:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:01:00 [---] [cpu_sched_debug] enforce_run_list: end 19:01:00 [---] choose_project(): 68460.000000 19:01:00 [WUProp@Home] doing RPC to get work 19:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367139.09 cpu 2115452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115452.90 19:01:00 [---] Got 0 tasks 19:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:02:00 [---] [rr_sim] doing sim: CPU sched 19:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972740728.06G/439.78G) 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211885.92 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 221188.59 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221188.592007 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990697.33 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 199069.73 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420258.324813 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791627.60 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 179162.76 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599421.084339 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612464.84 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 161246.48 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760667.567912 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451218.35 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 145121.84 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905789.403128 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306096.52 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 130609.65 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036399.054822 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175486.87 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 117548.69 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153947.741347 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057938.18 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 105793.82 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259741.559219 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952144.36 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 95214.44 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354955.995305 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856929.92 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 85692.99 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440648.987781 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771236.93 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 77123.69 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517772.681010 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694113.24 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 69411.32 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587184.004916 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624701.92 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 62470.19 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649654.196432 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562231.72 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 56223.17 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705877.368795 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 506008.55 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 50600.86 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756478.223923 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455407.70 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 45540.77 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1802018.993538 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409866.93 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 40986.69 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1843005.686191 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368880.23 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 36888.02 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879893.709579 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331992.21 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 33199.22 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913092.930628 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298792.99 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 29879.30 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942972.229572 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268913.69 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 26891.37 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969863.598622 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242022.32 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 24202.23 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994065.830767 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217820.09 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 21782.01 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015847.839697 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196038.08 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 19603.81 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035451.647735 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176434.27 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 17643.43 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053095.074968 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158790.85 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 15879.08 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068974.159478 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142911.76 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 14291.18 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083265.335538 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128620.58 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 12862.06 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096127.393991 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115758.53 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 11575.85 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107703.246599 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104182.67 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 10418.27 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118121.513946 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93764.41 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 9376.44 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127497.954558 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84387.97 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 8438.80 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135936.751110 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75949.17 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 7594.92 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143531.668006 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68354.25 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 6835.43 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150367.093212 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61518.83 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 6151.88 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156518.975898 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55366.94 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 5536.69 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162055.670315 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49830.25 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 4983.02 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167038.695291 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44847.22 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 4484.72 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171523.417769 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40362.50 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 4036.25 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175559.667999 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36326.25 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3632.63 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179192.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32693.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182792.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29093.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186392.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25493.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189992.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21893.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193592.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18293.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197192.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14693.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200792.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11093.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204392.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7493.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207992.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3893.63 sec) 19:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211592.293206 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 293.63 sec) 19:02:00 [GPUGRID] [rr_sim] 2211885.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972740728.06G/439.78G) 19:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853446.83 19:02:00 [---] pick_jobs_to_run() start 19:02:00 [---] pick_jobs_to_run() end 19:02:00 [---] [rr_sim] end 19:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:02:00 [---] [cpu_sched_debug] preliminary job list: 19:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:02:00 [---] [cpu_sched_debug] final job list: 19:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:02:00 [---] [cpu_sched_debug] enforce_run_list: end 19:02:00 [---] choose_project(): 68520.000000 19:02:00 [WUProp@Home] doing RPC to get work 19:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367079.09 cpu 2115392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115392.90 19:02:00 [---] Got 0 tasks 19:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:03:00 [---] [rr_sim] doing sim: CPU sched 19:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972713137.71G/439.78G) 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211823.18 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 221182.32 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221182.318319 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990640.86 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 199064.09 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420246.404806 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791576.78 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 179157.68 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599404.082644 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612419.10 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 161241.91 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760645.992698 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451177.19 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 145117.72 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905763.711747 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306059.47 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 130605.95 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036369.658892 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175453.52 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 117545.35 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153915.011321 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057908.17 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 105790.82 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259705.828508 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952117.35 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 95211.74 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354917.563976 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856905.62 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 85690.56 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440608.125897 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771215.06 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 77121.51 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517729.631626 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694093.55 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 69409.36 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587138.986782 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624684.20 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 62468.42 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649607.406423 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562215.78 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 56221.58 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705828.984100 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505994.20 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 50599.42 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756428.404008 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455394.78 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 45539.48 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801967.881926 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409855.30 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 40985.53 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842953.412053 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368869.77 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 36886.98 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879840.389166 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331982.79 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 33198.28 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1913038.668568 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298784.51 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 29878.45 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942917.120030 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268906.06 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 26890.61 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969807.726346 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242015.46 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 24201.55 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1994009.272030 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217813.91 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 21781.39 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015790.663146 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196032.52 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 19603.25 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035393.915150 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176429.27 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 17642.93 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053036.841954 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158786.34 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 15878.63 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068915.476078 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142907.71 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 14290.77 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083206.246789 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128616.94 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 12861.69 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096067.940429 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115755.24 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 11575.52 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107643.464705 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104179.72 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 10417.97 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118061.436553 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93761.75 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 9376.17 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127437.611216 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84385.57 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 8438.56 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135876.168414 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75947.01 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 7594.70 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143470.869891 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68352.31 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 6835.23 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150306.101221 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61517.08 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 6151.71 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156457.809418 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55365.37 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 5536.54 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161994.346795 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49828.84 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 4982.88 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166977.230434 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44845.95 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 4484.60 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171461.825709 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40361.36 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 4036.14 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175497.961457 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36325.22 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3632.52 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179130.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32692.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182730.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29092.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186330.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25492.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189930.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21892.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193530.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18292.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197130.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14692.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200730.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11092.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204330.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7492.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207930.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3892.70 sec) 19:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211530.483630 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 292.70 sec) 19:03:00 [GPUGRID] [rr_sim] 2211823.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972713137.71G/439.78G) 19:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853444.10 19:03:00 [---] pick_jobs_to_run() start 19:03:00 [---] pick_jobs_to_run() end 19:03:00 [---] [rr_sim] end 19:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:03:00 [---] [cpu_sched_debug] preliminary job list: 19:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:03:00 [---] [cpu_sched_debug] final job list: 19:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:03:00 [---] [cpu_sched_debug] enforce_run_list: end 19:03:00 [---] choose_project(): 68580.000000 19:03:00 [WUProp@Home] doing RPC to get work 19:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 367019.09 cpu 2115332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115332.90 19:03:00 [---] Got 0 tasks 19:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:04:00 [---] [rr_sim] doing sim: CPU sched 19:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972685547.35G/439.78G) 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211760.45 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 221176.04 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221176.044631 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990584.40 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 199058.44 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420234.484798 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791525.96 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 179152.60 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599387.080949 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612373.37 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 161237.34 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760624.417485 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451136.03 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 145113.60 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905738.020367 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1306022.43 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 130602.24 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036340.262961 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175420.18 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 117542.02 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153882.281295 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057878.17 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 105787.82 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259670.097796 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952090.35 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 95209.03 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354879.132647 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856881.31 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 85688.13 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440567.264013 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771193.18 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 77119.32 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517686.582243 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694073.86 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 69407.39 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587093.968649 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624666.48 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 62466.65 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649560.616415 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562199.83 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 56219.98 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705780.599404 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505979.85 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 50597.98 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756378.584094 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455381.86 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 45538.19 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801916.770315 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409843.68 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 40984.37 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842901.137914 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368859.31 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 36885.93 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879787.068753 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331973.38 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 33197.34 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912984.406509 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298776.04 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 29877.60 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942862.010488 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268898.44 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 26889.84 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969751.854070 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242008.59 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 24200.86 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993952.713294 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217807.73 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 21780.77 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015733.486595 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196026.96 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 19602.70 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035336.182566 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176424.26 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 17642.43 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052978.608940 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158781.84 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 15878.18 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068856.792677 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142903.65 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 14290.37 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083147.158040 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128613.29 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 12861.33 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096008.486866 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115751.96 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 11575.20 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107583.682810 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104176.76 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 10417.68 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118001.359160 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93759.09 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 9375.91 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127377.267875 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84383.18 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 8438.32 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135815.585718 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75944.86 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 7594.49 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143410.071777 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68350.37 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 6835.04 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150245.109229 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61515.34 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 6151.53 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156396.642937 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55363.80 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 5536.38 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161933.023274 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49827.42 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 4982.74 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166915.765577 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44844.68 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 4484.47 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171400.233650 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40360.21 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 4036.02 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175436.254916 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36324.19 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3632.42 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179068.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32691.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182668.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29091.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186268.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25491.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189868.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21891.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193468.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18291.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197068.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14691.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200668.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11091.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204268.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7491.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207868.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3891.77 sec) 19:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211468.674055 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 291.77 sec) 19:04:00 [GPUGRID] [rr_sim] 2211760.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972685547.35G/439.78G) 19:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853441.36 19:04:00 [---] pick_jobs_to_run() start 19:04:00 [---] pick_jobs_to_run() end 19:04:00 [---] [rr_sim] end 19:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:04:00 [---] [cpu_sched_debug] preliminary job list: 19:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:04:00 [---] [cpu_sched_debug] final job list: 19:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:04:00 [---] [cpu_sched_debug] enforce_run_list: end 19:04:00 [---] choose_project(): 68640.000000 19:04:00 [WUProp@Home] doing RPC to get work 19:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366959.09 cpu 2115272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115272.90 19:04:00 [---] Got 0 tasks 19:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:05:00 [---] [rr_sim] doing sim: CPU sched 19:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972657956.99G/439.78G) 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211697.71 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 221169.77 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221169.770942 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990527.94 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 199052.79 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420222.564791 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791475.14 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 179147.51 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599370.079254 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612327.63 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 161232.76 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760602.842271 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451094.87 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 145109.49 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905712.328986 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305985.38 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 130598.54 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036310.867030 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175386.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 117538.68 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153849.551269 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057848.16 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 105784.82 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259634.367085 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952063.34 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 95206.33 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354840.701319 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856857.01 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 85685.70 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440526.402129 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771171.31 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 77117.13 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517643.532859 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694054.18 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 69405.42 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587048.950515 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624648.76 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 62464.88 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649513.826406 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562183.88 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 56218.39 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705732.214708 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505965.49 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 50596.55 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756328.764180 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455368.95 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 45536.89 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801865.658704 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409832.05 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 40983.21 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842848.863776 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368848.85 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 36884.88 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879733.748341 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331963.96 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 33196.40 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912930.144449 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298767.56 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 29876.76 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942806.900947 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268890.81 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 26889.08 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969695.981794 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242001.73 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 24200.17 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993896.154557 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217801.55 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 21780.16 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015676.310044 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196021.40 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 19602.14 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035278.449982 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176419.26 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 17641.93 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052920.375926 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158777.33 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 15877.73 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068798.109276 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142899.60 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 14289.96 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083088.069291 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128609.64 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 12860.96 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095949.033304 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115748.68 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 11574.87 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107523.900916 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104173.81 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 10417.38 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117941.281767 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93756.43 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 9375.64 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127316.924533 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84380.78 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 8438.08 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135755.003022 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75942.71 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 7594.27 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143349.273662 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68348.44 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 6834.84 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150184.117238 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61513.59 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 6151.36 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156335.476457 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55362.23 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 5536.22 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161871.699753 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49826.01 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 4982.60 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166854.300720 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44843.41 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 4484.34 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171338.641591 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40359.07 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 4035.91 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175374.548374 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36323.16 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3632.32 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179006.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32690.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182606.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29090.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186206.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25490.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189806.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21890.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193406.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18290.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197006.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14690.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200606.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11090.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204206.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7490.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207806.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3890.84 sec) 19:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211406.864479 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 290.84 sec) 19:05:00 [GPUGRID] [rr_sim] 2211697.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972657956.99G/439.78G) 19:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853438.62 19:05:00 [---] pick_jobs_to_run() start 19:05:00 [---] pick_jobs_to_run() end 19:05:00 [---] [rr_sim] end 19:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:05:00 [---] [cpu_sched_debug] preliminary job list: 19:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:05:00 [---] [cpu_sched_debug] final job list: 19:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:05:00 [---] [cpu_sched_debug] enforce_run_list: end 19:05:00 [---] choose_project(): 68700.000000 19:05:00 [WUProp@Home] doing RPC to get work 19:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366899.09 cpu 2115212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115212.90 19:05:00 [---] Got 0 tasks 19:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:06:00 [---] [rr_sim] doing sim: CPU sched 19:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972630366.64G/439.78G) 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211634.97 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 221163.50 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221163.497254 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990471.48 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 199047.15 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420210.644783 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791424.33 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 179142.43 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599353.077559 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612281.89 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 161228.19 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760581.267057 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451053.71 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 145105.37 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905686.637606 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305948.33 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 130594.83 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036281.471099 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175353.50 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 117535.35 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153816.821243 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057818.15 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 105781.82 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259598.636373 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952036.34 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 95203.63 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354802.269990 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856832.70 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 85683.27 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440485.540245 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771149.43 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 77114.94 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517600.483475 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694034.49 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 69403.45 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1587003.932382 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624631.04 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 62463.10 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649467.036398 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562167.94 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 56216.79 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705683.830012 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505951.14 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 50595.11 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756278.944265 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455356.03 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 45535.60 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801814.547093 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409820.43 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 40982.04 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842796.589638 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368838.38 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 36883.84 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879680.427928 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331954.54 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 33195.45 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912875.882389 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298759.09 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 29875.91 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942751.791405 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268883.18 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 26888.32 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969640.109518 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241994.86 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 24199.49 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993839.595821 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217795.38 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 21779.54 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015619.133493 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196015.84 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 19601.58 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035220.717398 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176414.26 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 17641.43 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052862.142912 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158772.83 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 15877.28 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068739.425875 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142895.55 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 14289.55 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083028.980542 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128605.99 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 12860.60 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095889.579742 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115745.39 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 11574.54 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107464.119022 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104170.85 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 10417.09 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117881.204374 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93753.77 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 9375.38 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127256.581191 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84378.39 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 8437.84 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135694.420326 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75940.55 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 7594.06 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143288.475547 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68346.50 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 6834.65 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150123.125247 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61511.85 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 6151.18 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156274.309976 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55360.66 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 5536.07 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161810.376233 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49824.60 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 4982.46 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166792.835864 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44842.14 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 4484.21 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171277.049532 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40357.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 4035.79 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175312.841833 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36322.13 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3632.21 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178945.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32689.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182545.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29089.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186145.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25489.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189745.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21889.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193345.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18289.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196945.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14689.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200545.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11089.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204145.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7489.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207745.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3889.92 sec) 19:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211345.054903 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 289.92 sec) 19:06:00 [GPUGRID] [rr_sim] 2211634.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972630366.64G/439.78G) 19:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853435.89 19:06:00 [---] pick_jobs_to_run() start 19:06:00 [---] pick_jobs_to_run() end 19:06:00 [---] [rr_sim] end 19:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:06:00 [---] [cpu_sched_debug] preliminary job list: 19:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:06:00 [---] [cpu_sched_debug] final job list: 19:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:06:00 [---] [cpu_sched_debug] enforce_run_list: end 19:06:00 [---] choose_project(): 68760.000000 19:06:00 [WUProp@Home] doing RPC to get work 19:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366839.09 cpu 2115152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115152.90 19:06:00 [---] Got 0 tasks 19:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:07:00 [---] [rr_sim] doing sim: CPU sched 19:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972602776.28G/439.78G) 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211572.24 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 221157.22 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221157.223566 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990415.01 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 199041.50 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420198.724775 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791373.51 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 179137.35 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599336.075864 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612236.16 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 161223.62 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760559.691843 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1451012.54 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 145101.25 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905660.946225 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305911.29 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 130591.13 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036252.075168 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175320.16 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 117532.02 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153784.091218 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057788.14 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 105778.81 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259562.905662 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 952009.33 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 95200.93 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354763.838662 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856808.40 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 85680.84 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440444.678361 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771127.56 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 77112.76 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517557.434091 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 694014.80 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 69401.48 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586958.914248 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624613.32 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 62461.33 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649420.246389 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562151.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 56215.20 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705635.445316 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505936.79 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 50593.68 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756229.124351 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455343.11 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 45534.31 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801763.435481 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409808.80 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 40980.88 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842744.315499 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368827.92 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 36882.79 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879627.107515 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331945.13 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 33194.51 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912821.620330 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298750.62 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 29875.06 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942696.681863 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268875.55 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 26887.56 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969584.237242 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241988.00 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 24198.80 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993783.037084 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217789.20 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 21778.92 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015561.956942 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196010.28 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 19601.03 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035162.984813 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176409.25 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 17640.93 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052803.909898 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158768.33 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 15876.83 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068680.742474 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142891.49 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 14289.15 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082969.891793 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128602.34 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 12860.23 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095830.126179 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115742.11 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 11574.21 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107404.337127 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104167.90 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 10416.79 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117821.126981 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93751.11 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 9375.11 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127196.237849 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84376.00 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 8437.60 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135633.837630 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75938.40 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 7593.84 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143227.677433 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68344.56 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 6834.46 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150062.133255 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61510.10 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 6151.01 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156213.143496 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55359.09 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 5535.91 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161749.052712 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49823.18 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 4982.32 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166731.371007 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44840.86 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 4484.09 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171215.457472 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40356.78 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 4035.68 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175251.135291 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36321.10 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3632.11 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178883.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32688.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182483.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29088.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186083.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25488.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189683.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21888.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193283.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18288.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196883.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14688.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200483.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11088.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204083.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7488.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207683.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3888.99 sec) 19:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211283.245328 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 288.99 sec) 19:07:00 [GPUGRID] [rr_sim] 2211572.24: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972602776.28G/439.78G) 19:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853433.15 19:07:00 [---] pick_jobs_to_run() start 19:07:00 [---] pick_jobs_to_run() end 19:07:00 [---] [rr_sim] end 19:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:07:00 [---] [cpu_sched_debug] preliminary job list: 19:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:07:00 [---] [cpu_sched_debug] final job list: 19:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:07:00 [---] [cpu_sched_debug] enforce_run_list: end 19:07:00 [---] choose_project(): 68820.000000 19:07:00 [WUProp@Home] doing RPC to get work 19:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366779.09 cpu 2115092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115092.90 19:07:00 [---] Got 0 tasks 19:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:08:00 [---] [rr_sim] doing sim: CPU sched 19:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972575185.93G/439.78G) 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211509.50 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 221150.95 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221150.949878 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990358.55 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 199035.85 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420186.804768 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791322.69 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 179132.27 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599319.074169 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612190.42 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 161219.04 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760538.116630 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450971.38 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 145097.14 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905635.254844 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305874.24 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 130587.42 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036222.679238 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175286.82 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 117528.68 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153751.361192 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057758.14 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 105775.81 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259527.174950 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951982.32 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 95198.23 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354725.407333 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856784.09 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 85678.41 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440403.816477 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771105.68 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 77110.57 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517514.384707 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693995.11 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 69399.51 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586913.896114 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624595.60 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 62459.56 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649373.456381 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562136.04 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 56213.60 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705587.060620 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505922.44 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 50592.24 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756179.304436 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455330.19 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 45533.02 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801712.323870 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409797.17 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 40979.72 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842692.041361 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368817.46 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 36881.75 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879573.787103 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331935.71 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 33193.57 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912767.358270 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298742.14 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 29874.21 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942641.572321 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268867.93 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 26886.79 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969528.364966 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241981.13 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 24198.11 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993726.478348 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217783.02 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 21778.30 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015504.780391 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196004.72 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 19600.47 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035105.252229 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176404.25 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 17640.42 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052745.676884 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158763.82 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 15876.38 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068622.059073 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142887.44 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 14288.74 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082910.803044 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128598.70 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 12859.87 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095770.672617 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115738.83 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 11573.88 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107344.555233 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104164.94 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 10416.49 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117761.049588 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93748.45 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 9374.84 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127135.894507 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84373.60 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 8437.36 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135573.254934 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75936.24 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 7593.62 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143166.879318 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68342.62 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 6834.26 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150001.141264 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61508.36 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 6150.84 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156151.977015 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55357.52 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 5535.75 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161687.729192 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49821.77 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 4982.18 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166669.906150 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44839.59 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 4483.96 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171153.865413 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40355.63 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 4035.56 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175189.428749 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36320.07 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3632.01 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178821.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32688.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182421.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29088.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186021.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25488.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189621.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21888.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193221.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18288.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196821.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14688.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200421.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11088.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204021.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7488.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207621.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3888.06 sec) 19:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211221.435752 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 288.06 sec) 19:08:00 [GPUGRID] [rr_sim] 2211509.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972575185.93G/439.78G) 19:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853430.41 19:08:00 [---] pick_jobs_to_run() start 19:08:00 [---] pick_jobs_to_run() end 19:08:00 [---] [rr_sim] end 19:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:08:00 [---] [cpu_sched_debug] preliminary job list: 19:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:08:00 [---] [cpu_sched_debug] final job list: 19:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:08:00 [---] [cpu_sched_debug] enforce_run_list: end 19:08:00 [---] choose_project(): 68880.000000 19:08:00 [WUProp@Home] doing RPC to get work 19:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366719.09 cpu 2115032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2115032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2115032.90 19:08:00 [---] Got 0 tasks 19:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:09:00 [---] [rr_sim] doing sim: CPU sched 19:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972547595.57G/439.78G) 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211446.76 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 221144.68 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221144.676190 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990302.09 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 199030.21 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420174.884760 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791271.88 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 179127.19 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599302.072474 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612144.69 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 161214.47 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760516.541416 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450930.22 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 145093.02 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905609.563464 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305837.20 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 130583.72 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036193.283307 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175253.48 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 117525.35 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153718.631166 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057728.13 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 105772.81 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259491.444239 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951955.32 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 95195.53 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354686.976004 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856759.79 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 85675.98 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440362.954593 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771083.81 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 77108.38 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517471.335324 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693975.43 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 69397.54 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586868.877981 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624577.88 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 62457.79 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649326.666372 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562120.10 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 56212.01 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705538.675925 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505908.09 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 50590.81 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756129.484522 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455317.28 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 45531.73 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801661.212259 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409785.55 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 40978.55 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842639.767223 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368806.99 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 36880.70 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879520.466690 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331926.30 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 33192.63 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912713.096210 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298733.67 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 29873.37 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942586.462779 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268860.30 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 26886.03 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969472.492691 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241974.27 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 24197.43 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993669.919611 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217776.84 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 21777.68 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015447.603839 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195999.16 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 19599.92 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035047.519645 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176399.24 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 17639.92 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052687.443870 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158759.32 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 15875.93 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068563.375673 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142883.39 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 14288.34 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082851.714295 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128595.05 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 12859.50 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095711.219055 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115735.54 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 11573.55 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107284.773339 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104161.99 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 10416.20 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117700.972195 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93745.79 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 9374.58 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127075.551165 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84371.21 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 8437.12 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135512.672238 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75934.09 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 7593.41 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143106.081203 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68340.68 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 6834.07 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149940.149273 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61506.61 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 6150.66 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156090.810535 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55355.95 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 5535.60 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161626.405671 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49820.36 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 4982.04 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166608.441293 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44838.32 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 4483.83 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171092.273354 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40354.49 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 4035.45 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175127.722208 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36319.04 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3631.90 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178759.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32687.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182359.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29087.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185959.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25487.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189559.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21887.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193159.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18287.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196759.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14687.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200359.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11087.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203959.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7487.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207559.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3887.14 sec) 19:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211159.626177 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 287.14 sec) 19:09:00 [GPUGRID] [rr_sim] 2211446.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972547595.57G/439.78G) 19:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853427.67 19:09:00 [---] pick_jobs_to_run() start 19:09:00 [---] pick_jobs_to_run() end 19:09:00 [---] [rr_sim] end 19:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:09:00 [---] [cpu_sched_debug] preliminary job list: 19:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:09:00 [---] [cpu_sched_debug] final job list: 19:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:09:00 [---] [cpu_sched_debug] enforce_run_list: end 19:09:00 [---] choose_project(): 68940.000000 19:09:00 [WUProp@Home] doing RPC to get work 19:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366659.09 cpu 2114972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114972.90 19:09:00 [---] Got 0 tasks 19:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:10:00 [---] [rr_sim] doing sim: CPU sched 19:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972520005.21G/439.78G) 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211384.03 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 221138.40 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221138.402501 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990245.62 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 199024.56 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420162.964752 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791221.06 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 179122.11 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599285.070779 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612098.95 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 161209.90 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760494.966202 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450889.06 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 145088.91 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905583.872083 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305800.15 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 130580.02 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036163.887376 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175220.14 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 117522.01 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153685.901140 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057698.12 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 105769.81 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259455.713527 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951928.31 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 95192.83 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354648.544676 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856735.48 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 85673.55 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440322.092709 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771061.93 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 77106.19 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517428.285940 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693955.74 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 69395.57 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586823.859847 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624560.17 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 62456.02 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649279.876364 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562104.15 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 56210.41 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705490.291229 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505893.73 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 50589.37 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756079.664607 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455304.36 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 45530.44 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801610.100648 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409773.92 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 40977.39 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842587.493084 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368796.53 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 36879.65 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879467.146277 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331916.88 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 33191.69 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912658.834151 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298725.19 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 29872.52 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942531.353237 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268852.67 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 26885.27 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969416.620415 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241967.40 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 24196.74 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993613.360874 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217770.66 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 21777.07 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015390.427288 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195993.60 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 19599.36 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034989.787061 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176394.24 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 17639.42 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052629.210856 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158754.81 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 15875.48 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068504.692272 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142879.33 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 14287.93 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082792.625546 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128591.40 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 12859.14 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095651.765493 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115732.26 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 11573.23 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107224.991445 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104159.03 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 10415.90 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117640.894802 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93743.13 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 9374.31 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127015.207823 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84368.82 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 8436.88 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135452.089542 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75931.94 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 7593.19 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143045.283089 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68338.74 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 6833.87 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149879.157281 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61504.87 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 6150.49 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156029.644054 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55354.38 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 5535.44 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161565.082150 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49818.94 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 4981.89 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166546.976437 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44837.05 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 4483.70 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171030.681294 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40353.34 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 4035.33 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175066.015666 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36318.01 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3631.80 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178697.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32686.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182297.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29086.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185897.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25486.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189497.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21886.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193097.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18286.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196697.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14686.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200297.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11086.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203897.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7486.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207497.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3886.21 sec) 19:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211097.816601 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 286.21 sec) 19:10:00 [GPUGRID] [rr_sim] 2211384.03: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972520005.21G/439.78G) 19:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853424.94 19:10:00 [---] pick_jobs_to_run() start 19:10:00 [---] pick_jobs_to_run() end 19:10:00 [---] [rr_sim] end 19:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:10:00 [---] [cpu_sched_debug] preliminary job list: 19:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:10:00 [---] [cpu_sched_debug] final job list: 19:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:10:00 [---] [cpu_sched_debug] enforce_run_list: end 19:10:00 [---] choose_project(): 69000.000000 19:10:00 [WUProp@Home] doing RPC to get work 19:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366599.09 cpu 2114912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114912.90 19:10:00 [---] Got 0 tasks 19:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:11:00 [---] [rr_sim] doing sim: CPU sched 19:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972492414.86G/439.78G) 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211321.29 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 221132.13 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221132.128813 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990189.16 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 199018.92 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420151.044745 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791170.24 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 179117.02 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599268.069083 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612053.22 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 161205.32 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760473.390988 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450847.90 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 145084.79 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905558.180703 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305763.11 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 130576.31 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036134.491445 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175186.80 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 117518.68 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153653.171114 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057668.12 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 105766.81 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259419.982816 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951901.31 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 95190.13 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354610.113347 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856711.17 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 85671.12 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440281.230826 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771040.06 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 77104.01 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517385.236556 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693936.05 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 69393.61 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586778.841714 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624542.45 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 62454.24 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649233.086355 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562088.20 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 56208.82 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705441.906533 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505879.38 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 50587.94 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1756029.844693 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455291.44 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 45529.14 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801558.989037 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409762.30 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 40976.23 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842535.218946 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368786.07 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 36878.61 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879413.825864 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331907.46 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 33190.75 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912604.572091 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298716.72 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 29871.67 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942476.243695 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268845.04 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 26884.50 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969360.748139 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241960.54 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 24196.05 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993556.802138 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217764.49 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 21776.45 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015333.250737 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195988.04 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 19598.80 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034932.054477 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176389.23 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 17638.92 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052570.977842 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158750.31 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 15875.03 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068446.008871 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142875.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 14287.53 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082733.536797 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128587.75 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 12858.78 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095592.311930 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115728.98 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 11572.90 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107165.209550 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104156.08 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 10415.61 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117580.817408 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93740.47 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 9374.05 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126954.864481 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84366.42 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 8436.64 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135391.506846 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75929.78 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 7592.98 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142984.484974 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68336.80 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 6833.68 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149818.165290 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61503.12 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 6150.31 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155968.477574 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55352.81 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 5535.28 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161503.758630 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49817.53 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 4981.75 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166485.511580 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44835.78 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 4483.58 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170969.089235 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40352.20 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 4035.22 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175004.309125 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36316.98 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3631.70 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178636.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32685.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182236.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29085.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185836.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25485.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189436.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21885.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193036.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18285.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196636.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14685.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200236.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11085.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203836.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7485.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207436.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3885.28 sec) 19:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2211036.007025 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 285.28 sec) 19:11:00 [GPUGRID] [rr_sim] 2211321.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972492414.86G/439.78G) 19:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853422.20 19:11:00 [---] pick_jobs_to_run() start 19:11:00 [---] pick_jobs_to_run() end 19:11:00 [---] [rr_sim] end 19:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:11:00 [---] [cpu_sched_debug] preliminary job list: 19:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:11:00 [---] [cpu_sched_debug] final job list: 19:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:11:00 [---] [cpu_sched_debug] enforce_run_list: end 19:11:00 [---] choose_project(): 69060.000000 19:11:00 [WUProp@Home] doing RPC to get work 19:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366539.09 cpu 2114852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114852.90 19:11:00 [---] Got 0 tasks 19:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:12:00 [---] [rr_sim] doing sim: CPU sched 19:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972464824.50G/439.78G) 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211258.55 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 221125.86 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221125.855125 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990132.70 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 199013.27 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420139.124737 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791119.43 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 179111.94 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599251.067388 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1612007.48 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 161200.75 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760451.815774 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450806.74 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 145080.67 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905532.489322 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305726.06 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 130572.61 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036105.095515 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175153.46 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 117515.35 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153620.441088 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057638.11 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 105763.81 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259384.252104 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951874.30 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 95187.43 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354571.682019 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856686.87 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 85668.69 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440240.368942 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 771018.18 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 77101.82 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517342.187172 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693916.36 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 69391.64 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586733.823580 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624524.73 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 62452.47 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649186.296347 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562072.25 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 56207.23 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705393.521837 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505865.03 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 50586.50 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755980.024778 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455278.53 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 45527.85 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801507.877425 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409750.67 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 40975.07 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842482.944808 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368775.61 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 36877.56 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879360.505452 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331898.05 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 33189.80 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912550.310031 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298708.24 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 29870.82 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942421.134153 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268837.42 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 26883.74 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969304.875863 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241953.68 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 24195.37 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993500.243401 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217758.31 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 21775.83 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015276.074186 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195982.48 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 19598.25 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034874.321892 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176384.23 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 17638.42 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052512.744828 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158745.81 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 15874.58 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068387.325470 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142871.23 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 14287.12 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082674.448048 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128584.10 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 12858.41 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095532.858368 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115725.69 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 11572.57 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107105.427656 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104153.12 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 10415.31 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117520.740015 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93737.81 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 9373.78 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126894.521139 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84364.03 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 8436.40 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135330.924150 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75927.63 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 7592.76 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142923.686860 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68334.86 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 6833.49 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149757.173299 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61501.38 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 6150.14 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155907.311094 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55351.24 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 5535.12 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161442.435109 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49816.12 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 4981.61 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166424.046723 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44834.50 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 4483.45 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170907.497176 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40351.05 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 4035.11 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174942.602583 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36315.95 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3631.59 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178574.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32684.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182174.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29084.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185774.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25484.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189374.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21884.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192974.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18284.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196574.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14684.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200174.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11084.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203774.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7484.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207374.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3884.35 sec) 19:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210974.197450 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 284.35 sec) 19:12:00 [GPUGRID] [rr_sim] 2211258.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972464824.50G/439.78G) 19:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853419.46 19:12:00 [---] pick_jobs_to_run() start 19:12:00 [---] pick_jobs_to_run() end 19:12:00 [---] [rr_sim] end 19:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:12:00 [---] [cpu_sched_debug] preliminary job list: 19:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:12:00 [---] [cpu_sched_debug] final job list: 19:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:12:00 [---] [cpu_sched_debug] enforce_run_list: end 19:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:13:00 [---] [rr_sim] doing sim: CPU sched 19:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972437234.15G/439.78G) 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211195.81 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 221119.58 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221119.581437 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990076.23 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 199007.62 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420127.204730 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791068.61 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 179106.86 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599234.065693 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611961.75 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 161196.17 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760430.240561 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450765.57 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 145076.56 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905506.797941 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305689.02 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 130568.90 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036075.699584 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175120.11 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 117512.01 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153587.711062 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057608.10 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 105760.81 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259348.521393 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951847.29 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 95184.73 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354533.250690 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856662.56 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 85666.26 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440199.507058 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770996.31 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 77099.63 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517299.137789 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693896.68 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 69389.67 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586688.805446 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624507.01 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 62450.70 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649139.506338 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562056.31 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 56205.63 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705345.137141 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505850.68 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 50585.07 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755930.204864 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455265.61 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 45526.56 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801456.765814 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409739.05 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 40973.90 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842430.670669 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368765.14 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 36876.51 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879307.185039 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331888.63 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 33188.86 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912496.047972 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298699.77 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 29869.98 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942366.024611 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268829.79 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 26882.98 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969249.003587 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241946.81 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 24194.68 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993443.684665 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217752.13 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 21775.21 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015218.897635 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195976.92 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 19597.69 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034816.589308 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176379.23 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 17637.92 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052454.511814 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158741.30 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 15874.13 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068328.642069 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142867.17 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 14286.72 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082615.359299 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128580.46 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 12858.05 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095473.404806 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115722.41 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 11572.24 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107045.645762 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104150.17 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 10415.02 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117460.662622 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93735.15 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 9373.52 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126834.177797 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84361.64 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 8436.16 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135270.341454 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75925.47 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 7592.55 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142862.888745 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68332.93 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 6833.29 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149696.181307 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61499.63 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 6149.96 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155846.144613 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55349.67 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 5534.97 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161381.111588 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49814.70 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 4981.47 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166362.581866 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44833.23 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 4483.32 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170845.905116 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40349.91 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 4034.99 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174880.896041 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36314.92 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3631.49 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178512.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32683.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182112.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29083.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185712.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25483.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189312.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21883.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192912.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18283.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196512.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14683.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200112.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11083.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203712.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7483.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207312.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3883.43 sec) 19:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210912.387874 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 283.43 sec) 19:13:00 [GPUGRID] [rr_sim] 2211195.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972437234.15G/439.78G) 19:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853416.73 19:13:00 [---] pick_jobs_to_run() start 19:13:00 [---] pick_jobs_to_run() end 19:13:00 [---] [rr_sim] end 19:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:13:00 [---] [cpu_sched_debug] preliminary job list: 19:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:13:00 [---] [cpu_sched_debug] final job list: 19:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:13:00 [---] [cpu_sched_debug] enforce_run_list: end 19:13:00 [---] choose_project(): 69180.000000 19:13:00 [WUProp@Home] doing RPC to get work 19:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366419.09 cpu 2114732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114732.90 19:13:00 [---] Got 0 tasks 19:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:14:00 [---] [rr_sim] doing sim: CPU sched 19:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972409643.79G/439.78G) 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211133.08 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 221113.31 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221113.307748 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990019.77 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 199001.98 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420115.284722 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791017.79 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 179101.78 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599217.063998 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611916.01 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 161191.60 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760408.665347 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450724.41 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 145072.44 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905481.106561 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305651.97 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 130565.20 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036046.303653 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175086.77 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 117508.68 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153554.981036 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057578.10 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 105757.81 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259312.790681 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951820.29 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 95182.03 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354494.819361 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856638.26 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 85663.83 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440158.645174 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770974.43 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 77097.44 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517256.088405 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693876.99 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 69387.70 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586643.787313 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624489.29 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 62448.93 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649092.716330 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562040.36 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 56204.04 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705296.752445 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505836.33 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 50583.63 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755880.384949 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455252.69 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 45525.27 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801405.654203 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409727.42 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 40972.74 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842378.396531 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368754.68 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 36875.47 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879253.864626 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331879.21 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 33187.92 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912441.785912 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298691.29 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 29869.13 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942310.915069 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268822.16 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 26882.22 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969193.131311 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241939.95 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 24193.99 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993387.125928 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217745.95 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 21774.60 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015161.721084 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195971.36 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 19597.14 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034758.856724 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176374.22 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 17637.42 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052396.278800 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158736.80 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 15873.68 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068269.958668 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142863.12 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 14286.31 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082556.270550 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128576.81 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 12857.68 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095413.951243 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115719.13 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 11571.91 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106985.863868 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104147.21 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 10414.72 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117400.585229 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93732.49 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 9373.25 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126773.834455 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84359.24 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 8435.92 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135209.758758 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75923.32 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 7592.33 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142802.090630 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68330.99 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 6833.10 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149635.189316 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61497.89 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 6149.79 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155784.978133 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55348.10 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 5534.81 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161319.788068 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49813.29 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 4981.33 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166301.117010 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44831.96 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 4483.20 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170784.313057 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40348.76 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 4034.88 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174819.189500 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36313.89 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3631.39 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178450.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32682.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182050.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29082.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185650.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25482.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189250.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21882.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192850.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18282.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196450.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14682.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200050.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11082.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203650.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7482.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207250.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3882.50 sec) 19:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210850.578298 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 282.50 sec) 19:14:00 [GPUGRID] [rr_sim] 2211133.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972409643.79G/439.78G) 19:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853413.99 19:14:00 [---] pick_jobs_to_run() start 19:14:00 [---] pick_jobs_to_run() end 19:14:00 [---] [rr_sim] end 19:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:14:00 [---] [cpu_sched_debug] preliminary job list: 19:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:14:00 [---] [cpu_sched_debug] final job list: 19:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:14:00 [---] [cpu_sched_debug] enforce_run_list: end 19:14:00 [---] choose_project(): 69240.000000 19:14:00 [WUProp@Home] doing RPC to get work 19:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366359.09 cpu 2114672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114672.90 19:14:00 [---] Got 0 tasks 19:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:15:00 [---] [rr_sim] doing sim: CPU sched 19:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972409643.79G/439.78G) 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211133.08 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 221113.31 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221113.307748 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1990019.77 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 199001.98 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420115.284722 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1791017.79 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 179101.78 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599217.063998 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611916.01 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 161191.60 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760408.665347 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450724.41 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 145072.44 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905481.106561 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305651.97 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 130565.20 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036046.303653 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175086.77 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 117508.68 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153554.981036 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057578.10 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 105757.81 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259312.790681 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951820.29 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 95182.03 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354494.819361 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856638.26 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 85663.83 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440158.645174 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770974.43 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 77097.44 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517256.088405 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693876.99 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 69387.70 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586643.787313 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624489.29 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 62448.93 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649092.716330 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562040.36 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 56204.04 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705296.752445 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505836.33 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 50583.63 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755880.384949 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455252.69 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 45525.27 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801405.654203 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409727.42 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 40972.74 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842378.396531 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368754.68 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 36875.47 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879253.864626 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331879.21 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 33187.92 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912441.785912 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298691.29 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 29869.13 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942310.915069 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268822.16 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 26882.22 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969193.131311 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241939.95 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 24193.99 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993387.125928 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217745.95 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 21774.60 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015161.721084 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195971.36 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 19597.14 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034758.856724 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176374.22 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 17637.42 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052396.278800 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158736.80 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 15873.68 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068269.958668 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142863.12 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 14286.31 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082556.270550 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128576.81 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 12857.68 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095413.951243 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115719.13 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 11571.91 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106985.863868 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104147.21 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 10414.72 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117400.585229 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93732.49 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 9373.25 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126773.834455 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84359.24 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 8435.92 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135209.758758 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75923.32 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 7592.33 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142802.090630 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68330.99 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 6833.10 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149635.189316 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61497.89 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 6149.79 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155784.978133 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55348.10 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 5534.81 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161319.788068 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49813.29 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 4981.33 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166301.117010 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44831.96 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 4483.20 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170784.313057 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40348.76 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 4034.88 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174819.189500 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36313.89 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3631.39 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178450.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32682.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182050.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29082.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185650.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25482.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189250.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21882.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192850.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18282.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196450.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14682.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200050.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11082.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203650.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7482.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207250.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3882.50 sec) 19:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210850.578298 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 282.50 sec) 19:15:00 [GPUGRID] [rr_sim] 2211133.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972409643.79G/439.78G) 19:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853473.99 19:15:00 [---] pick_jobs_to_run() start 19:15:00 [---] pick_jobs_to_run() end 19:15:00 [---] [rr_sim] end 19:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:15:00 [---] [cpu_sched_debug] preliminary job list: 19:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:15:00 [---] [cpu_sched_debug] final job list: 19:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:15:00 [---] [cpu_sched_debug] enforce_run_list: end 19:15:00 [---] choose_project(): 69300.000000 19:15:00 [WUProp@Home] doing RPC to get work 19:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366299.09 cpu 2114672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114672.90 19:15:00 [---] Got 0 tasks 19:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:16:00 [---] [rr_sim] doing sim: CPU sched 19:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972382053.43G/439.78G) 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211070.34 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 221107.03 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221107.034060 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989963.31 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 198996.33 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420103.364714 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790966.98 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 179096.70 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599200.062303 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611870.28 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 161187.03 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760387.090133 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450683.25 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 145068.33 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905455.415180 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305614.93 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 130561.49 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036016.907722 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175053.43 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 117505.34 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153522.251010 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057548.09 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 105754.81 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259277.059969 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951793.28 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 95179.33 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354456.388033 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856613.95 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 85661.40 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440117.783290 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770952.56 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 77095.26 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517213.039021 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693857.30 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 69385.73 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586598.769179 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624471.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 62447.16 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649045.926321 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562024.41 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 56202.44 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705248.367750 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505821.97 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 50582.20 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755830.565035 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455239.78 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 45523.98 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801354.542592 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409715.80 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 40971.58 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842326.122393 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368744.22 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 36874.42 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879200.544214 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331869.80 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 33186.98 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912387.523852 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298682.82 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 29868.28 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942255.805527 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268814.54 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 26881.45 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969137.259035 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241933.08 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 24193.31 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993330.567192 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217739.77 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 21773.98 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015104.544533 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195965.80 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 19596.58 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034701.124140 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176369.22 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 17636.92 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052338.045786 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158732.29 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 15873.23 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068211.275268 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142859.07 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 14285.91 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082497.181801 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128573.16 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 12857.32 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095354.497681 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115715.84 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 11571.58 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106926.081973 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104144.26 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 10414.43 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117340.507836 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93729.83 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 9372.98 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126713.491113 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84356.85 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 8435.68 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135149.176062 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75921.16 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 7592.12 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142741.292516 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68329.05 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 6832.90 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149574.197324 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61496.14 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 6149.61 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155723.811652 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55346.53 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 5534.65 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161258.464547 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49811.88 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 4981.19 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166239.652153 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44830.69 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 4483.07 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170722.720998 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40347.62 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 4034.76 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174757.482958 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36312.86 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3631.29 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178388.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32681.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181988.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29081.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185588.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25481.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189188.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21881.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192788.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18281.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196388.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14681.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199988.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11081.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203588.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7481.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207188.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3881.57 sec) 19:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210788.768723 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 281.57 sec) 19:16:00 [GPUGRID] [rr_sim] 2211070.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972382053.43G/439.78G) 19:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853471.25 19:16:00 [---] pick_jobs_to_run() start 19:16:00 [---] pick_jobs_to_run() end 19:16:00 [---] [rr_sim] end 19:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:16:00 [---] [cpu_sched_debug] preliminary job list: 19:16:00 [---] [cpu_sched_debug] final job list: 19:16:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 19:16:00 [---] [work_fetch] Request work fetch: CPUs idle 19:16:00 [---] [cpu_sched_debug] enforce_run_list: end 19:16:00 [---] choose_project(): 69360.000000 19:16:00 [WUProp@Home] doing RPC to get work 19:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366239.09 cpu 2114612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114612.90 19:16:00 [---] Got 0 tasks 19:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:17:00 [---] [rr_sim] doing sim: CPU sched 19:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972382053.43G/439.78G) 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211070.34 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 221107.03 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221107.034060 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989963.31 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 198996.33 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420103.364714 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790966.98 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 179096.70 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599200.062303 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611870.28 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 161187.03 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760387.090133 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450683.25 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 145068.33 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905455.415180 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305614.93 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 130561.49 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1036016.907722 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175053.43 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 117505.34 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153522.251010 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057548.09 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 105754.81 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259277.059969 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951793.28 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 95179.33 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354456.388033 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856613.95 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 85661.40 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440117.783290 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770952.56 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 77095.26 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517213.039021 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693857.30 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 69385.73 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586598.769179 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624471.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 62447.16 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1649045.926321 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562024.41 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 56202.44 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705248.367750 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505821.97 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 50582.20 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755830.565035 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455239.78 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 45523.98 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801354.542592 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409715.80 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 40971.58 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842326.122393 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368744.22 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 36874.42 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879200.544214 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331869.80 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 33186.98 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912387.523852 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298682.82 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 29868.28 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942255.805527 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268814.54 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 26881.45 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969137.259035 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241933.08 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 24193.31 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993330.567192 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217739.77 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 21773.98 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015104.544533 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195965.80 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 19596.58 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034701.124140 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176369.22 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 17636.92 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052338.045786 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158732.29 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 15873.23 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068211.275268 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142859.07 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 14285.91 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082497.181801 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128573.16 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 12857.32 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095354.497681 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115715.84 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 11571.58 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106926.081973 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104144.26 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 10414.43 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117340.507836 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93729.83 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 9372.98 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126713.491113 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84356.85 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 8435.68 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135149.176062 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75921.16 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 7592.12 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142741.292516 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68329.05 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 6832.90 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149574.197324 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61496.14 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 6149.61 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155723.811652 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55346.53 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 5534.65 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161258.464547 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49811.88 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 4981.19 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166239.652153 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44830.69 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 4483.07 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170722.720998 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40347.62 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 4034.76 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174757.482958 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36312.86 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3631.29 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178388.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32681.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181988.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29081.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185588.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25481.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189188.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21881.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192788.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18281.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196388.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14681.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199988.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11081.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203588.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7481.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207188.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3881.57 sec) 19:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210788.768723 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 281.57 sec) 19:17:00 [GPUGRID] [rr_sim] 2211070.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972382053.43G/439.78G) 19:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853531.25 19:17:00 [---] pick_jobs_to_run() start 19:17:00 [---] pick_jobs_to_run() end 19:17:00 [---] [rr_sim] end 19:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:17:00 [---] [cpu_sched_debug] preliminary job list: 19:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:17:00 [---] [cpu_sched_debug] final job list: 19:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 19:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:17:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 972382053.43G 19:17:00 [---] [cpu_sched_debug] enforce_run_list: end 19:17:00 [---] choose_project(): 69420.000000 19:17:00 [WUProp@Home] doing RPC to get work 19:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366179.09 cpu 2114612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114612.90 19:17:00 [---] Got 0 tasks 19:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:18:00 [---] [rr_sim] doing sim: CPU sched 19:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972354463.08G/439.78G) 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2211007.60 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 221100.76 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221100.760372 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989906.84 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 198990.68 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420091.444707 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790916.16 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 179091.62 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599183.060608 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611824.54 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 161182.45 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760365.514919 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450642.09 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 145064.21 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905429.723799 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305577.88 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 130557.79 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035987.511792 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1175020.09 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 117502.01 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153489.520984 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057518.08 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 105751.81 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259241.329258 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951766.27 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 95176.63 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354417.956704 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856589.65 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 85658.96 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440076.921406 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770930.68 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 77093.07 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517169.989637 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693837.61 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 69383.76 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586553.751045 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624453.85 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 62445.39 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648999.136313 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 562008.47 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 56200.85 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705199.983054 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505807.62 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 50580.76 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755780.745120 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455226.86 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 45522.69 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801303.430980 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409704.17 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 40970.42 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842273.848254 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368733.76 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 36873.38 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879147.223801 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331860.38 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 33186.04 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912333.261793 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298674.34 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 29867.43 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942200.695986 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268806.91 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 26880.69 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969081.386759 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241926.22 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 24192.62 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993274.008455 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217733.60 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 21773.36 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2015047.367982 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195960.24 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 19596.02 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034643.391555 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176364.21 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 17636.42 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052279.812772 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158727.79 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 15872.78 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068152.591867 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142855.01 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 14285.50 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082438.093052 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128569.51 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 12856.95 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095295.044119 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115712.56 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 11571.26 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106866.300079 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104141.30 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 10414.13 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117280.430443 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93727.17 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 9372.72 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126653.147771 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84354.46 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 8435.45 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135088.593366 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75919.01 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 7591.90 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142680.494401 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68327.11 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 6832.71 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149513.205333 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61494.40 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 6149.44 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155662.645172 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55344.96 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 5534.50 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161197.141027 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49810.46 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 4981.05 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166178.187296 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44829.42 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 4482.94 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170661.128938 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40346.47 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 4034.65 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174695.776417 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36311.83 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3631.18 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178326.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32680.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181926.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29080.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185526.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25480.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189126.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21880.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192726.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18280.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196326.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14680.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199926.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11080.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203526.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7480.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207126.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3880.64 sec) 19:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210726.959147 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 280.64 sec) 19:18:00 [GPUGRID] [rr_sim] 2211007.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972354463.08G/439.78G) 19:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853528.52 19:18:00 [---] pick_jobs_to_run() start 19:18:00 [---] pick_jobs_to_run() end 19:18:00 [---] [rr_sim] end 19:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:18:00 [---] [cpu_sched_debug] preliminary job list: 19:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 19:18:00 [---] [cpu_sched_debug] final job list: 19:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:18:00 [---] [cpu_sched_debug] enforce_run_list: end 19:18:00 [---] choose_project(): 69480.000000 19:18:00 [WUProp@Home] doing RPC to get work 19:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366119.09 cpu 2114552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114552.90 19:18:00 [---] Got 0 tasks 19:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:19:00 [---] [rr_sim] doing sim: CPU sched 19:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972326872.72G/439.78G) 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210944.87 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 221094.49 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221094.486684 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989850.38 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 198985.04 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420079.524699 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790865.34 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 179086.53 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599166.058913 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611778.81 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 161177.88 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760343.939706 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450600.93 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 145060.09 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905404.032419 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305540.83 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 130554.08 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035958.115861 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174986.75 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 117498.68 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153456.790958 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057488.08 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 105748.81 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259205.598546 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951739.27 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 95173.93 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354379.525376 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856565.34 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 85656.53 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1440036.059522 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770908.81 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 77090.88 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517126.940253 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693817.93 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 69381.79 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586508.732912 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624436.13 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 62443.61 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648952.346304 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561992.52 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 56199.25 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705151.598358 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505793.27 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 50579.33 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755730.925206 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455213.94 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 45521.39 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801252.319369 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409692.55 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 40969.25 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842221.574116 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368723.29 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 36872.33 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879093.903388 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331850.96 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 33185.10 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912278.999733 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298665.87 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 29866.59 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942145.586444 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268799.28 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 26879.93 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1969025.514483 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241919.35 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 24191.94 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993217.449719 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217727.42 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 21772.74 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014990.191430 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195954.68 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 19595.47 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034585.658971 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176359.21 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 17635.92 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052221.579758 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158723.29 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 15872.33 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068093.908466 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142850.96 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 14285.10 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082379.004303 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128565.86 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 12856.59 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095235.590557 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115709.28 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 11570.93 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106806.518185 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104138.35 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 10413.83 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117220.353050 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93724.51 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 9372.45 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126592.804429 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84352.06 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 8435.21 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135028.010670 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75916.86 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 7591.69 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142619.696287 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68325.17 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 6832.52 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149452.213342 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61492.65 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 6149.27 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155601.478691 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55343.39 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 5534.34 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161135.817506 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49809.05 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 4980.90 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166116.722439 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44828.14 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 4482.81 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170599.536879 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40345.33 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 4034.53 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174634.069875 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36310.80 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3631.08 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178265.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32679.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181865.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29079.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185465.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25479.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189065.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21879.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192665.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18279.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196265.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14679.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199865.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11079.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203465.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7479.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207065.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3879.72 sec) 19:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210665.149571 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 279.72 sec) 19:19:00 [GPUGRID] [rr_sim] 2210944.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972326872.72G/439.78G) 19:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853525.78 19:19:00 [---] pick_jobs_to_run() start 19:19:00 [---] pick_jobs_to_run() end 19:19:00 [---] [rr_sim] end 19:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:19:00 [---] [cpu_sched_debug] preliminary job list: 19:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:19:00 [---] [cpu_sched_debug] final job list: 19:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:19:00 [---] [cpu_sched_debug] enforce_run_list: end 19:19:00 [---] choose_project(): 69540.000000 19:19:00 [WUProp@Home] doing RPC to get work 19:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 366059.09 cpu 2114492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114492.90 19:19:00 [---] Got 0 tasks 19:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:20:00 [---] [rr_sim] doing sim: CPU sched 19:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972299282.36G/439.78G) 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210882.13 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 221088.21 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221088.212996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989793.92 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 198979.39 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420067.604692 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790814.53 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 179081.45 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599149.057218 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611733.07 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 161173.31 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760322.364492 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450559.77 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 145055.98 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905378.341038 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305503.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 130550.38 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035928.719930 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174953.41 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 117495.34 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153424.060933 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057458.07 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 105745.81 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259169.867835 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951712.26 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 95171.23 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354341.094047 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856541.04 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 85654.10 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439995.197638 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770886.93 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 77088.69 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517083.890870 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693798.24 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 69379.82 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586463.714778 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624418.42 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 62441.84 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648905.556296 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561976.57 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 56197.66 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705103.213662 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505778.92 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 50577.89 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755681.105291 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455201.02 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 45520.10 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801201.207758 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409680.92 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 40968.09 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842169.299978 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368712.83 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 36871.28 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1879040.582975 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331841.55 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 33184.15 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912224.737673 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298657.39 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 29865.74 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942090.476902 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268791.65 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 26879.17 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968969.642207 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241912.49 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 24191.25 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993160.890982 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217721.24 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 21772.12 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014933.014879 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195949.12 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 19594.91 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034527.926387 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176354.20 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 17635.42 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052163.346744 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158718.78 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 15871.88 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068035.225065 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142846.90 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 14284.69 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082319.915554 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128562.21 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 12856.22 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095176.136994 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115705.99 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 11570.60 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106746.736290 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104135.39 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 10413.54 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117160.275657 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93721.85 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 9372.19 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126532.461087 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84349.67 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 8434.97 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134967.427974 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75914.70 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 7591.47 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142558.898172 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68323.23 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 6832.32 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149391.221350 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61490.91 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 6149.09 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155540.312211 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55341.82 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 5534.18 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161074.493985 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49807.64 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 4980.76 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166055.257582 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44826.87 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 4482.69 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170537.944820 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40344.19 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 4034.42 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174572.363333 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36309.77 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3630.98 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178203.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32678.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181803.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29078.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185403.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25478.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189003.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21878.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192603.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18278.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196203.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14678.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199803.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11078.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203403.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7478.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207003.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3878.79 sec) 19:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210603.339996 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 278.79 sec) 19:20:00 [GPUGRID] [rr_sim] 2210882.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972299282.36G/439.78G) 19:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853523.04 19:20:00 [---] pick_jobs_to_run() start 19:20:00 [---] pick_jobs_to_run() end 19:20:00 [---] [rr_sim] end 19:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:20:00 [---] [cpu_sched_debug] preliminary job list: 19:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:20:00 [---] [cpu_sched_debug] final job list: 19:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:20:00 [---] [cpu_sched_debug] enforce_run_list: end 19:20:00 [---] choose_project(): 69600.000000 19:20:00 [WUProp@Home] doing RPC to get work 19:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365999.09 cpu 2114432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114432.90 19:20:00 [---] Got 0 tasks 19:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:21:00 [---] [rr_sim] doing sim: CPU sched 19:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972271692.01G/439.78G) 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210819.39 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 221081.94 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221081.939307 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989737.45 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 198973.75 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420055.684684 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790763.71 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 179076.37 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599132.055523 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611687.34 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 161168.73 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760300.789278 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450518.60 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 145051.86 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905352.649658 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305466.74 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 130546.67 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035899.323999 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174920.07 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 117492.01 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153391.330907 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057428.06 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 105742.81 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259134.137123 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951685.26 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 95168.53 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354302.662718 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856516.73 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 85651.67 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439954.335754 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770865.06 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 77086.51 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1517040.841486 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693778.55 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 69377.86 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586418.696645 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624400.70 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 62440.07 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648858.766288 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561960.63 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 56196.06 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705054.828966 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505764.56 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 50576.46 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755631.285377 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455188.11 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 45518.81 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801150.096147 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409669.30 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 40966.93 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842117.025839 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368702.37 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 36870.24 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878987.262563 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331832.13 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 33183.21 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912170.475614 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298648.92 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 29864.89 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1942035.367360 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268784.03 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 26878.40 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968913.769931 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241905.62 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 24190.56 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993104.332245 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217715.06 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 21771.51 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014875.838328 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195943.55 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 19594.36 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034470.193803 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176349.20 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 17634.92 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052105.113730 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158714.28 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 15871.43 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067976.541664 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142842.85 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 14284.29 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082260.826805 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128558.57 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 12855.86 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095116.683432 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115702.71 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 11570.27 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106686.954396 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104132.44 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 10413.24 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117100.198264 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93719.19 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 9371.92 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126472.117745 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84347.28 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 8434.73 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134906.845278 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75912.55 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 7591.25 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142498.100057 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68321.29 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 6832.13 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149330.229359 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61489.16 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 6148.92 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155479.145730 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55340.25 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 5534.02 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161013.170465 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49806.22 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 4980.62 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165993.792726 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44825.60 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 4482.56 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170476.352760 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40343.04 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 4034.30 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174510.656792 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36308.74 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3630.87 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178141.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32677.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181741.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29077.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185341.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25477.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188941.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21877.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192541.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18277.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196141.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14677.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199741.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11077.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203341.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7477.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206941.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3877.86 sec) 19:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210541.530420 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 277.86 sec) 19:21:00 [GPUGRID] [rr_sim] 2210819.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972271692.01G/439.78G) 19:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853520.31 19:21:00 [---] pick_jobs_to_run() start 19:21:00 [---] pick_jobs_to_run() end 19:21:00 [---] [rr_sim] end 19:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:21:00 [---] [cpu_sched_debug] preliminary job list: 19:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:21:00 [---] [cpu_sched_debug] final job list: 19:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:21:00 [---] [cpu_sched_debug] enforce_run_list: end 19:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:22:00 [---] [rr_sim] doing sim: CPU sched 19:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972244101.65G/439.78G) 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210756.66 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 221075.67 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221075.665619 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989680.99 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 198968.10 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420043.764676 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790712.89 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 179071.29 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599115.053828 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611641.60 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 161164.16 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760279.214064 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450477.44 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 145047.74 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905326.958277 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305429.70 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 130542.97 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035869.928068 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174886.73 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 117488.67 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153358.600881 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057398.06 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 105739.81 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259098.406412 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951658.25 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 95165.82 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354264.231390 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856492.42 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 85649.24 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439913.473870 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770843.18 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 77084.32 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516997.792102 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693758.86 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 69375.89 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586373.678511 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624382.98 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 62438.30 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648811.976279 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561944.68 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 56194.47 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1705006.444270 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505750.21 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 50575.02 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755581.465462 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455175.19 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 45517.52 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801098.984535 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409657.67 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 40965.77 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842064.751701 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368691.90 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 36869.19 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878933.942150 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331822.71 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 33182.27 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912116.213554 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298640.44 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 29864.04 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941980.257818 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268776.40 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 26877.64 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968857.897655 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241898.76 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 24189.88 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1993047.773509 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217708.88 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 21770.89 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014818.661777 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195937.99 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 19593.80 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034412.461219 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176344.19 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 17634.42 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052046.880716 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158709.78 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 15870.98 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067917.858263 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142838.80 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 14283.88 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082201.738056 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128554.92 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 12855.49 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095057.229870 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115699.43 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 11569.94 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106627.172502 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104129.48 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 10412.95 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117040.120871 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93716.54 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 9371.65 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126411.774403 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84344.88 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 8434.49 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134846.262582 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75910.39 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 7591.04 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142437.301943 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68319.35 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 6831.94 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149269.237368 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61487.42 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 6148.74 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155417.979250 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55338.68 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 5533.87 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160951.846944 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49804.81 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 4980.48 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165932.327869 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44824.33 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 4482.43 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170414.760701 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40341.90 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 4034.19 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174448.950250 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36307.71 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3630.77 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178079.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32676.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181679.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29076.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185279.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25476.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188879.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21876.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192479.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18276.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196079.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14676.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199679.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11076.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203279.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7476.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206879.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3876.94 sec) 19:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210479.720844 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276.94 sec) 19:22:00 [GPUGRID] [rr_sim] 2210756.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972244101.65G/439.78G) 19:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853517.57 19:22:00 [---] pick_jobs_to_run() start 19:22:00 [---] pick_jobs_to_run() end 19:22:00 [---] [rr_sim] end 19:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:22:00 [---] [cpu_sched_debug] preliminary job list: 19:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:22:00 [---] [cpu_sched_debug] final job list: 19:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:22:00 [---] [cpu_sched_debug] enforce_run_list: end 19:22:00 [---] choose_project(): 69720.000000 19:22:00 [WUProp@Home] doing RPC to get work 19:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365879.09 cpu 2114312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114312.90 19:22:00 [---] Got 0 tasks 19:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:23:00 [---] [rr_sim] doing sim: CPU sched 19:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972216511.30G/439.78G) 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210693.92 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 221069.39 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221069.391931 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989624.53 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 198962.45 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420031.844669 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790662.07 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 179066.21 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599098.052133 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611595.87 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 161159.59 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760257.638850 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450436.28 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 145043.63 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905301.266896 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305392.65 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 130539.27 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035840.532138 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174853.39 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 117485.34 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153325.870855 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057368.05 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 105736.80 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259062.675700 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951631.24 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 95163.12 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354225.800061 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856468.12 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 85646.81 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439872.611986 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770821.31 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 77082.13 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516954.742718 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693739.18 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 69373.92 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586328.660377 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624365.26 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 62436.53 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648765.186271 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561928.73 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 56192.87 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704958.059574 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505735.86 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 50573.59 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755531.645548 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455162.27 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 45516.23 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1801047.872924 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409646.05 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 40964.60 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1842012.477563 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368681.44 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 36868.14 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878880.621737 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331813.30 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 33181.33 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912061.951494 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298631.97 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 29863.20 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941925.148276 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268768.77 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 26876.88 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968802.025379 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241891.89 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 24189.19 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992991.214772 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217702.70 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 21770.27 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014761.485226 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195932.43 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 19593.24 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034354.728634 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176339.19 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 17633.92 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051988.647702 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158705.27 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 15870.53 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067859.174863 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142834.74 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 14283.47 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082142.649307 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128551.27 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 12855.13 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094997.776307 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115696.14 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 11569.61 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106567.390608 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104126.53 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 10412.65 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116980.043478 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93713.88 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 9371.39 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126351.431061 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84342.49 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 8434.25 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134785.679886 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75908.24 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 7590.82 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142376.503828 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68317.42 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 6831.74 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149208.245376 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61485.67 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 6148.57 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155356.812770 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55337.11 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 5533.71 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160890.523424 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49803.40 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 4980.34 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165870.863012 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44823.06 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 4482.31 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170353.168642 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40340.75 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 4034.08 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174387.243709 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36306.68 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3630.67 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178017.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32676.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181617.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29076.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185217.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25476.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188817.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21876.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192417.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18276.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196017.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14676.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199617.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11076.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203217.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7476.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206817.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3876.01 sec) 19:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210417.911269 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 276.01 sec) 19:23:00 [GPUGRID] [rr_sim] 2210693.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972216511.30G/439.78G) 19:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853514.83 19:23:00 [---] pick_jobs_to_run() start 19:23:00 [---] pick_jobs_to_run() end 19:23:00 [---] [rr_sim] end 19:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:23:00 [---] [cpu_sched_debug] preliminary job list: 19:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:23:00 [---] [cpu_sched_debug] final job list: 19:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:23:00 [---] [cpu_sched_debug] enforce_run_list: end 19:23:00 [---] choose_project(): 69780.000000 19:23:00 [WUProp@Home] doing RPC to get work 19:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365819.09 cpu 2114252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114252.90 19:23:00 [---] Got 0 tasks 19:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:24:00 [---] [rr_sim] doing sim: CPU sched 19:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972188920.94G/439.78G) 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210631.18 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 221063.12 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221063.118243 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989568.06 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 198956.81 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420019.924661 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790611.26 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 179061.13 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599081.050438 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611550.13 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 161155.01 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760236.063637 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450395.12 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 145039.51 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905275.575516 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305355.61 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 130535.56 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035811.136207 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174820.05 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 117482.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153293.140829 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057338.04 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 105733.80 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1259026.944989 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951604.24 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 95160.42 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354187.368733 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856443.81 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 85644.38 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439831.750102 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770799.43 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 77079.94 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516911.693335 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693719.49 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 69371.95 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586283.642244 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624347.54 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 62434.75 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648718.396262 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561912.79 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 56191.28 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704909.674879 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505721.51 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 50572.15 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755481.825633 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455149.36 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 45514.94 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800996.761313 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409634.42 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 40963.44 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841960.203424 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368670.98 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 36867.10 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878827.301325 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331803.88 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 33180.39 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1912007.689435 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298623.49 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 29862.35 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941870.038734 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268761.14 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 26876.11 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968746.153103 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241885.03 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 24188.50 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992934.656036 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217696.53 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 21769.65 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014704.308675 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195926.87 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 19592.69 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034296.996050 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176334.19 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 17633.42 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051930.414688 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158700.77 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 15870.08 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067800.491462 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142830.69 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 14283.07 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082083.560558 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128547.62 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 12854.76 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094938.322745 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115692.86 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 11569.29 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106507.608713 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104123.57 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 10412.36 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116919.966085 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93711.22 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 9371.12 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126291.087719 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84340.09 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 8434.01 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134725.097190 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75906.09 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 7590.61 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142315.705714 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68315.48 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 6831.55 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149147.253385 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61483.93 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 6148.39 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155295.646289 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55335.54 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 5533.55 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160829.199903 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49801.98 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 4980.20 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165809.398155 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44821.78 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 4482.18 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170291.576583 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40339.61 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 4033.96 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174325.537167 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36305.65 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3630.56 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177956.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32675.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181556.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29075.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185156.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25475.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188756.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21875.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192356.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18275.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195956.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14675.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199556.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11075.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203156.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7475.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206756.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3875.08 sec) 19:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210356.101693 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 275.08 sec) 19:24:00 [GPUGRID] [rr_sim] 2210631.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972188920.94G/439.78G) 19:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853512.10 19:24:00 [---] pick_jobs_to_run() start 19:24:00 [---] pick_jobs_to_run() end 19:24:00 [---] [rr_sim] end 19:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:24:00 [---] [cpu_sched_debug] preliminary job list: 19:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:24:00 [---] [cpu_sched_debug] final job list: 19:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:24:00 [---] [cpu_sched_debug] enforce_run_list: end 19:24:00 [---] choose_project(): 69840.000000 19:24:00 [WUProp@Home] doing RPC to get work 19:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365759.09 cpu 2114192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114192.90 19:24:00 [---] Got 0 tasks 19:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:25:00 [---] [rr_sim] doing sim: CPU sched 19:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972161330.58G/439.78G) 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210568.45 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 221056.84 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221056.844554 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989511.60 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 198951.16 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 420008.004654 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790560.44 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 179056.04 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599064.048743 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611504.40 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 161150.44 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760214.488423 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450353.96 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 145035.40 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905249.884135 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305318.56 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 130531.86 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035781.740276 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174786.71 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 117478.67 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153260.410803 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057308.03 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 105730.80 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258991.214277 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951577.23 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 95157.72 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354148.937404 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856419.51 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 85641.95 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439790.888218 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770777.56 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 77077.76 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516868.643951 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693699.80 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 69369.98 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586238.624110 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624329.82 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 62432.98 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648671.606254 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561896.84 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 56189.68 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704861.290183 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505707.16 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 50570.72 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755432.005719 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455136.44 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 45513.64 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800945.649702 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409622.80 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 40962.28 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841907.929286 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368660.52 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 36866.05 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878773.980912 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331794.46 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 33179.45 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911953.427375 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298615.02 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 29861.50 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941814.929192 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268753.52 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 26875.35 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968690.280827 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241878.16 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 24187.82 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992878.097299 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217690.35 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 21769.03 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014647.132124 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195921.31 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 19592.13 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034239.263466 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176329.18 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 17632.92 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051872.181674 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158696.26 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 15869.63 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067741.808061 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142826.64 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 14282.66 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082024.471809 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128543.97 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 12854.40 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094878.869183 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115689.58 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 11568.96 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106447.826819 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104120.62 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 10412.06 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116859.888692 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93708.56 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 9370.86 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126230.744377 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84337.70 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 8433.77 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134664.514494 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75903.93 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 7590.39 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142254.907599 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68313.54 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 6831.35 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149086.261394 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61482.18 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 6148.22 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155234.479809 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55333.97 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 5533.40 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160767.876382 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49800.57 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 4980.06 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165747.933299 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44820.51 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 4482.05 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170229.984523 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40338.46 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 4033.85 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174263.830625 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36304.61 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3630.46 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177894.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32674.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181494.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29074.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185094.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25474.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188694.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21874.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192294.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18274.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195894.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14674.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199494.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11074.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203094.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7474.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206694.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3874.15 sec) 19:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210294.292117 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 274.15 sec) 19:25:00 [GPUGRID] [rr_sim] 2210568.45: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972161330.58G/439.78G) 19:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853509.36 19:25:00 [---] pick_jobs_to_run() start 19:25:00 [---] pick_jobs_to_run() end 19:25:00 [---] [rr_sim] end 19:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:25:00 [---] [cpu_sched_debug] preliminary job list: 19:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:25:00 [---] [cpu_sched_debug] final job list: 19:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:25:00 [---] [cpu_sched_debug] enforce_run_list: end 19:25:00 [---] choose_project(): 69900.000000 19:25:00 [WUProp@Home] doing RPC to get work 19:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365699.09 cpu 2114132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114132.90 19:25:00 [---] Got 0 tasks 19:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:26:00 [---] [rr_sim] doing sim: CPU sched 19:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972133740.23G/439.78G) 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210505.71 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 221050.57 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221050.570866 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989455.14 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 198945.51 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419996.084646 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790509.62 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 179050.96 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599047.047048 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611458.66 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 161145.87 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760192.913209 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450312.80 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 145031.28 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905224.192755 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305281.52 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 130528.15 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035752.344345 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174753.36 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 117475.34 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153227.680777 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057278.03 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 105727.80 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258955.483566 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951550.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 95155.02 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354110.506075 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856395.20 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 85639.52 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439750.026334 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770755.68 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 77075.57 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516825.594567 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693680.11 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 69368.01 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586193.605977 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624312.10 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 62431.21 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648624.816245 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561880.89 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 56188.09 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704812.905487 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505692.80 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 50569.28 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755382.185805 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455123.52 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 45512.35 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800894.538090 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409611.17 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 40961.12 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841855.655148 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368650.05 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 36865.01 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878720.660499 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331785.05 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 33178.50 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911899.165316 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298606.54 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 29860.65 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941759.819650 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268745.89 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 26874.59 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968634.408552 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241871.30 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 24187.13 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992821.538563 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217684.17 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 21768.42 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014589.955573 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195915.75 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 19591.58 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034181.530882 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176324.18 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 17632.42 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051813.948660 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158691.76 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 15869.18 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067683.124660 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142822.58 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 14282.26 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081965.383060 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128540.33 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 12854.03 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094819.415621 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115686.29 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 11568.63 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106388.044925 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104117.66 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 10411.77 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116799.811299 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93705.90 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 9370.59 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126170.401035 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84335.31 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 8433.53 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134603.931798 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75901.78 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 7590.18 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142194.109484 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68311.60 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 6831.16 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149025.269402 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61480.44 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 6148.04 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155173.313328 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55332.40 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 5533.24 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160706.552862 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49799.16 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 4979.92 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165686.468442 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44819.24 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 4481.92 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170168.392464 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40337.32 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 4033.73 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174202.124084 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36303.58 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3630.36 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177832.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32673.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181432.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29073.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185032.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25473.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188632.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21873.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192232.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18273.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195832.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14673.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199432.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11073.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203032.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7473.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206632.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3873.23 sec) 19:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210232.482542 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 273.23 sec) 19:26:00 [GPUGRID] [rr_sim] 2210505.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972133740.23G/439.78G) 19:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853506.62 19:26:00 [---] pick_jobs_to_run() start 19:26:00 [---] pick_jobs_to_run() end 19:26:00 [---] [rr_sim] end 19:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:26:00 [---] [cpu_sched_debug] preliminary job list: 19:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:26:00 [---] [cpu_sched_debug] final job list: 19:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:26:00 [---] [cpu_sched_debug] enforce_run_list: end 19:26:00 [---] choose_project(): 69960.000000 19:26:00 [WUProp@Home] doing RPC to get work 19:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365639.09 cpu 2114072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114072.90 19:26:00 [---] Got 0 tasks 19:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:27:00 [---] [rr_sim] doing sim: CPU sched 19:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972106149.87G/439.78G) 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210442.97 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 221044.30 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221044.297178 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989398.67 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 198939.87 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419984.164638 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790458.81 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 179045.88 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599030.045353 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611412.93 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 161141.29 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760171.337995 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450271.63 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 145027.16 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905198.501374 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305244.47 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 130524.45 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035722.948415 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174720.02 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 117472.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153194.950751 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057248.02 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 105724.80 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258919.752854 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951523.22 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 95152.32 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354072.074747 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856370.90 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 85637.09 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439709.164450 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770733.81 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 77073.38 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516782.545183 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693660.43 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 69366.04 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586148.587843 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624294.38 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 62429.44 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648578.026237 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561864.95 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 56186.49 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704764.520791 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505678.45 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 50567.85 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755332.365890 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455110.61 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 45511.06 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800843.426479 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409599.55 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 40959.95 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841803.381009 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368639.59 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 36863.96 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878667.340086 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331775.63 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 33177.56 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911844.903256 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298598.07 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 29859.81 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941704.710108 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268738.26 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 26873.83 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968578.536276 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241864.44 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 24186.44 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992764.979826 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217677.99 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 21767.80 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014532.779022 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195910.19 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 19591.02 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034123.798297 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176319.17 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 17631.92 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051755.715646 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158687.26 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 15868.73 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067624.441259 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142818.53 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 14281.85 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081906.294311 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128536.68 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 12853.67 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094759.962058 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115683.01 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 11568.30 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106328.263031 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104114.71 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 10411.47 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116739.733906 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93703.24 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 9370.32 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126110.057693 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84332.91 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 8433.29 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134543.349102 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75899.62 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 7589.96 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142133.311370 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68309.66 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 6830.97 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148964.277411 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61478.69 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 6147.87 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155112.146848 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55330.82 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 5533.08 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160645.229341 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49797.74 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 4979.77 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165625.003585 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44817.97 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 4481.80 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170106.800405 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40336.17 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 4033.62 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174140.417542 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36302.55 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3630.26 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177770.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32672.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181370.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29072.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184970.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25472.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188570.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21872.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192170.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18272.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195770.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14672.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199370.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11072.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202970.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7472.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206570.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3872.30 sec) 19:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210170.672966 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 272.30 sec) 19:27:00 [GPUGRID] [rr_sim] 2210442.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972106149.87G/439.78G) 19:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853503.88 19:27:00 [---] pick_jobs_to_run() start 19:27:00 [---] pick_jobs_to_run() end 19:27:00 [---] [rr_sim] end 19:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:27:00 [---] [cpu_sched_debug] preliminary job list: 19:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:27:00 [---] [cpu_sched_debug] final job list: 19:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:27:00 [---] [cpu_sched_debug] enforce_run_list: end 19:27:00 [---] choose_project(): 70020.000000 19:27:00 [WUProp@Home] doing RPC to get work 19:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365579.09 cpu 2114012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2114012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2114012.90 19:27:00 [---] Got 0 tasks 19:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:28:00 [---] [rr_sim] doing sim: CPU sched 19:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972078559.51G/439.78G) 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210380.23 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 221038.02 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221038.023490 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989342.21 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 198934.22 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419972.244631 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790407.99 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 179040.80 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 599013.043657 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611367.19 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 161136.72 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760149.762782 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450230.47 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 145023.05 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905172.809993 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305207.42 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 130520.74 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035693.552484 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174686.68 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 117468.67 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153162.220725 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057218.01 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 105721.80 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258884.022143 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951496.21 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 95149.62 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1354033.643418 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856346.59 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 85634.66 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439668.302566 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770711.93 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 77071.19 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516739.495799 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693640.74 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 69364.07 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586103.569709 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624276.67 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 62427.67 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648531.236228 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561849.00 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 56184.90 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704716.136095 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505664.10 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 50566.41 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755282.545976 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455097.69 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 45509.77 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800792.314868 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409587.92 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 40958.79 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841751.106871 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368629.13 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 36862.91 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878614.019674 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331766.22 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 33176.62 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911790.641196 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298589.59 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 29858.96 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941649.600566 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268730.63 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 26873.06 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968522.664000 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241857.57 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 24185.76 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992708.421090 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217671.81 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 21767.18 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014475.602470 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195904.63 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 19590.46 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034066.065713 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176314.17 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 17631.42 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051697.482632 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158682.75 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 15868.28 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067565.757858 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142814.48 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 14281.45 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081847.205562 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128533.03 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 12853.30 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094700.508496 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115679.73 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 11567.97 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106268.481136 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104111.75 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 10411.18 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116679.656513 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93700.58 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 9370.06 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126049.714351 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84330.52 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 8433.05 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134482.766406 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75897.47 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 7589.75 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142072.513255 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68307.72 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 6830.77 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148903.285419 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61476.95 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 6147.69 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155050.980367 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55329.25 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 5532.93 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160583.905820 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49796.33 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 4979.63 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165563.538728 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44816.70 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 4481.67 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170045.208345 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40335.03 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 4033.50 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174078.711001 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36301.52 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3630.15 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177708.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32671.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181308.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29071.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184908.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25471.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188508.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21871.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192108.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18271.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195708.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14671.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199308.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11071.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202908.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7471.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206508.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3871.37 sec) 19:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210108.863390 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 271.37 sec) 19:28:00 [GPUGRID] [rr_sim] 2210380.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972078559.51G/439.78G) 19:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853501.15 19:28:00 [---] pick_jobs_to_run() start 19:28:00 [---] pick_jobs_to_run() end 19:28:00 [---] [rr_sim] end 19:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:28:00 [---] [cpu_sched_debug] preliminary job list: 19:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:28:00 [---] [cpu_sched_debug] final job list: 19:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:28:00 [---] [cpu_sched_debug] enforce_run_list: end 19:28:00 [---] choose_project(): 70080.000000 19:28:00 [WUProp@Home] doing RPC to get work 19:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365519.09 cpu 2113952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113952.90 19:28:00 [---] Got 0 tasks 19:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:29:00 [---] [rr_sim] doing sim: CPU sched 19:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972050969.16G/439.78G) 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210317.50 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 221031.75 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221031.749802 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989285.75 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 198928.57 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419960.324623 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790357.17 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 179035.72 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598996.041962 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611321.46 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 161132.15 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760128.187568 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450189.31 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 145018.93 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905147.118613 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305170.38 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 130517.04 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035664.156553 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174653.34 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 117465.33 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153129.490699 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057188.01 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 105718.80 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258848.291431 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951469.21 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 95146.92 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353995.212090 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856322.29 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 85632.23 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439627.440682 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770690.06 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 77069.01 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516696.446416 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693621.05 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 69362.11 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586058.551576 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624258.95 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 62425.89 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648484.446220 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561833.05 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 56183.31 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704667.751399 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505649.75 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 50564.97 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755232.726061 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455084.77 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 45508.48 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800741.203257 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409576.29 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 40957.63 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841698.832733 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368618.67 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 36861.87 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878560.699261 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331756.80 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 33175.68 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911736.379137 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298581.12 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 29858.11 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941594.491025 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268723.01 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 26872.30 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968466.791724 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241850.71 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 24185.07 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992651.862353 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217665.64 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 21766.56 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014418.425919 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195899.07 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 19589.91 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034008.333129 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176309.16 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 17630.92 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051639.249618 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158678.25 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 15867.82 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067507.074458 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142810.42 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 14281.04 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081788.116813 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128529.38 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 12852.94 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094641.054934 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115676.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 11567.64 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106208.699242 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104108.80 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 10410.88 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116619.579119 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93697.92 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 9369.79 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125989.371009 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84328.13 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 8432.81 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134422.183710 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75895.31 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 7589.53 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142011.715141 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68305.78 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 6830.58 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148842.293428 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61475.20 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 6147.52 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154989.813887 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55327.68 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 5532.77 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160522.582300 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49794.92 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 4979.49 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165502.073871 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44815.42 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 4481.54 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169983.616286 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40333.88 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 4033.39 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174017.004459 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36300.49 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3630.05 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177647.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32670.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181247.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29070.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184847.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25470.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188447.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21870.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192047.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18270.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195647.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14670.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199247.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11070.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202847.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7470.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206447.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3870.44 sec) 19:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2210047.053815 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 270.44 sec) 19:29:00 [GPUGRID] [rr_sim] 2210317.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972050969.16G/439.78G) 19:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853498.41 19:29:00 [---] pick_jobs_to_run() start 19:29:00 [---] pick_jobs_to_run() end 19:29:00 [---] [rr_sim] end 19:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:29:00 [---] [cpu_sched_debug] preliminary job list: 19:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:29:00 [---] [cpu_sched_debug] final job list: 19:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:29:00 [---] [cpu_sched_debug] enforce_run_list: end 19:29:00 [---] choose_project(): 70140.000000 19:29:00 [WUProp@Home] doing RPC to get work 19:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365459.09 cpu 2113892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113892.90 19:29:00 [---] Got 0 tasks 19:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:30:00 [---] [rr_sim] doing sim: CPU sched 19:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (972023378.80G/439.78G) 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210254.76 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 221025.48 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221025.476113 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989229.29 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 198922.93 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419948.404615 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790306.36 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 179030.64 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598979.040267 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611275.72 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 161127.57 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760106.612354 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450148.15 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 145014.81 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905121.427232 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305133.33 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 130513.33 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035634.760622 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174620.00 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 117462.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153096.760673 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057158.00 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 105715.80 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258812.560720 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951442.20 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 95144.22 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353956.780761 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856297.98 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 85629.80 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439586.578798 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770668.18 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 77066.82 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516653.397032 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693601.36 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 69360.14 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1586013.533442 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624241.23 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 62424.12 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648437.656211 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561817.10 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 56181.71 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704619.366704 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505635.39 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 50563.54 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755182.906147 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455071.85 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 45507.19 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800690.091645 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409564.67 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 40956.47 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841646.558594 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368608.20 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 36860.82 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878507.378848 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331747.38 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 33174.74 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911682.117077 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298572.64 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 29857.26 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941539.381483 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268715.38 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 26871.54 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968410.919448 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241843.84 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 24184.38 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992595.303616 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217659.46 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 21765.95 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014361.249368 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195893.51 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 19589.35 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033950.600545 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176304.16 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 17630.42 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051581.016604 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158673.74 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 15867.37 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067448.391057 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142806.37 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 14280.64 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081729.028065 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128525.73 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 12852.57 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094581.601371 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115673.16 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 11567.32 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106148.917348 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104105.84 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 10410.58 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116559.501726 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93695.26 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 9369.53 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125929.027667 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84325.73 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 8432.57 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134361.601014 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75893.16 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 7589.32 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141950.917026 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68303.84 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 6830.38 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148781.301437 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61473.46 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 6147.35 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154928.647406 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55326.11 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 5532.61 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160461.258779 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49793.50 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 4979.35 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165440.609015 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44814.15 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 4481.42 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169922.024227 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40332.74 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 4033.27 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173955.297917 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36299.46 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3629.95 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177585.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32669.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181185.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29069.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184785.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25469.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188385.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21869.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191985.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18269.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195585.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14669.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199185.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11069.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202785.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7469.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206385.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3869.52 sec) 19:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209985.244239 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 269.52 sec) 19:30:00 [GPUGRID] [rr_sim] 2210254.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (972023378.80G/439.78G) 19:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853495.67 19:30:00 [---] pick_jobs_to_run() start 19:30:00 [---] pick_jobs_to_run() end 19:30:00 [---] [rr_sim] end 19:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:30:00 [---] [cpu_sched_debug] preliminary job list: 19:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:30:00 [---] [cpu_sched_debug] final job list: 19:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:30:00 [---] [cpu_sched_debug] enforce_run_list: end 19:30:00 [---] choose_project(): 70200.000000 19:30:00 [WUProp@Home] doing RPC to get work 19:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365399.09 cpu 2113832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113832.90 19:30:00 [---] Got 0 tasks 19:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:33:00 [---] [rr_sim] doing sim: CPU sched 19:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971995788.45G/439.78G) 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210192.02 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 221019.20 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221019.202425 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989172.82 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 198917.28 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419936.484608 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790255.54 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 179025.55 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598962.038572 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611229.99 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 161123.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760085.037140 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450106.99 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 145010.70 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905095.735851 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305096.29 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 130509.63 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035605.364691 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174586.66 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 117458.67 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153064.030648 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057127.99 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 105712.80 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258776.830008 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951415.19 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 95141.52 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353918.349432 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856273.67 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 85627.37 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439545.716914 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770646.31 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 77064.63 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516610.347648 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693581.68 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 69358.17 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585968.515309 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624223.51 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 62422.35 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648390.866203 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561801.16 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 56180.12 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704570.982008 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505621.04 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 50562.10 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755133.086232 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455058.94 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 45505.89 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800638.980034 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409553.04 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 40955.30 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841594.284456 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368597.74 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 36859.77 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878454.058436 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331737.97 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 33173.80 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911627.855017 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298564.17 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 29856.42 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941484.271941 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268707.75 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 26870.78 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968355.047172 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241836.98 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 24183.70 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992538.744880 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217653.28 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 21765.33 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014304.072817 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195887.95 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 19588.80 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033892.867961 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176299.16 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 17629.92 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051522.783590 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158669.24 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 15866.92 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067389.707656 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142802.32 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 14280.23 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081669.939316 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128522.08 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 12852.21 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094522.147809 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115669.88 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 11566.99 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106089.135453 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104102.89 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 10410.29 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116499.424333 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93692.60 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 9369.26 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125868.684325 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84323.34 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 8432.33 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134301.018318 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75891.01 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 7589.10 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141890.118911 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68301.91 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 6830.19 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148720.309445 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61471.71 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 6147.17 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154867.480926 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55324.54 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 5532.45 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160399.935259 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49792.09 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 4979.21 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165379.144158 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44812.88 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 4481.29 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169860.432167 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40331.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 4033.16 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173893.591376 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36298.43 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3629.84 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177523.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32668.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181123.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29068.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184723.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25468.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188323.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21868.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191923.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18268.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195523.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14668.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199123.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11068.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202723.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7468.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206323.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3868.59 sec) 19:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209923.434663 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268.59 sec) 19:33:00 [GPUGRID] [rr_sim] 2210192.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971995788.45G/439.78G) 19:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853612.94 19:33:00 [---] pick_jobs_to_run() start 19:33:00 [---] pick_jobs_to_run() end 19:33:00 [---] [rr_sim] end 19:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:33:00 [---] [cpu_sched_debug] preliminary job list: 19:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:33:00 [---] [cpu_sched_debug] final job list: 19:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:33:00 [---] [cpu_sched_debug] enforce_run_list: end 19:33:00 [---] choose_project(): 70380.000000 19:33:00 [WUProp@Home] doing RPC to get work 19:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365219.09 cpu 2113772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113772.90 19:33:00 [---] Got 0 tasks 19:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:34:00 [---] [rr_sim] doing sim: CPU sched 19:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971968198.09G/439.78G) 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210129.29 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 221012.93 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221012.928737 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989116.36 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 198911.64 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419924.564600 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790204.72 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 179020.47 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598945.036877 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611184.25 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 161118.43 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760063.461926 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450065.83 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 145006.58 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905070.044471 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305059.24 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 130505.92 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035575.968761 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174553.32 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 117455.33 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1153031.300622 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057097.99 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 105709.80 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258741.099296 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951388.19 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 95138.82 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353879.918104 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856249.37 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 85624.94 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439504.855030 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770624.43 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 77062.44 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516567.298264 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693561.99 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 69356.20 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585923.497175 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624205.79 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 62420.58 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648344.076194 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561785.21 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 56178.52 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704522.597312 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505606.69 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 50560.67 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755083.266318 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455046.02 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 45504.60 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800587.868423 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409541.42 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 40954.14 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841542.010318 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368587.28 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 36858.73 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878400.738023 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331728.55 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 33172.85 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911573.592958 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298555.69 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 29855.57 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941429.162399 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268700.12 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 26870.01 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968299.174896 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241830.11 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 24183.01 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992482.186143 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217647.10 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 21764.71 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014246.896266 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195882.39 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 19588.24 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033835.135376 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176294.15 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 17629.42 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051464.550576 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158664.74 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 15866.47 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067331.024255 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142798.26 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 14279.83 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081610.850567 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128518.44 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 12851.84 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094462.694247 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115666.59 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 11566.66 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106029.353559 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104099.93 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 10409.99 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116439.346940 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93689.94 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 9368.99 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125808.340983 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84320.95 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 8432.09 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134240.435622 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75888.85 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 7588.89 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141829.320797 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68299.97 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 6830.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148659.317454 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61469.97 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 6147.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154806.314446 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55322.97 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 5532.30 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160338.611738 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49790.68 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 4979.07 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165317.679301 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44811.61 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 4481.16 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169798.840108 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40330.45 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 4033.04 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173831.884834 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36297.40 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3629.74 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177461.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32667.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181061.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29067.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184661.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25467.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188261.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21867.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191861.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18267.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195461.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14667.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199061.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11067.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202661.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7467.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206261.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3867.66 sec) 19:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209861.625088 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267.66 sec) 19:34:00 [GPUGRID] [rr_sim] 2210129.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971968198.09G/439.78G) 19:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853610.20 19:34:00 [---] pick_jobs_to_run() start 19:34:00 [---] pick_jobs_to_run() end 19:34:00 [---] [rr_sim] end 19:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:34:00 [---] [cpu_sched_debug] preliminary job list: 19:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:34:00 [---] [cpu_sched_debug] final job list: 19:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:34:00 [---] [cpu_sched_debug] enforce_run_list: end 19:34:00 [---] choose_project(): 70440.000000 19:34:00 [WUProp@Home] doing RPC to get work 19:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365159.09 cpu 2113712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113712.90 19:34:00 [---] Got 0 tasks 19:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:35:00 [---] [rr_sim] doing sim: CPU sched 19:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971940607.73G/439.78G) 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210066.55 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 221006.66 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221006.655049 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989059.90 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 198905.99 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419912.644593 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790153.91 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 179015.39 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598928.035182 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611138.52 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 161113.85 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760041.886713 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1450024.66 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 145002.47 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905044.353090 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1305022.20 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 130502.22 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035546.572830 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174519.98 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 117452.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152998.570596 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057067.98 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 105706.80 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258705.368585 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951361.18 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 95136.12 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353841.486775 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856225.06 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 85622.51 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439463.993146 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770602.56 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 77060.26 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516524.248881 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693542.30 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 69354.23 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585878.479041 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624188.07 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 62418.81 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648297.286186 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561769.26 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 56176.93 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704474.212616 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505592.34 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 50559.23 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1755033.446403 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455033.10 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 45503.31 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800536.756812 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409529.79 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 40952.98 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841489.736179 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368576.81 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 36857.68 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878347.417610 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331719.13 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 33171.91 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911519.330898 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298547.22 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 29854.72 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941374.052857 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268692.50 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 26869.25 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968243.302620 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241823.25 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 24182.32 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992425.627407 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217640.92 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 21764.09 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014189.719715 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195876.83 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 19587.68 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033777.402792 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176289.15 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 17628.91 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051406.317562 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158660.23 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 15866.02 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067272.340854 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142794.21 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 14279.42 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081551.761818 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128514.79 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 12851.48 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094403.240685 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115663.31 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 11566.33 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105969.571665 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104096.98 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 10409.70 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116379.269547 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93687.28 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 9368.73 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125747.997641 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84318.55 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 8431.86 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134179.852926 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75886.70 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 7588.67 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141768.522682 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68298.03 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 6829.80 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148598.325463 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61468.23 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 6146.82 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154745.147965 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55321.40 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 5532.14 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160277.288217 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49789.26 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 4978.93 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165256.214444 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44810.34 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 4481.03 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169737.248049 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40329.30 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 4032.93 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173770.178293 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36296.37 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3629.64 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177399.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32666.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180999.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29066.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184599.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25466.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188199.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21866.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191799.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18266.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195399.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14666.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198999.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11066.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202599.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7466.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206199.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3866.73 sec) 19:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209799.815512 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266.73 sec) 19:35:00 [GPUGRID] [rr_sim] 2210066.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971940607.73G/439.78G) 19:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853607.46 19:35:00 [---] pick_jobs_to_run() start 19:35:00 [---] pick_jobs_to_run() end 19:35:00 [---] [rr_sim] end 19:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:35:00 [---] [cpu_sched_debug] preliminary job list: 19:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:35:00 [---] [cpu_sched_debug] final job list: 19:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:35:00 [---] [cpu_sched_debug] enforce_run_list: end 19:35:00 [---] choose_project(): 70500.000000 19:35:00 [WUProp@Home] doing RPC to get work 19:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365099.09 cpu 2113652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113652.90 19:35:00 [---] Got 0 tasks 19:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:36:00 [---] [rr_sim] doing sim: CPU sched 19:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971913017.38G/439.78G) 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2210003.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 221000.38 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 221000.381361 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1989003.43 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 198900.34 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419900.724585 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790103.09 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 179010.31 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598911.033487 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611092.78 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 161109.28 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 760020.311499 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449983.50 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 144998.35 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 905018.661710 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304985.15 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 130498.52 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035517.176899 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174486.64 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 117448.66 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152965.840570 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057037.97 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 105703.80 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258669.637873 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951334.18 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 95133.42 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353803.055447 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856200.76 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 85620.08 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439423.131262 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770580.68 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 77058.07 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516481.199497 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693522.61 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 69352.26 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585833.460908 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624170.35 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 62417.04 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648250.496177 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561753.32 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 56175.33 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704425.827920 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505577.99 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 50557.80 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754983.626489 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455020.19 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 45502.02 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800485.645200 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409518.17 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 40951.82 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841437.462041 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368566.35 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 36856.64 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878294.097197 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331709.72 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 33170.97 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911465.068838 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298538.74 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 29853.87 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941318.943315 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268684.87 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 26868.49 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968187.430344 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241816.38 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 24181.64 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992369.068670 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217634.74 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 21763.47 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014132.543164 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195871.27 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 19587.13 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033719.670208 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176284.14 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 17628.41 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051348.084548 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158655.73 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 15865.57 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067213.657453 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142790.16 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 14279.02 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081492.673069 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128511.14 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 12851.11 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094343.787122 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115660.03 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 11566.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105909.789771 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104094.02 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 10409.40 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116319.192154 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93684.62 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 9368.46 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125687.654299 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84316.16 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 8431.62 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134119.270230 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75884.54 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 7588.45 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141707.724567 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68296.09 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 6829.61 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148537.333471 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61466.48 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 6146.65 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154683.981485 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55319.83 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 5531.98 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160215.964697 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49787.85 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 4978.78 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165194.749588 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44809.06 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 4480.91 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169675.655989 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40328.16 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 4032.82 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173708.471751 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36295.34 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3629.53 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177338.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32665.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180938.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29065.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184538.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25465.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188138.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21865.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191738.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18265.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195338.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14665.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198938.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11065.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202538.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7465.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206138.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3865.81 sec) 19:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209738.005936 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 265.81 sec) 19:36:00 [GPUGRID] [rr_sim] 2210003.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971913017.38G/439.78G) 19:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853604.73 19:36:00 [---] pick_jobs_to_run() start 19:36:00 [---] pick_jobs_to_run() end 19:36:00 [---] [rr_sim] end 19:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:36:00 [---] [cpu_sched_debug] preliminary job list: 19:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:36:00 [---] [cpu_sched_debug] final job list: 19:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:36:00 [---] [cpu_sched_debug] enforce_run_list: end 19:36:00 [---] choose_project(): 70560.000000 19:36:00 [WUProp@Home] doing RPC to get work 19:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 365039.09 cpu 2113592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113592.90 19:36:00 [---] Got 0 tasks 19:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:37:00 [---] [rr_sim] doing sim: CPU sched 19:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971885427.02G/439.78G) 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209941.08 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 220994.11 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220994.107672 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988946.97 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 198894.70 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419888.804577 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790052.27 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 179005.23 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598894.031792 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611047.04 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 161104.70 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759998.736285 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449942.34 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 144994.23 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904992.970329 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304948.11 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 130494.81 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035487.780968 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174453.30 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 117445.33 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152933.110544 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1057007.97 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 105700.80 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258633.907162 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951307.17 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 95130.72 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353764.624118 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856176.45 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 85617.65 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439382.269379 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770558.81 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 77055.88 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516438.150113 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693502.93 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 69350.29 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585788.442774 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624152.63 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 62415.26 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648203.706169 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561737.37 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 56173.74 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704377.443224 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505563.63 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 50556.36 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754933.806574 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 455007.27 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 45500.73 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800434.533589 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409506.54 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 40950.65 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841385.187903 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368555.89 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 36855.59 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878240.776785 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331700.30 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 33170.03 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911410.806779 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298530.27 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 29853.03 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941263.833773 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268677.24 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 26867.72 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968131.558068 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241809.52 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 24180.95 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992312.509934 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217628.57 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 21762.86 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014075.366613 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195865.71 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 19586.57 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033661.937624 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176279.14 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 17627.91 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051289.851534 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158651.23 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 15865.12 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067154.974053 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142786.10 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 14278.61 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081433.584320 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128507.49 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 12850.75 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094284.333560 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115656.74 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 11565.67 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105850.007876 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104091.07 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 10409.11 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116259.114761 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93681.96 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 9368.20 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125627.310957 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84313.77 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 8431.38 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134058.687534 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75882.39 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 7588.24 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141646.926453 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68294.15 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 6829.42 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148476.341480 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61464.74 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 6146.47 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154622.815004 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55318.26 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 5531.83 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160154.641176 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49786.44 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 4978.64 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165133.284731 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44807.79 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 4480.78 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169614.063930 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40327.01 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 4032.70 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173646.765209 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36294.31 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3629.43 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177276.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32664.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180876.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29064.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184476.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25464.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188076.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21864.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191676.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18264.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195276.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14664.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198876.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11064.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202476.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7464.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206076.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3864.88 sec) 19:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209676.196361 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 264.88 sec) 19:37:00 [GPUGRID] [rr_sim] 2209941.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971885427.02G/439.78G) 19:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853601.99 19:37:00 [---] pick_jobs_to_run() start 19:37:00 [---] pick_jobs_to_run() end 19:37:00 [---] [rr_sim] end 19:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:37:00 [---] [cpu_sched_debug] preliminary job list: 19:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:37:00 [---] [cpu_sched_debug] final job list: 19:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:37:00 [---] [cpu_sched_debug] enforce_run_list: end 19:37:00 [---] choose_project(): 70620.000000 19:37:00 [WUProp@Home] doing RPC to get work 19:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364979.09 cpu 2113532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113532.90 19:37:00 [---] Got 0 tasks 19:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:38:00 [---] [rr_sim] doing sim: CPU sched 19:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971857836.66G/439.78G) 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209878.34 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 220987.83 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220987.833984 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988890.51 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 198889.05 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419876.884570 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1790001.46 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 179000.15 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598877.030097 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1611001.31 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 161100.13 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759977.161071 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449901.18 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 144990.12 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904967.278948 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304911.06 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 130491.11 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035458.385038 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174419.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 117442.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152900.380518 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056977.96 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 105697.80 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258598.176450 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951280.16 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 95128.02 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353726.192789 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856152.15 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 85615.21 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439341.407495 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770536.93 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 77053.69 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516395.100729 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693483.24 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 69348.32 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585743.424640 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624134.92 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 62413.49 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648156.916161 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561721.42 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 56172.14 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704329.058529 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505549.28 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 50554.93 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754883.986660 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454994.35 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 45499.44 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800383.421978 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409494.92 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 40949.49 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841332.913764 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368545.43 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 36854.54 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878187.456372 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331690.88 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 33169.09 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911356.544719 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298521.80 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 29852.18 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941208.724231 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268669.62 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 26866.96 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968075.685792 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241802.65 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 24180.27 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992255.951197 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217622.39 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 21762.24 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2014018.190061 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195860.15 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 19586.01 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033604.205039 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176274.13 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 17627.41 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051231.618520 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158646.72 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 15864.67 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067096.290652 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142782.05 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 14278.20 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081374.495571 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128503.84 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 12850.38 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094224.879998 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115653.46 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 11565.35 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105790.225982 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104088.11 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 10408.81 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116199.037368 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93679.30 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 9367.93 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125566.967615 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84311.37 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 8431.14 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133998.104838 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75880.24 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 7588.02 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141586.128338 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68292.21 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 6829.22 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148415.349489 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61462.99 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 6146.30 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154561.648524 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55316.69 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 5531.67 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160093.317656 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49785.02 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 4978.50 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165071.819874 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44806.52 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 4480.65 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169552.471871 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40325.87 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 4032.59 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173585.058668 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36293.28 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3629.33 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177214.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32663.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180814.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29063.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184414.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25463.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188014.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21863.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191614.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18263.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195214.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14663.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198814.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11063.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202414.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7463.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206014.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3863.95 sec) 19:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209614.386785 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 263.95 sec) 19:38:00 [GPUGRID] [rr_sim] 2209878.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971857836.66G/439.78G) 19:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853599.25 19:38:00 [---] pick_jobs_to_run() start 19:38:00 [---] pick_jobs_to_run() end 19:38:00 [---] [rr_sim] end 19:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:38:00 [---] [cpu_sched_debug] preliminary job list: 19:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:38:00 [---] [cpu_sched_debug] final job list: 19:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:38:00 [---] [cpu_sched_debug] enforce_run_list: end 19:38:00 [---] choose_project(): 70680.000000 19:38:00 [WUProp@Home] doing RPC to get work 19:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364919.09 cpu 2113472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113472.90 19:38:00 [---] Got 0 tasks 19:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:39:00 [---] [rr_sim] doing sim: CPU sched 19:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971830246.31G/439.78G) 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209815.60 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 220981.56 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220981.560296 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988834.04 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 198883.40 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419864.964562 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789950.64 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 178995.06 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598860.028402 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610955.57 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 161095.56 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759955.585858 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449860.02 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 144986.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904941.587568 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304874.02 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 130487.40 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035428.989107 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174386.61 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 117438.66 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152867.650492 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056947.95 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 105694.80 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258562.445739 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951253.16 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 95125.32 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353687.761461 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856127.84 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 85612.78 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439300.545611 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770515.06 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 77051.51 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516352.051345 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693463.55 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 69346.36 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585698.406507 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624117.20 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 62411.72 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648110.126152 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561705.48 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 56170.55 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704280.673833 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505534.93 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 50553.49 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754834.166745 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454981.44 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 45498.14 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800332.310367 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409483.29 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 40948.33 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841280.639626 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368534.96 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 36853.50 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878134.135959 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331681.47 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 33168.15 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911302.282659 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298513.32 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 29851.33 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941153.614689 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268661.99 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 26866.20 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1968019.813516 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241795.79 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 24179.58 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992199.392460 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217616.21 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 21761.62 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013961.013510 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195854.59 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 19585.46 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033546.472455 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176269.13 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 17626.91 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051173.385506 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158642.22 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 15864.22 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067037.607251 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142778.00 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 14277.80 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081315.406822 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128500.20 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 12850.02 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094165.426435 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115650.18 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 11565.02 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105730.444088 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104085.16 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 10408.52 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116138.959975 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93676.64 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 9367.66 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125506.624273 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84308.98 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 8430.90 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133937.522142 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75878.08 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 7587.81 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141525.330224 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68290.27 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 6829.03 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148354.357497 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61461.25 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 6146.12 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154500.482043 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55315.12 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 5531.51 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160031.994135 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49783.61 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 4978.36 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165010.355017 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44805.25 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 4480.52 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169490.879811 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40324.72 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 4032.47 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173523.352126 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36292.25 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3629.23 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177152.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32663.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180752.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29063.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184352.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25463.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187952.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21863.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191552.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18263.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195152.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14663.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198752.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11063.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202352.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7463.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205952.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3863.03 sec) 19:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209552.577210 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 263.03 sec) 19:39:00 [GPUGRID] [rr_sim] 2209815.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971830246.31G/439.78G) 19:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853596.52 19:39:00 [---] pick_jobs_to_run() start 19:39:00 [---] pick_jobs_to_run() end 19:39:00 [---] [rr_sim] end 19:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:39:00 [---] [cpu_sched_debug] preliminary job list: 19:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:39:00 [---] [cpu_sched_debug] final job list: 19:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:39:00 [---] [cpu_sched_debug] enforce_run_list: end 19:39:00 [---] choose_project(): 70740.000000 19:39:00 [WUProp@Home] doing RPC to get work 19:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364859.09 cpu 2113412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113412.90 19:39:00 [---] Got 0 tasks 19:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:40:00 [---] [rr_sim] doing sim: CPU sched 19:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971802655.95G/439.78G) 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209752.87 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 220975.29 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220975.286608 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988777.58 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 198877.76 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419853.044555 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789899.82 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 178989.98 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598843.026707 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610909.84 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 161090.98 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759934.010644 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449818.86 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 144981.89 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904915.896187 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304836.97 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 130483.70 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035399.593176 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174353.27 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 117435.33 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152834.920466 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056917.95 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 105691.79 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258526.715027 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951226.15 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 95122.62 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353649.330132 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856103.54 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 85610.35 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439259.683727 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770493.18 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 77049.32 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516309.001962 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693443.86 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 69344.39 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585653.388373 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624099.48 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 62409.95 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648063.336144 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561689.53 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 56168.95 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704232.289137 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505520.58 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 50552.06 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754784.346831 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454968.52 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 45496.85 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800281.198755 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409471.67 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 40947.17 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841228.365488 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368524.50 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 36852.45 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878080.815547 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331672.05 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 33167.21 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911248.020600 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298504.85 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 29850.48 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941098.505147 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268654.36 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 26865.44 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967963.941240 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241788.92 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 24178.89 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992142.833724 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217610.03 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 21761.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013903.836959 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195849.03 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 19584.90 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033488.739871 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176264.13 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 17626.41 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051115.152492 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158637.71 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 15863.77 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066978.923850 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142773.94 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 14277.39 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081256.318073 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128496.55 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 12849.65 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094105.972873 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115646.89 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 11564.69 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105670.662194 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104082.20 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 10408.22 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116078.882582 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93673.98 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 9367.40 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125446.280931 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84306.59 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 8430.66 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133876.939446 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75875.93 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 7587.59 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141464.532109 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68288.33 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 6828.83 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148293.365506 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61459.50 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 6145.95 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154439.315563 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55313.55 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 5531.36 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159970.670614 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49782.20 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 4978.22 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164948.890161 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44803.98 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 4480.40 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169429.287752 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40323.58 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 4032.36 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173461.645585 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36291.22 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3629.12 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177090.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32662.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180690.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29062.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184290.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25462.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187890.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21862.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191490.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18262.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195090.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14662.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198690.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11062.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202290.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7462.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205890.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3862.10 sec) 19:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209490.767634 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 262.10 sec) 19:40:00 [GPUGRID] [rr_sim] 2209752.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971802655.95G/439.78G) 19:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853593.78 19:40:00 [---] pick_jobs_to_run() start 19:40:00 [---] pick_jobs_to_run() end 19:40:00 [---] [rr_sim] end 19:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:40:00 [---] [cpu_sched_debug] preliminary job list: 19:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:40:00 [---] [cpu_sched_debug] final job list: 19:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:40:00 [---] [cpu_sched_debug] enforce_run_list: end 19:40:00 [---] choose_project(): 70800.000000 19:40:00 [WUProp@Home] doing RPC to get work 19:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364799.09 cpu 2113352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113352.90 19:40:00 [---] Got 0 tasks 19:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:41:00 [---] [rr_sim] doing sim: CPU sched 19:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971775065.60G/439.78G) 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209690.13 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 220969.01 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220969.012919 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988721.12 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 198872.11 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419841.124547 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789849.00 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 178984.90 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598826.025012 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610864.10 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 161086.41 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759912.435430 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449777.69 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 144977.77 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904890.204807 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304799.92 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 130479.99 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035370.197245 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174319.93 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 117431.99 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152802.190440 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056887.94 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 105688.79 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258490.984316 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951199.14 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 95119.91 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353610.898804 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856079.23 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 85607.92 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439218.821843 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770471.31 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 77047.13 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516265.952578 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693424.18 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 69342.42 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585608.370240 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624081.76 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 62408.18 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1648016.546135 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561673.58 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 56167.36 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704183.904441 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505506.22 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 50550.62 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754734.526916 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454955.60 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 45495.56 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800230.087144 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409460.04 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 40946.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841176.091349 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368514.04 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 36851.40 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1878027.495134 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331662.63 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 33166.26 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911193.758540 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298496.37 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 29849.64 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1941043.395605 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268646.73 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 26864.67 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967908.068964 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241782.06 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 24178.21 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992086.274987 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217603.85 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 21760.39 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013846.660408 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195843.47 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 19584.35 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033431.007287 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176259.12 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 17625.91 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051056.919478 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158633.21 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 15863.32 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066920.240449 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142769.89 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 14276.99 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081197.229324 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128492.90 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 12849.29 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094046.519311 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115643.61 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 11564.36 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105610.880299 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104079.25 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 10407.92 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116018.805189 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93671.32 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 9367.13 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125385.937589 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84304.19 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 8430.42 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133816.356750 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75873.77 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 7587.38 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141403.733994 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68286.40 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 6828.64 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148232.373514 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61457.76 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 6145.78 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154378.149082 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55311.98 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 5531.20 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159909.347094 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49780.78 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 4978.08 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164887.425304 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44802.70 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 4480.27 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169367.695693 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40322.43 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 4032.24 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173399.939043 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36290.19 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3629.02 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177028.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32661.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180628.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29061.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184228.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25461.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187828.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21861.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191428.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18261.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195028.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14661.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198628.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11061.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202228.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7461.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205828.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3861.17 sec) 19:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209428.958058 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 261.17 sec) 19:41:00 [GPUGRID] [rr_sim] 2209690.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971775065.60G/439.78G) 19:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853591.04 19:41:00 [---] pick_jobs_to_run() start 19:41:00 [---] pick_jobs_to_run() end 19:41:00 [---] [rr_sim] end 19:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:41:00 [---] [cpu_sched_debug] preliminary job list: 19:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:41:00 [---] [cpu_sched_debug] final job list: 19:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:41:00 [---] [cpu_sched_debug] enforce_run_list: end 19:41:00 [---] choose_project(): 70860.000000 19:41:00 [WUProp@Home] doing RPC to get work 19:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364739.09 cpu 2113292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113292.90 19:41:00 [---] Got 0 tasks 19:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:42:00 [---] [rr_sim] doing sim: CPU sched 19:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971747475.24G/439.78G) 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209627.39 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 220962.74 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220962.739231 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988664.65 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 198866.47 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419829.204539 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789798.19 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 178979.82 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598809.023317 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610818.37 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 161081.84 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759890.860216 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449736.53 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 144973.65 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904864.513426 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304762.88 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 130476.29 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035340.801315 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174286.59 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 117428.66 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152769.460414 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056857.93 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 105685.79 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258455.253604 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951172.14 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 95117.21 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353572.467475 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856054.92 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 85605.49 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439177.959959 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770449.43 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 77044.94 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516222.903194 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693404.49 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 69340.45 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585563.352106 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624064.04 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 62406.40 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647969.756127 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561657.64 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 56165.76 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704135.519745 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505491.87 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 50549.19 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754684.707002 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454942.69 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 45494.27 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800178.975533 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409448.42 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 40944.84 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841123.817211 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368503.58 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 36850.36 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877974.174721 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331653.22 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 33165.32 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911139.496480 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298487.90 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 29848.79 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940988.286063 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268639.11 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 26863.91 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967852.196688 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241775.20 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 24177.52 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1992029.716251 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217597.68 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 21759.77 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013789.483857 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195837.91 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 19583.79 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033373.274703 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176254.12 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 17625.41 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050998.686464 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158628.71 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 15862.87 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066861.557048 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142765.84 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 14276.58 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081138.140575 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128489.25 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 12848.93 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093987.065749 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115640.33 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 11564.03 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105551.098405 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104076.29 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 10407.63 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115958.727796 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93668.66 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 9366.87 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125325.594247 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84301.80 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 8430.18 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133755.774054 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75871.62 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 7587.16 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141342.935880 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68284.46 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 6828.45 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148171.381523 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61456.01 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 6145.60 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154316.982602 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55310.41 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 5531.04 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159848.023573 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49779.37 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 4977.94 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164825.960447 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44801.43 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 4480.14 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169306.103634 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40321.29 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 4032.13 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173338.232501 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36289.16 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3628.92 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176967.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32660.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180567.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29060.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184167.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25460.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187767.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21860.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191367.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18260.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194967.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14660.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198567.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11060.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202167.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7460.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205767.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3860.24 sec) 19:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209367.148483 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 260.24 sec) 19:42:00 [GPUGRID] [rr_sim] 2209627.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971747475.24G/439.78G) 19:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853588.31 19:42:00 [---] pick_jobs_to_run() start 19:42:00 [---] pick_jobs_to_run() end 19:42:00 [---] [rr_sim] end 19:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:42:00 [---] [cpu_sched_debug] preliminary job list: 19:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:42:00 [---] [cpu_sched_debug] final job list: 19:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:42:00 [---] [cpu_sched_debug] enforce_run_list: end 19:42:00 [---] choose_project(): 70920.000000 19:42:00 [WUProp@Home] doing RPC to get work 19:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364679.09 cpu 2113232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113232.90 19:42:00 [---] Got 0 tasks 19:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:43:00 [---] [rr_sim] doing sim: CPU sched 19:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971719884.88G/439.78G) 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209564.66 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 220956.47 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220956.465543 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988608.19 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 198860.82 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419817.284532 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789747.37 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 178974.74 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598792.021622 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610772.63 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 161077.26 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759869.285003 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449695.37 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 144969.54 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904838.822045 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304725.83 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 130472.58 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035311.405384 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174253.25 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 117425.33 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152736.730388 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056827.93 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 105682.79 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258419.522893 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951145.13 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 95114.51 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353534.036146 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856030.62 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 85603.06 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439137.098075 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770427.56 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 77042.76 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516179.853810 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693384.80 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 69338.48 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585518.333972 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624046.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 62404.63 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647922.966118 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561641.69 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 56164.17 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704087.135049 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505477.52 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 50547.75 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754634.887087 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454929.77 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 45492.98 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800127.863922 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409436.79 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 40943.68 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841071.543073 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368493.11 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 36849.31 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877920.854308 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331643.80 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 33164.38 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911085.234421 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298479.42 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 29847.94 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940933.176522 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268631.48 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 26863.15 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967796.324412 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241768.33 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 24176.83 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991973.157514 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217591.50 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 21759.15 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013732.307306 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195832.35 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 19583.23 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033315.542118 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176249.11 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 17624.91 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050940.453450 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158624.20 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 15862.42 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066802.873648 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142761.78 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 14276.18 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081079.051826 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128485.60 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 12848.56 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093927.612186 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115637.04 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 11563.70 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105491.316511 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104073.34 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 10407.33 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115898.650403 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93666.01 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 9366.60 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125265.250905 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84299.40 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 8429.94 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133695.191358 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75869.46 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 7586.95 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141282.137765 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68282.52 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 6828.25 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148110.389532 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61454.27 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 6145.43 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154255.816122 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55308.84 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 5530.88 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159786.700052 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49777.96 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 4977.80 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164764.495590 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44800.16 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 4480.02 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169244.511574 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40320.14 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 4032.01 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173276.525960 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36288.13 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3628.81 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176905.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32659.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180505.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29059.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184105.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25459.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187705.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21859.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191305.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18259.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194905.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14659.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198505.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11059.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202105.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7459.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205705.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3859.32 sec) 19:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209305.338907 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 259.32 sec) 19:43:00 [GPUGRID] [rr_sim] 2209564.66: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971719884.88G/439.78G) 19:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853585.57 19:43:00 [---] pick_jobs_to_run() start 19:43:00 [---] pick_jobs_to_run() end 19:43:00 [---] [rr_sim] end 19:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:43:00 [---] [cpu_sched_debug] preliminary job list: 19:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:43:00 [---] [cpu_sched_debug] final job list: 19:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:43:00 [---] [cpu_sched_debug] enforce_run_list: end 19:43:00 [---] choose_project(): 70980.000000 19:43:00 [WUProp@Home] doing RPC to get work 19:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364619.09 cpu 2113172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113172.90 19:43:00 [---] Got 0 tasks 19:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:44:00 [---] [rr_sim] doing sim: CPU sched 19:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971692294.53G/439.78G) 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209501.92 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 220950.19 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220950.191855 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988551.73 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 198855.17 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419805.364524 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789696.55 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 178969.66 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598775.019927 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610726.90 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 161072.69 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759847.709789 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449654.21 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 144965.42 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904813.130665 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304688.79 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 130468.88 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035282.009453 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174219.91 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 117421.99 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152704.000363 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056797.92 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 105679.79 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258383.792181 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951118.13 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 95111.81 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353495.604818 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 856006.31 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 85600.63 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439096.236191 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770405.68 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 77040.57 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516136.804427 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693365.11 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 69336.51 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585473.315839 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624028.60 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 62402.86 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647876.176110 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561625.74 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 56162.57 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1704038.750354 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505463.17 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 50546.32 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754585.067173 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454916.85 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 45491.69 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800076.752311 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409425.17 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 40942.52 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1841019.268934 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368482.65 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 36848.26 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877867.533896 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331634.38 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 33163.44 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1911030.972361 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298470.95 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 29847.09 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940878.066980 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268623.85 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 26862.39 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967740.452137 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241761.47 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 24176.15 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991916.598778 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217585.32 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 21758.53 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013675.130755 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195826.79 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 19582.68 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033257.809534 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176244.11 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 17624.41 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050882.220435 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158619.70 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 15861.97 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066744.190247 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142757.73 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 14275.77 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081019.963077 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128481.96 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 12848.20 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093868.158624 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115633.76 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 11563.38 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105431.534616 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104070.38 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 10407.04 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115838.573010 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93663.35 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 9366.33 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125204.907563 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84297.01 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 8429.70 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133634.608662 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75867.31 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 7586.73 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141221.339651 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68280.58 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 6828.06 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148049.397540 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61452.52 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 6145.25 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154194.649641 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55307.27 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 5530.73 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159725.376532 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49776.54 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 4977.65 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164703.030733 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44798.89 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 4479.89 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169182.919515 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40319.00 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 4031.90 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173214.819418 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36287.10 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3628.71 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176843.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32658.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180443.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29058.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184043.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25458.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187643.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21858.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191243.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18258.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194843.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14658.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198443.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11058.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202043.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7458.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205643.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3858.39 sec) 19:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209243.529331 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 258.39 sec) 19:44:00 [GPUGRID] [rr_sim] 2209501.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971692294.53G/439.78G) 19:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853582.83 19:44:00 [---] pick_jobs_to_run() start 19:44:00 [---] pick_jobs_to_run() end 19:44:00 [---] [rr_sim] end 19:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:44:00 [---] [cpu_sched_debug] preliminary job list: 19:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:44:00 [---] [cpu_sched_debug] final job list: 19:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:44:00 [---] [cpu_sched_debug] enforce_run_list: end 19:44:00 [---] choose_project(): 71040.000000 19:44:00 [WUProp@Home] doing RPC to get work 19:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364559.09 cpu 2113112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113112.90 19:44:00 [---] Got 0 tasks 19:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:45:00 [---] [rr_sim] doing sim: CPU sched 19:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971664704.17G/439.78G) 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209439.18 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 220943.92 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220943.918167 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988495.26 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 198849.53 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419793.444517 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789645.74 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 178964.57 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598758.018231 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610681.16 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 161068.12 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759826.134575 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449613.05 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 144961.30 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904787.439284 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304651.74 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 130465.17 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035252.613522 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174186.57 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 117418.66 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152671.270337 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056767.91 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 105676.79 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258348.061470 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951091.12 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 95109.11 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353457.173489 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855982.01 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 85598.20 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439055.374307 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770383.81 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 77038.38 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516093.755043 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693345.43 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 69334.54 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585428.297705 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 624010.88 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 62401.09 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647829.386101 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561609.80 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 56160.98 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703990.365658 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505448.82 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 50544.88 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754535.247259 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454903.93 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 45490.39 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1800025.640699 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409413.54 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 40941.35 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840966.994796 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368472.19 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 36847.22 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877814.213483 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331624.97 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 33162.50 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910976.710301 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298462.47 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 29846.25 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940822.957438 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268616.22 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 26861.62 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967684.579861 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241754.60 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 24175.46 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991860.040041 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217579.14 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 21757.91 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013617.954204 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195821.23 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 19582.12 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033200.076950 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176239.10 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 17623.91 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050823.987421 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158615.19 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 15861.52 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066685.506846 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142753.67 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 14275.37 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080960.874328 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128478.31 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 12847.83 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093808.705062 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115630.48 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 11563.05 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105371.752722 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104067.43 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 10406.74 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115778.495617 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93660.69 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 9366.07 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125144.564222 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84294.62 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 8429.46 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133574.025966 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75865.16 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 7586.52 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141160.541536 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68278.64 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 6827.86 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147988.405549 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61450.78 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 6145.08 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154133.483161 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55305.70 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 5530.57 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159664.053011 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49775.13 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 4977.51 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164641.565877 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44797.62 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 4479.76 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169121.327456 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40317.85 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 4031.79 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173153.112877 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36286.07 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3628.61 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176781.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32657.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180381.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29057.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183981.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25457.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187581.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21857.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191181.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18257.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194781.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14657.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198381.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11057.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201981.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7457.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205581.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3857.46 sec) 19:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209181.719756 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 257.46 sec) 19:45:00 [GPUGRID] [rr_sim] 2209439.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971664704.17G/439.78G) 19:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853580.09 19:45:00 [---] pick_jobs_to_run() start 19:45:00 [---] pick_jobs_to_run() end 19:45:00 [---] [rr_sim] end 19:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:45:00 [---] [cpu_sched_debug] preliminary job list: 19:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:45:00 [---] [cpu_sched_debug] final job list: 19:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:45:00 [---] [cpu_sched_debug] enforce_run_list: end 19:45:00 [---] choose_project(): 71100.000000 19:45:00 [WUProp@Home] doing RPC to get work 19:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364499.09 cpu 2113052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2113052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2113052.90 19:45:00 [---] Got 0 tasks 19:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:46:00 [---] [rr_sim] doing sim: CPU sched 19:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971637113.81G/439.78G) 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209376.44 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 220937.64 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220937.644478 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988438.80 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 198843.88 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419781.524509 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789594.92 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 178959.49 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598741.016536 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610635.43 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 161063.54 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759804.559361 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449571.89 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 144957.19 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904761.747903 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304614.70 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 130461.47 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035223.217591 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174153.23 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 117415.32 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152638.540311 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056737.90 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 105673.79 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258312.330758 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951064.11 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 95106.41 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353418.742161 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855957.70 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 85595.77 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1439014.512423 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770361.93 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 77036.19 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516050.705659 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693325.74 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 69332.57 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585383.279572 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623993.17 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 62399.32 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647782.596093 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561593.85 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 56159.38 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703941.980962 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505434.46 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 50543.45 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754485.427344 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454891.02 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 45489.10 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799974.529088 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409401.92 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 40940.19 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840914.720658 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368461.72 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 36846.17 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877760.893070 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331615.55 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 33161.56 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910922.448242 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298454.00 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 29845.40 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940767.847896 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268608.60 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 26860.86 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967628.707585 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241747.74 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 24174.77 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991803.481305 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217572.96 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 21757.30 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013560.777652 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195815.67 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 19581.57 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033142.344366 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176234.10 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 17623.41 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050765.754407 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158610.69 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 15861.07 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066626.823445 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142749.62 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 14274.96 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080901.785579 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128474.66 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 12847.47 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093749.251499 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115627.19 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 11562.72 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105311.970828 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104064.47 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 10406.45 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115718.418223 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93658.03 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 9365.80 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125084.220880 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84292.22 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 8429.22 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133513.443270 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75863.00 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 7586.30 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141099.743421 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68276.70 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 6827.67 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147927.413558 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61449.03 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 6144.90 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154072.316680 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55304.13 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 5530.41 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159602.729491 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49773.72 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 4977.37 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164580.101020 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44796.34 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 4479.63 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169059.735396 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40316.71 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 4031.67 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173091.406335 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36285.04 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3628.50 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176719.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32656.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180319.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29056.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183919.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25456.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187519.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21856.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191119.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18256.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194719.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14656.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198319.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11056.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201919.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7456.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205519.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3856.53 sec) 19:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209119.910180 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 256.53 sec) 19:46:00 [GPUGRID] [rr_sim] 2209376.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971637113.81G/439.78G) 19:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853577.36 19:46:00 [---] pick_jobs_to_run() start 19:46:00 [---] pick_jobs_to_run() end 19:46:00 [---] [rr_sim] end 19:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:46:00 [---] [cpu_sched_debug] preliminary job list: 19:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:46:00 [---] [cpu_sched_debug] final job list: 19:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:46:00 [---] [cpu_sched_debug] enforce_run_list: end 19:46:00 [---] choose_project(): 71160.000000 19:46:00 [WUProp@Home] doing RPC to get work 19:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364439.09 cpu 2112992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112992.90 19:46:00 [---] Got 0 tasks 19:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:47:00 [---] [rr_sim] doing sim: CPU sched 19:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971609523.46G/439.78G) 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209313.71 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 220931.37 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220931.370790 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988382.34 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 198838.23 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419769.604501 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789544.10 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 178954.41 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598724.014841 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610589.69 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 161058.97 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759782.984147 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449530.72 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 144953.07 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904736.056523 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304577.65 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 130457.77 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035193.821661 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174119.89 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 117411.99 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152605.810285 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056707.90 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 105670.79 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258276.600046 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951037.11 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 95103.71 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353380.310832 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855933.40 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 85593.34 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438973.650539 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770340.06 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 77034.01 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1516007.656275 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693306.05 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 69330.61 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585338.261438 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623975.45 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 62397.54 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647735.806084 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561577.90 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 56157.79 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703893.596266 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505420.11 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 50542.01 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754435.607430 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454878.10 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 45487.81 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799923.417477 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409390.29 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 40939.03 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840862.446519 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368451.26 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 36845.13 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877707.572658 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331606.14 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 33160.61 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910868.186182 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298445.52 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 29844.55 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940712.738354 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268600.97 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 26860.10 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967572.835309 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241740.87 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 24174.09 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991746.922568 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217566.79 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 21756.68 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013503.601101 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195810.11 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 19581.01 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033084.611781 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176229.10 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 17622.91 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050707.521393 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158606.19 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 15860.62 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066568.140044 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142745.57 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 14274.56 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080842.696830 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128471.01 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 12847.10 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093689.797937 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115623.91 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 11562.39 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105252.188934 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104061.52 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 10406.15 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115658.340830 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93655.37 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 9365.54 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125023.877538 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84289.83 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 8428.98 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133452.860574 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75860.85 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 7586.08 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141038.945307 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68274.76 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 6827.48 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147866.421566 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61447.29 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 6144.73 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154011.150200 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55302.56 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 5530.26 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159541.405970 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49772.30 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 4977.23 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164518.636163 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44795.07 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 4479.51 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168998.143337 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40315.56 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 4031.56 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173029.699793 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36284.01 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3628.40 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176658.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32655.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180258.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29055.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183858.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25455.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187458.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21855.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191058.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18255.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194658.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14655.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198258.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11055.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201858.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7455.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205458.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3855.61 sec) 19:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2209058.100604 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 255.61 sec) 19:47:00 [GPUGRID] [rr_sim] 2209313.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971609523.46G/439.78G) 19:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853574.62 19:47:00 [---] pick_jobs_to_run() start 19:47:00 [---] pick_jobs_to_run() end 19:47:00 [---] [rr_sim] end 19:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:47:00 [---] [cpu_sched_debug] preliminary job list: 19:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:47:00 [---] [cpu_sched_debug] final job list: 19:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:47:00 [---] [cpu_sched_debug] enforce_run_list: end 19:47:00 [---] choose_project(): 71220.000000 19:47:00 [WUProp@Home] doing RPC to get work 19:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364379.09 cpu 2112932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112932.90 19:47:00 [---] Got 0 tasks 19:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:48:00 [---] [rr_sim] doing sim: CPU sched 19:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971581933.10G/439.78G) 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209250.97 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 220925.10 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220925.097102 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988325.87 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 198832.59 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419757.684494 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789493.29 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 178949.33 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598707.013146 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610543.96 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 161054.40 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759761.408934 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449489.56 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 144948.96 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904710.365142 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304540.61 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 130454.06 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035164.425730 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174086.55 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 117408.65 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152573.080259 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056677.89 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 105667.79 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258240.869335 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 951010.10 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 95101.01 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353341.879503 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855909.09 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 85590.91 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438932.788655 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770318.18 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 77031.82 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515964.606891 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693286.36 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 69328.64 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585293.243304 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623957.73 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 62395.77 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647689.016076 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561561.95 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 56156.20 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703845.211570 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505405.76 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 50540.58 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754385.787515 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454865.18 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 45486.52 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799872.305866 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409378.67 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 40937.87 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840810.172381 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368440.80 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 36844.08 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877654.252245 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331596.72 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 33159.67 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910813.924122 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298437.05 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 29843.70 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940657.628812 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268593.34 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 26859.33 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967516.963033 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241734.01 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 24173.40 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991690.363831 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217560.61 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 21756.06 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013446.424550 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195804.55 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 19580.45 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033026.879197 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176224.09 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 17622.41 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050649.288379 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158601.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 15860.17 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066509.456643 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142741.51 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 14274.15 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080783.608081 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128467.36 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 12846.74 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093630.344375 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115620.63 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 11562.06 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105192.407039 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104058.56 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 10405.86 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115598.263437 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93652.71 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 9365.27 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124963.534196 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84287.44 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 8428.74 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133392.277878 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75858.69 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 7585.87 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140978.147192 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68272.82 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 6827.28 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147805.429575 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61445.54 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 6144.55 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153949.983719 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55300.99 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 5530.10 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159480.082449 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49770.89 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 4977.09 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164457.171306 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44793.80 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 4479.38 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168936.551278 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40314.42 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 4031.44 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172967.993252 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36282.98 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3628.30 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176596.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32654.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180196.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29054.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183796.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25454.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187396.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21854.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190996.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18254.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194596.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14654.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198196.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11054.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201796.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7454.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205396.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3854.68 sec) 19:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208996.291029 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 254.68 sec) 19:48:00 [GPUGRID] [rr_sim] 2209250.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971581933.10G/439.78G) 19:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853571.88 19:48:00 [---] pick_jobs_to_run() start 19:48:00 [---] pick_jobs_to_run() end 19:48:00 [---] [rr_sim] end 19:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:48:00 [---] [cpu_sched_debug] preliminary job list: 19:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:48:00 [---] [cpu_sched_debug] final job list: 19:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:48:00 [---] [cpu_sched_debug] enforce_run_list: end 19:48:00 [---] choose_project(): 71280.000000 19:48:00 [WUProp@Home] doing RPC to get work 19:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364319.09 cpu 2112872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112872.90 19:48:00 [---] Got 0 tasks 19:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:49:00 [---] [rr_sim] doing sim: CPU sched 19:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971554342.75G/439.78G) 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209188.23 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 220918.82 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220918.823414 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988269.41 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 198826.94 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419745.764486 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789442.47 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 178944.25 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598690.011451 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610498.22 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 161049.82 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759739.833720 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449448.40 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 144944.84 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904684.673762 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304503.56 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 130450.36 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035135.029799 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174053.20 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 117405.32 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152540.350233 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056647.88 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 105664.79 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258205.138623 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950983.10 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 95098.31 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353303.448175 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855884.79 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 85588.48 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438891.926771 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770296.31 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 77029.63 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515921.557508 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693266.68 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 69326.67 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585248.225171 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623940.01 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 62394.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647642.226067 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561546.01 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 56154.60 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703796.826874 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505391.41 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 50539.14 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754335.967601 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454852.27 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 45485.23 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799821.194254 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409367.04 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 40936.70 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840757.898243 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368430.34 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 36843.03 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877600.931832 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331587.30 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 33158.73 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910759.662063 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298428.57 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 29842.86 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940602.519270 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268585.71 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 26858.57 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967461.090757 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241727.14 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 24172.71 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991633.805095 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217554.43 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 21755.44 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013389.247999 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195798.99 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 19579.90 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032969.146613 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176219.09 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 17621.91 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050591.055365 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158597.18 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 15859.72 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066450.773243 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142737.46 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 14273.75 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080724.519332 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128463.71 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 12846.37 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093570.890813 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115617.34 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 11561.73 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105132.625145 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104055.61 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 10405.56 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115538.186044 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93650.05 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 9365.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124903.190854 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84285.04 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 8428.50 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133331.695182 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75856.54 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 7585.65 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140917.349078 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68270.89 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 6827.09 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147744.437584 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61443.80 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 6144.38 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153888.817239 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55299.42 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 5529.94 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159418.758929 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49769.48 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 4976.95 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164395.706450 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44792.53 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 4479.25 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168874.959218 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40313.27 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 4031.33 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172906.286710 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36281.95 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3628.19 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176534.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32653.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180134.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29053.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183734.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25453.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187334.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21853.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190934.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18253.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194534.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14653.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198134.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11053.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201734.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7453.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205334.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3853.75 sec) 19:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208934.481453 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 253.75 sec) 19:49:00 [GPUGRID] [rr_sim] 2209188.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971554342.75G/439.78G) 19:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853569.15 19:49:00 [---] pick_jobs_to_run() start 19:49:00 [---] pick_jobs_to_run() end 19:49:00 [---] [rr_sim] end 19:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:49:00 [---] [cpu_sched_debug] preliminary job list: 19:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:49:00 [---] [cpu_sched_debug] final job list: 19:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:49:00 [---] [cpu_sched_debug] enforce_run_list: end 19:49:00 [---] choose_project(): 71340.000000 19:49:00 [WUProp@Home] doing RPC to get work 19:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364259.09 cpu 2112812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112812.90 19:49:00 [---] Got 0 tasks 19:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:50:00 [---] [rr_sim] doing sim: CPU sched 19:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971526752.39G/439.78G) 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209125.50 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 220912.55 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220912.549726 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988212.95 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 198821.29 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419733.844478 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789391.65 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 178939.17 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598673.009756 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610452.49 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 161045.25 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759718.258506 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449407.24 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 144940.72 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904658.982381 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304466.51 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 130446.65 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035105.633868 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1174019.86 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 117401.99 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152507.620207 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056617.88 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 105661.79 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258169.407912 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950956.09 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 95095.61 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353265.016846 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855860.48 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 85586.05 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438851.064887 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770274.43 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 77027.44 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515878.508124 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693246.99 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 69324.70 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585203.207037 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623922.29 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 62392.23 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647595.436059 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561530.06 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 56153.01 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703748.442179 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505377.06 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 50537.71 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754286.147686 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454839.35 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 45483.93 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799770.082643 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409355.41 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 40935.54 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840705.624104 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368419.87 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 36841.99 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877547.611419 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331577.89 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 33157.79 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910705.400003 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298420.10 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 29842.01 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940547.409728 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268578.09 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 26857.81 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967405.218481 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241720.28 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 24172.03 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991577.246358 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217548.25 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 21754.83 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013332.071448 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195793.43 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 19579.34 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032911.414029 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176214.08 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 17621.41 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050532.822351 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158592.67 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 15859.27 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066392.089842 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142733.41 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 14273.34 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080665.430583 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128460.07 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 12846.01 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093511.437250 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115614.06 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 11561.41 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105072.843251 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104052.65 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 10405.27 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115478.108651 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93647.39 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 9364.74 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124842.847512 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84282.65 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 8428.26 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133271.112486 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75854.38 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 7585.44 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140856.550963 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68268.95 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 6826.89 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147683.445592 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61442.05 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 6144.21 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153827.650758 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55297.85 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 5529.78 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159357.435408 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49768.06 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 4976.81 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164334.241593 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44791.26 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 4479.13 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168813.367159 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40312.13 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 4031.21 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172844.580169 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36280.92 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3628.09 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176472.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32652.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180072.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29052.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183672.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25452.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187272.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21852.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190872.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18252.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194472.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14652.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198072.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11052.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201672.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7452.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205272.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3852.83 sec) 19:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208872.671877 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 252.83 sec) 19:50:00 [GPUGRID] [rr_sim] 2209125.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971526752.39G/439.78G) 19:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853566.41 19:50:00 [---] pick_jobs_to_run() start 19:50:00 [---] pick_jobs_to_run() end 19:50:00 [---] [rr_sim] end 19:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:50:00 [---] [cpu_sched_debug] preliminary job list: 19:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:50:00 [---] [cpu_sched_debug] final job list: 19:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:50:00 [---] [cpu_sched_debug] enforce_run_list: end 19:50:00 [---] choose_project(): 71400.000000 19:50:00 [WUProp@Home] doing RPC to get work 19:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364199.09 cpu 2112752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112752.90 19:50:00 [---] Got 0 tasks 19:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:51:00 [---] [rr_sim] doing sim: CPU sched 19:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971499162.03G/439.78G) 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209062.76 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 220906.28 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220906.276037 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988156.48 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 198815.65 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419721.924471 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789340.84 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 178934.08 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598656.008061 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610406.75 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 161040.68 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759696.683292 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449366.08 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 144936.61 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904633.291000 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304429.47 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 130442.95 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035076.237938 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173986.52 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 117398.65 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152474.890181 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056587.87 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 105658.79 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258133.677200 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950929.08 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 95092.91 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353226.585518 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855836.17 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 85583.62 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438810.203003 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770252.56 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 77025.26 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515835.458740 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693227.30 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 69322.73 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585158.188903 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623904.57 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 62390.46 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647548.646050 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561514.11 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 56151.41 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703700.057483 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505362.70 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 50536.27 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754236.327772 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454826.43 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 45482.64 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799718.971032 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409343.79 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 40934.38 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840653.349966 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368409.41 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 36840.94 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877494.291007 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331568.47 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 33156.85 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910651.137943 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298411.62 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 29841.16 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940492.300186 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268570.46 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 26857.05 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967349.346205 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241713.41 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 24171.34 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991520.687622 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217542.07 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 21754.21 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013274.894897 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195787.87 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 19578.79 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032853.681445 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176209.08 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 17620.91 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050474.589337 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158588.17 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 15858.82 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066333.406441 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142729.35 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 14272.94 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080606.341834 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128456.42 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 12845.64 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093451.983688 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115610.78 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 11561.08 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105013.061357 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104049.70 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 10404.97 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115418.031258 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93644.73 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 9364.47 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124782.504170 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84280.26 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 8428.03 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133210.529790 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75852.23 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 7585.22 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140795.752848 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68267.01 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 6826.70 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147622.453601 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61440.31 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 6144.03 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153766.484278 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55296.28 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 5529.63 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159296.111888 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49766.65 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 4976.66 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164272.776736 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44789.98 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 4479.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168751.775100 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40310.99 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 4031.10 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172782.873627 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36279.89 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3627.99 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176410.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32651.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180010.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29051.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183610.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25451.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187210.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21851.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190810.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18251.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194410.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14651.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198010.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11051.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201610.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7451.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205210.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3851.90 sec) 19:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208810.862302 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 251.90 sec) 19:51:00 [GPUGRID] [rr_sim] 2209062.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971499162.03G/439.78G) 19:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853563.67 19:51:00 [---] pick_jobs_to_run() start 19:51:00 [---] pick_jobs_to_run() end 19:51:00 [---] [rr_sim] end 19:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:51:00 [---] [cpu_sched_debug] preliminary job list: 19:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:51:00 [---] [cpu_sched_debug] final job list: 19:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:51:00 [---] [cpu_sched_debug] enforce_run_list: end 19:51:00 [---] choose_project(): 71460.000000 19:51:00 [WUProp@Home] doing RPC to get work 19:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364139.09 cpu 2112692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112692.90 19:51:00 [---] Got 0 tasks 19:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:52:00 [---] [rr_sim] doing sim: CPU sched 19:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971471571.68G/439.78G) 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2209000.02 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 220900.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220900.002349 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988100.02 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 198810.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419710.004463 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789290.02 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 178929.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598639.006366 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610361.02 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 161036.10 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759675.108079 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449324.92 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 144932.49 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904607.599620 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304392.42 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 130439.24 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035046.842007 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173953.18 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 117395.32 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152442.160155 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056557.86 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 105655.79 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258097.946489 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950902.08 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 95090.21 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353188.154189 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855811.87 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 85581.19 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438769.341119 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770230.68 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 77023.07 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515792.409356 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693207.61 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 69320.76 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585113.170770 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623886.85 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 62388.69 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647501.856042 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561498.17 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 56149.82 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703651.672787 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505348.35 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 50534.84 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754186.507857 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454813.52 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 45481.35 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799667.859421 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409332.16 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 40933.22 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840601.075828 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368398.95 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 36839.89 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877440.970594 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331559.05 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 33155.91 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910596.875884 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298403.15 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 29840.31 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940437.190644 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268562.83 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 26856.28 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967293.473929 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241706.55 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 24170.65 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991464.128885 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217535.89 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 21753.59 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013217.718346 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195782.31 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 19578.23 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032795.948860 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176204.07 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 17620.41 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050416.356323 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158583.67 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 15858.37 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066274.723040 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142725.30 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 14272.53 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080547.253085 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128452.77 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 12845.28 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093392.530126 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115607.49 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 11560.75 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104953.279462 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104046.74 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 10404.67 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115357.953865 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93642.07 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 9364.21 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124722.160828 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84277.86 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 8427.79 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133149.947094 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75850.08 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 7585.01 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140734.954734 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68265.07 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 6826.51 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147561.461609 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61438.56 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 6143.86 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153705.317798 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55294.71 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 5529.47 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159234.788367 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49765.24 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 4976.52 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164211.311879 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44788.71 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 4478.87 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168690.183040 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40309.84 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 4030.98 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172721.167085 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36278.86 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3627.89 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176349.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32650.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179949.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29050.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183549.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25450.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187149.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21850.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190749.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18250.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194349.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14650.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197949.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11050.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201549.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7450.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205149.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3850.97 sec) 19:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208749.052726 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 250.97 sec) 19:52:00 [GPUGRID] [rr_sim] 2209000.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971471571.68G/439.78G) 19:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853560.94 19:52:00 [---] pick_jobs_to_run() start 19:52:00 [---] pick_jobs_to_run() end 19:52:00 [---] [rr_sim] end 19:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:52:00 [---] [cpu_sched_debug] preliminary job list: 19:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:52:00 [---] [cpu_sched_debug] final job list: 19:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:52:00 [---] [cpu_sched_debug] enforce_run_list: end 19:52:00 [---] choose_project(): 71520.000000 19:52:00 [WUProp@Home] doing RPC to get work 19:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364079.09 cpu 2112632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112632.90 19:52:00 [---] Got 0 tasks 19:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:53:00 [---] [rr_sim] doing sim: CPU sched 19:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971443981.32G/439.78G) 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208937.29 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 220893.73 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220893.728661 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1988043.56 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 198804.36 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419698.084456 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789239.20 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 178923.92 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598622.004671 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610315.28 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 161031.53 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759653.532865 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449283.75 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 144928.38 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904581.908239 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304355.38 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 130435.54 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1035017.446076 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173919.84 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 117391.98 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152409.430129 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056527.86 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 105652.79 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258062.215777 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950875.07 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 95087.51 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353149.722860 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855787.56 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 85578.76 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438728.479235 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770208.81 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 77020.88 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515749.359973 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693187.93 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 69318.79 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585068.152636 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623869.13 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 62386.91 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647455.066033 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561482.22 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 56148.22 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703603.288091 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505334.00 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 50533.40 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754136.687943 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454800.60 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 45480.06 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799616.747809 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409320.54 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 40932.05 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840548.801689 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368388.48 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 36838.85 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877387.650181 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331549.64 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 33154.96 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910542.613824 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298394.67 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 29839.47 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940382.081102 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268555.21 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 26855.52 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967237.601653 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241699.68 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 24169.97 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991407.570149 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217529.72 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 21752.97 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013160.541795 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195776.74 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 19577.67 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032738.216276 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176199.07 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 17619.91 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050358.123309 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158579.16 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 15857.92 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066216.039639 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142721.25 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 14272.12 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080488.164336 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128449.12 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 12844.91 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093333.076563 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115604.21 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 11560.42 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104893.497568 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104043.79 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 10404.38 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115297.876472 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93639.41 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 9363.94 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124661.817486 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84275.47 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 8427.55 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133089.364398 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75847.92 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 7584.79 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140674.156619 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68263.13 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 6826.31 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147500.469618 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61436.82 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 6143.68 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153644.151317 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55293.14 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 5529.31 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159173.464846 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49763.82 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 4976.38 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164149.847023 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44787.44 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 4478.74 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168628.590981 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40308.70 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 4030.87 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172659.460544 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36277.83 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3627.78 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176287.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32650.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179887.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29050.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183487.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25450.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187087.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21850.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190687.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18250.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194287.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14650.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197887.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11050.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201487.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7450.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205087.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3850.04 sec) 19:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208687.243150 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 250.04 sec) 19:53:00 [GPUGRID] [rr_sim] 2208937.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971443981.32G/439.78G) 19:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853558.20 19:53:00 [---] pick_jobs_to_run() start 19:53:00 [---] pick_jobs_to_run() end 19:53:00 [---] [rr_sim] end 19:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:53:00 [---] [cpu_sched_debug] preliminary job list: 19:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:53:00 [---] [cpu_sched_debug] final job list: 19:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:53:00 [---] [cpu_sched_debug] enforce_run_list: end 19:53:00 [---] choose_project(): 71580.000000 19:53:00 [WUProp@Home] doing RPC to get work 19:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 364019.09 cpu 2112572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112572.90 19:53:00 [---] Got 0 tasks 19:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:54:00 [---] [rr_sim] doing sim: CPU sched 19:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971416390.97G/439.78G) 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208874.55 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 220887.45 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220887.454973 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987987.09 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 198798.71 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419686.164448 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789188.39 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 178918.84 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598605.002976 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610269.55 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 161026.95 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759631.957651 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449242.59 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 144924.26 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904556.216859 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304318.33 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 130431.83 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034988.050145 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173886.50 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 117388.65 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152376.700103 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056497.85 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 105649.78 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1258026.485066 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950848.06 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 95084.81 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353111.291532 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855763.26 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 85576.33 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438687.617351 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770186.93 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 77018.69 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515706.310589 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693168.24 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 69316.82 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1585023.134503 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623851.42 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 62385.14 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647408.276025 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561466.27 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 56146.63 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703554.903395 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505319.65 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 50531.96 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754086.868028 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454787.68 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 45478.77 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799565.636198 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409308.91 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 40930.89 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840496.527551 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368378.02 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 36837.80 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877334.329769 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331540.22 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 33154.02 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910488.351764 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298386.20 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 29838.62 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940326.971561 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268547.58 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 26854.76 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967181.729377 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241692.82 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 24169.28 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991351.011412 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217523.54 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 21752.35 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013103.365244 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195771.18 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 19577.12 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032680.483692 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176194.07 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 17619.41 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050299.890295 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158574.66 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 15857.47 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066157.356238 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142717.19 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 14271.72 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080429.075587 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128445.47 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 12844.55 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093273.623001 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115600.93 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 11560.09 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104833.715674 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104040.83 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 10404.08 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115237.799079 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93636.75 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 9363.68 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124601.474144 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84273.08 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 8427.31 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133028.781702 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75845.77 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 7584.58 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140613.358504 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68261.19 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 6826.12 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147439.477627 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61435.07 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 6143.51 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153582.984837 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55291.56 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 5529.16 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159112.141326 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49762.41 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 4976.24 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164088.382166 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44786.17 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 4478.62 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168566.998922 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40307.55 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 4030.76 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172597.754002 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36276.80 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3627.68 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176225.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32649.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179825.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29049.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183425.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25449.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187025.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21849.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190625.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18249.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194225.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14649.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197825.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11049.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201425.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7449.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205025.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3849.12 sec) 19:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208625.433575 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 249.12 sec) 19:54:00 [GPUGRID] [rr_sim] 2208874.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971416390.97G/439.78G) 19:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853555.46 19:54:00 [---] pick_jobs_to_run() start 19:54:00 [---] pick_jobs_to_run() end 19:54:00 [---] [rr_sim] end 19:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:54:00 [---] [cpu_sched_debug] preliminary job list: 19:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:54:00 [---] [cpu_sched_debug] final job list: 19:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:54:00 [---] [cpu_sched_debug] enforce_run_list: end 19:54:00 [---] choose_project(): 71640.000000 19:54:00 [WUProp@Home] doing RPC to get work 19:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363959.09 cpu 2112512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112512.90 19:54:00 [---] Got 0 tasks 19:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:55:00 [---] [rr_sim] doing sim: CPU sched 19:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971388800.61G/439.78G) 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208811.81 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 220881.18 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220881.181284 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987930.63 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 198793.06 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419674.244440 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789137.57 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 178913.76 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598588.001281 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610223.81 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 161022.38 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759610.382437 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449201.43 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 144920.14 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904530.525478 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304281.29 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 130428.13 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034958.654215 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173853.16 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 117385.32 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152343.970078 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056467.84 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 105646.78 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257990.754354 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950821.06 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 95082.11 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353072.860203 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855738.95 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 85573.90 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438646.755467 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770165.06 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 77016.51 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515663.261205 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693148.55 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 69314.86 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584978.116369 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623833.70 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 62383.37 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647361.486017 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561450.33 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 56145.03 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703506.518699 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505305.29 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 50530.53 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1754037.048114 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454774.76 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 45477.48 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799514.524587 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409297.29 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 40929.73 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840444.253413 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368367.56 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 36836.76 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877281.009356 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331530.80 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 33153.08 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910434.089705 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298377.72 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 29837.77 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940271.862019 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268539.95 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 26854.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967125.857101 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241685.96 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 24168.60 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991294.452676 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217517.36 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 21751.74 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2013046.188692 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195765.62 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 19576.56 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032622.751108 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176189.06 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 17618.91 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050241.657281 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158570.16 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 15857.02 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066098.672838 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142713.14 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 14271.31 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080369.986838 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128441.83 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 12844.18 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093214.169439 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115597.64 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 11559.76 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104773.933779 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104037.88 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 10403.79 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115177.721686 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93634.09 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 9363.41 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124541.130802 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84270.68 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 8427.07 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132968.199006 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75843.61 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 7584.36 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140552.560390 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68259.25 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 6825.93 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147378.485635 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61433.33 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 6143.33 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153521.818356 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55289.99 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 5529.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159050.817805 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49761.00 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 4976.10 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164026.917309 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44784.90 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 4478.49 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168505.406863 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40306.41 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 4030.64 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172536.047461 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36275.77 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3627.58 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176163.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32648.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179763.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29048.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183363.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25448.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186963.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21848.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190563.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18248.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194163.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14648.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197763.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11048.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201363.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7448.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204963.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3848.19 sec) 19:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208563.623999 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 248.19 sec) 19:55:00 [GPUGRID] [rr_sim] 2208811.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971388800.61G/439.78G) 19:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853552.73 19:55:00 [---] pick_jobs_to_run() start 19:55:00 [---] pick_jobs_to_run() end 19:55:00 [---] [rr_sim] end 19:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:55:00 [---] [cpu_sched_debug] preliminary job list: 19:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:55:00 [---] [cpu_sched_debug] final job list: 19:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:55:00 [---] [cpu_sched_debug] enforce_run_list: end 19:55:00 [---] choose_project(): 71700.000000 19:55:00 [WUProp@Home] doing RPC to get work 19:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363899.09 cpu 2112452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112452.90 19:55:00 [---] Got 0 tasks 19:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:56:00 [---] [rr_sim] doing sim: CPU sched 19:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971361210.25G/439.78G) 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208749.08 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 220874.91 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220874.907596 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987874.17 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 198787.42 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419662.324433 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789086.75 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 178908.68 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598570.999586 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610178.08 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 161017.81 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759588.807223 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449160.27 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 144916.03 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904504.834097 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304244.24 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 130424.42 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034929.258284 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173819.82 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 117381.98 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152311.240052 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056437.84 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 105643.78 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257955.023643 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950794.05 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 95079.41 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1353034.428875 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855714.65 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 85571.46 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438605.893583 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770143.18 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 77014.32 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515620.211821 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693128.86 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 69312.89 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584933.098235 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623815.98 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 62381.60 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647314.696008 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561434.38 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 56143.44 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703458.134003 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505290.94 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 50529.09 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753987.228199 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454761.85 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 45476.18 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799463.412976 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409285.66 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 40928.57 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840391.979274 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368357.10 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 36835.71 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877227.688943 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331521.39 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 33152.14 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910379.827645 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298369.25 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 29836.92 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940216.752477 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268532.32 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 26853.23 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967069.984825 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241679.09 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 24167.91 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991237.893939 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217511.18 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 21751.12 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012989.012141 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195760.06 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 19576.01 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032565.018523 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176184.06 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 17618.41 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050183.424267 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158565.65 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 15856.57 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066039.989437 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142709.09 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 14270.91 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080310.898089 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128438.18 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 12843.82 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093154.715877 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115594.36 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 11559.44 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104714.151885 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104034.92 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 10403.49 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115117.644293 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93631.43 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 9363.14 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124480.787460 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84268.29 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 8426.83 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132907.616310 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75841.46 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 7584.15 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140491.762275 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68257.31 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 6825.73 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147317.493644 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61431.58 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 6143.16 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153460.651876 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55288.42 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 5528.84 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158989.494284 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49759.58 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 4975.96 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163965.452452 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44783.62 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 4478.36 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168443.814803 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40305.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 4030.53 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172474.340919 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36274.74 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3627.47 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176101.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32647.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179701.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29047.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183301.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25447.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186901.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21847.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190501.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18247.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194101.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14647.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197701.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11047.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201301.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7447.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204901.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3847.26 sec) 19:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208501.814423 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 247.26 sec) 19:56:00 [GPUGRID] [rr_sim] 2208749.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971361210.25G/439.78G) 19:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853549.99 19:56:00 [---] pick_jobs_to_run() start 19:56:00 [---] pick_jobs_to_run() end 19:56:00 [---] [rr_sim] end 19:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:56:00 [---] [cpu_sched_debug] preliminary job list: 19:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:56:00 [---] [cpu_sched_debug] final job list: 19:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:56:00 [---] [cpu_sched_debug] enforce_run_list: end 19:56:00 [---] choose_project(): 71760.000000 19:56:00 [WUProp@Home] doing RPC to get work 19:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363839.09 cpu 2112392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112392.90 19:56:00 [---] Got 0 tasks 19:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:57:00 [---] [rr_sim] doing sim: CPU sched 19:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971333619.90G/439.78G) 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208686.34 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 220868.63 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220868.633908 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987817.71 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 198781.77 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419650.404425 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1789035.93 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 178903.59 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598553.997891 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610132.34 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 161013.23 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759567.232010 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449119.11 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 144911.91 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904479.142717 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304207.20 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 130420.72 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034899.862353 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173786.48 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 117378.65 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152278.510026 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056407.83 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 105640.78 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257919.292931 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950767.05 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 95076.70 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352995.997546 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855690.34 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 85569.03 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438565.031699 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770121.31 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 77012.13 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515577.162438 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693109.18 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 69310.92 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584888.080102 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623798.26 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 62379.83 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647267.906000 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561418.43 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 56141.84 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703409.749308 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505276.59 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 50527.66 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753937.408285 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454748.93 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 45474.89 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799412.301364 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409274.04 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 40927.40 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840339.705136 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368346.63 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 36834.66 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877174.368530 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331511.97 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 33151.20 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910325.565585 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298360.77 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 29836.08 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940161.642935 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268524.70 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 26852.47 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1967014.112549 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241672.23 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 24167.22 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991181.335202 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217505.00 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 21750.50 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012931.835590 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195754.50 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 19575.45 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032507.285939 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176179.05 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 17617.91 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050125.191253 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158561.15 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 15856.11 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065981.306036 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142705.03 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 14270.50 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080251.809340 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128434.53 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 12843.45 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093095.262314 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115591.08 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 11559.11 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104654.369991 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104031.97 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 10403.20 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115057.566900 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93628.77 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 9362.88 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124420.444118 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84265.89 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 8426.59 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132847.033614 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75839.31 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 7583.93 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140430.964161 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68255.37 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 6825.54 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147256.501653 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61429.84 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 6142.98 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153399.485395 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55286.85 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 5528.69 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158928.170764 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49758.17 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 4975.82 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163903.987595 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44782.35 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 4478.24 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168382.222744 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40304.12 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 4030.41 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172412.634378 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36273.70 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3627.37 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176040.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32646.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179640.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29046.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183240.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25446.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186840.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21846.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190440.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18246.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194040.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14646.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197640.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11046.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201240.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7446.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204840.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3846.33 sec) 19:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208440.004848 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 246.33 sec) 19:57:00 [GPUGRID] [rr_sim] 2208686.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971333619.90G/439.78G) 19:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853547.25 19:57:00 [---] pick_jobs_to_run() start 19:57:00 [---] pick_jobs_to_run() end 19:57:00 [---] [rr_sim] end 19:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:57:00 [---] [cpu_sched_debug] preliminary job list: 19:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:57:00 [---] [cpu_sched_debug] final job list: 19:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:57:00 [---] [cpu_sched_debug] enforce_run_list: end 19:57:00 [---] choose_project(): 71820.000000 19:57:00 [WUProp@Home] doing RPC to get work 19:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363779.09 cpu 2112332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112332.90 19:57:00 [---] Got 0 tasks 19:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:58:00 [---] [rr_sim] doing sim: CPU sched 19:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971306029.54G/439.78G) 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208623.60 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 220862.36 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220862.360220 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987761.24 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 198776.12 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419638.484418 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788985.12 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 178898.51 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598536.996196 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610086.61 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 161008.66 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759545.656796 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449077.95 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 144907.79 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904453.451336 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304170.15 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 130417.02 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034870.466422 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173753.14 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 117375.31 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152245.780000 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056377.82 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 105637.78 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257883.562220 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950740.04 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 95074.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352957.566217 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855666.04 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 85566.60 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438524.169815 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770099.43 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 77009.94 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515534.113054 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693089.49 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 69308.95 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584843.061968 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623780.54 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 62378.05 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647221.115991 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561402.49 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 56140.25 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703361.364612 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505262.24 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 50526.22 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753887.588370 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454736.01 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 45473.60 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799361.189753 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409262.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 40926.24 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840287.430998 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368336.17 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 36833.62 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877121.048118 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331502.55 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 33150.26 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910271.303526 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298352.30 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 29835.23 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940106.533393 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268517.07 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 26851.71 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966958.240273 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241665.36 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 24166.54 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991124.776466 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217498.83 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 21749.88 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012874.659039 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195748.94 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 19574.89 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032449.553355 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176174.05 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 17617.40 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050066.958239 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158556.64 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 15855.66 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065922.622635 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142700.98 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 14270.10 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080192.720591 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128430.88 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 12843.09 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093035.808752 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115587.79 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 11558.78 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104594.588097 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104029.01 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 10402.90 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114997.489507 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93626.11 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 9362.61 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124360.100776 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84263.50 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 8426.35 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132786.450918 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75837.15 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 7583.72 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140370.166046 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68253.44 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 6825.34 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147195.509661 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61428.09 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 6142.81 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153338.318915 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55285.28 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 5528.53 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158866.847243 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49756.75 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 4975.68 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163842.522739 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44781.08 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 4478.11 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168320.630685 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40302.97 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 4030.30 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172350.927836 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36272.67 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3627.27 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175978.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32645.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179578.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29045.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183178.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25445.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186778.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21845.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190378.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18245.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193978.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14645.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197578.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11045.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201178.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7445.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204778.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3845.41 sec) 19:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208378.195272 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 245.41 sec) 19:58:00 [GPUGRID] [rr_sim] 2208623.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971306029.54G/439.78G) 19:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853544.51 19:58:00 [---] pick_jobs_to_run() start 19:58:00 [---] pick_jobs_to_run() end 19:58:00 [---] [rr_sim] end 19:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:58:00 [---] [cpu_sched_debug] preliminary job list: 19:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:58:00 [---] [cpu_sched_debug] final job list: 19:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:58:00 [---] [cpu_sched_debug] enforce_run_list: end 19:58:00 [---] choose_project(): 71880.000000 19:58:00 [WUProp@Home] doing RPC to get work 19:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363719.09 cpu 2112272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112272.90 19:58:00 [---] Got 0 tasks 19:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 19:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 19:59:00 [---] [rr_sim] doing sim: CPU sched 19:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971278439.18G/439.78G) 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208560.87 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 220856.09 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220856.086532 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987704.78 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 198770.48 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419626.564410 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788934.30 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 178893.43 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598519.994501 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1610040.87 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 161004.09 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759524.081582 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1449036.78 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 144903.68 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904427.759955 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304133.11 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 130413.31 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034841.070491 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173719.79 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 117371.98 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152213.049974 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056347.82 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 105634.78 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257847.831508 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950713.03 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 95071.30 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352919.134889 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855641.73 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 85564.17 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438483.307932 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770077.56 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 77007.76 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515491.063670 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693069.80 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 69306.98 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584798.043835 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623762.82 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 62376.28 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647174.325983 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561386.54 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 56138.65 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703312.979916 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505247.89 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 50524.79 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753837.768456 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454723.10 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 45472.31 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799310.078142 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409250.79 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 40925.08 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840235.156859 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368325.71 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 36832.57 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877067.727705 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331493.14 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 33149.31 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910217.041466 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298343.82 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 29834.38 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1940051.423851 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268509.44 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 26850.94 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966902.367997 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241658.50 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 24165.85 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991068.217729 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217492.65 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 21749.26 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012817.482488 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195743.38 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 19574.34 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032391.820771 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176169.04 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 17616.90 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050008.725225 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158552.14 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 15855.21 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065863.939234 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142696.93 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 14269.69 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080133.631842 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128427.23 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 12842.72 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092976.355190 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115584.51 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 11558.45 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104534.806202 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104026.06 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 10402.61 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114937.412114 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93623.45 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 9362.35 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124299.757434 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84261.11 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 8426.11 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132725.868222 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75835.00 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 7583.50 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140309.367931 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68251.50 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 6825.15 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147134.517670 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61426.35 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 6142.63 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153277.152434 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55283.71 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 5528.37 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158805.523723 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49755.34 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 4975.53 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163781.057882 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44779.81 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 4477.98 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168259.038625 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40301.83 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 4030.18 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172289.221294 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36271.64 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3627.16 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175916.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32644.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179516.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29044.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183116.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25444.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186716.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21844.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190316.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18244.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193916.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14644.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197516.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11044.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201116.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7444.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204716.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3844.48 sec) 19:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208316.385696 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 244.48 sec) 19:59:00 [GPUGRID] [rr_sim] 2208560.87: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971278439.18G/439.78G) 19:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853541.78 19:59:00 [---] pick_jobs_to_run() start 19:59:00 [---] pick_jobs_to_run() end 19:59:00 [---] [rr_sim] end 19:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 19:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 19:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 19:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 19:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 19:59:00 [---] [cpu_sched_debug] preliminary job list: 19:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:59:00 [---] [cpu_sched_debug] final job list: 19:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 19:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 19:59:00 [---] [cpu_sched_debug] enforce_run_list: end 19:59:00 [---] choose_project(): 71940.000000 19:59:00 [WUProp@Home] doing RPC to get work 19:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363659.09 cpu 2112212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112212.90 19:59:00 [---] Got 0 tasks 20:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:00:00 [---] [rr_sim] doing sim: CPU sched 20:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971250848.83G/439.78G) 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208498.13 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 220849.81 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220849.812843 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987648.32 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 198764.83 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419614.644402 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788883.48 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 178888.35 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598502.992805 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609995.14 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 160999.51 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759502.506368 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448995.62 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 144899.56 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904402.068575 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304096.06 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 130409.61 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034811.674561 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173686.45 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 117368.65 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152180.319948 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056317.81 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 105631.78 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257812.100797 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950686.03 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 95068.60 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352880.703560 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855617.42 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 85561.74 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438442.446048 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770055.68 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 77005.57 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515448.014286 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693050.11 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 69305.01 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584753.025701 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623745.10 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 62374.51 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647127.535974 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561370.59 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 56137.06 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703264.595220 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505233.53 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 50523.35 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753787.948541 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454710.18 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 45471.02 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799258.966531 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409239.16 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 40923.92 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840182.882721 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368315.25 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 36831.52 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1877014.407292 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331483.72 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 33148.37 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910162.779406 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298335.35 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 29833.53 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939996.314309 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268501.81 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 26850.18 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966846.495722 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241651.63 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 24165.16 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1991011.658993 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217486.47 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 21748.65 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012760.305937 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195737.82 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 19573.78 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032334.088186 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176164.04 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 17616.40 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049950.492211 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158547.64 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 15854.76 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065805.255833 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142692.87 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 14269.29 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080074.543093 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128423.59 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 12842.36 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092916.901627 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115581.23 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 11558.12 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104475.024308 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104023.10 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 10402.31 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114877.334721 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93620.79 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 9362.08 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124239.414092 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84258.71 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 8425.87 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132665.285526 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75832.84 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 7583.28 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140248.569817 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68249.56 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 6824.96 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147073.525678 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61424.60 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 6142.46 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153215.985954 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55282.14 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 5528.21 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158744.200202 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49753.93 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 4975.39 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163719.593025 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44778.54 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 4477.85 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168197.446566 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40300.68 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 4030.07 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172227.514753 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36270.61 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3627.06 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175854.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32643.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179454.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29043.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183054.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25443.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186654.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21843.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190254.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18243.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193854.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14643.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197454.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11043.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201054.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7443.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204654.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3843.55 sec) 20:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208254.576121 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 243.55 sec) 20:00:00 [GPUGRID] [rr_sim] 2208498.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971250848.83G/439.78G) 20:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853539.04 20:00:00 [---] pick_jobs_to_run() start 20:00:00 [---] pick_jobs_to_run() end 20:00:00 [---] [rr_sim] end 20:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:00:00 [---] [cpu_sched_debug] preliminary job list: 20:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:00:00 [---] [cpu_sched_debug] final job list: 20:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:00:00 [---] [cpu_sched_debug] enforce_run_list: end 20:00:00 [---] choose_project(): 72000.000000 20:00:00 [WUProp@Home] doing RPC to get work 20:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363599.09 cpu 2112152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112152.90 20:00:00 [---] Got 0 tasks 20:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:01:00 [---] [rr_sim] doing sim: CPU sched 20:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971223258.47G/439.78G) 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208435.39 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 220843.54 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220843.539155 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987591.85 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 198759.19 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419602.724395 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788832.67 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 178883.27 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598485.991110 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609949.40 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 160994.94 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759480.931155 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448954.46 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 144895.45 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904376.377194 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304059.01 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 130405.90 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034782.278630 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173653.11 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 117365.31 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152147.589922 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056287.80 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 105628.78 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257776.370085 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950659.02 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 95065.90 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352842.272232 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855593.12 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 85559.31 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438401.584164 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770033.81 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 77003.38 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515404.964902 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693030.43 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 69303.04 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584708.007567 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623727.38 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 62372.74 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647080.745966 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561354.65 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 56135.46 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703216.210524 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505219.18 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 50521.92 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753738.128627 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454697.26 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 45469.73 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799207.854919 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409227.54 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 40922.75 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840130.608583 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368304.78 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 36830.48 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876961.086880 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331474.30 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 33147.43 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910108.517347 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298326.87 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 29832.69 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939941.204767 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268494.19 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 26849.42 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966790.623446 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241644.77 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 24164.48 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990955.100256 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217480.29 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 21748.03 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012703.129386 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195732.26 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 19573.23 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032276.355602 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176159.04 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 17615.90 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049892.259197 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158543.13 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 15854.31 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065746.572433 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142688.82 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 14268.88 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080015.454344 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128419.94 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 12841.99 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092857.448065 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115577.94 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 11557.79 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104415.242414 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104020.15 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 10402.01 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114817.257328 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93618.13 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 9361.81 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124179.070750 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84256.32 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 8425.63 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132604.702830 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75830.69 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 7583.07 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140187.771702 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68247.62 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 6824.76 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147012.533687 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61422.86 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 6142.29 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153154.819474 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55280.57 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 5528.06 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158682.876681 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49752.51 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 4975.25 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163658.128168 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44777.26 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 4477.73 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168135.854507 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40299.54 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 4029.95 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172165.808211 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36269.58 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3626.96 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175792.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32642.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179392.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29042.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182992.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25442.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186592.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21842.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190192.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18242.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193792.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14642.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197392.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11042.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200992.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7442.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204592.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3842.63 sec) 20:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208192.766545 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 242.63 sec) 20:01:00 [GPUGRID] [rr_sim] 2208435.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971223258.47G/439.78G) 20:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853536.30 20:01:00 [---] pick_jobs_to_run() start 20:01:00 [---] pick_jobs_to_run() end 20:01:00 [---] [rr_sim] end 20:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:01:00 [---] [cpu_sched_debug] preliminary job list: 20:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:01:00 [---] [cpu_sched_debug] final job list: 20:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:01:00 [---] [cpu_sched_debug] enforce_run_list: end 20:01:00 [---] choose_project(): 72060.000000 20:01:00 [WUProp@Home] doing RPC to get work 20:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363539.09 cpu 2112092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112092.90 20:01:00 [---] Got 0 tasks 20:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:05:00 [---] [rr_sim] doing sim: CPU sched 20:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971195668.12G/439.78G) 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208372.65 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 220837.27 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220837.265467 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987535.39 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 198753.54 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419590.804387 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788781.85 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 178878.19 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598468.989415 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609903.67 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 160990.37 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759459.355941 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448913.30 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 144891.33 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904350.685814 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1304021.97 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 130402.20 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034752.882699 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173619.77 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 117361.98 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152114.859896 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056257.79 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 105625.78 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257740.639373 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950632.02 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 95063.20 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352803.840903 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855568.81 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 85556.88 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438360.722280 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 770011.93 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 77001.19 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515361.915519 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 693010.74 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 69301.07 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584662.989434 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623709.67 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 62370.97 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1647033.955957 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561338.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 56133.87 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703167.825828 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505204.83 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 50520.48 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753688.308713 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454684.35 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 45468.43 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799156.743308 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409215.91 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 40921.59 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840078.334444 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368294.32 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 36829.43 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876907.766467 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331464.89 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 33146.49 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1910054.255287 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298318.40 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 29831.84 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939886.095225 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268486.56 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 26848.66 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966734.751170 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241637.90 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 24163.79 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990898.541520 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217474.11 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 21747.41 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012645.952835 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195726.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 19572.67 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032218.623018 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176154.03 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 17615.40 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049834.026183 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158538.63 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 15853.86 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065687.889032 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142684.77 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 14268.48 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079956.365596 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128416.29 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 12841.63 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092797.994503 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115574.66 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 11557.47 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104355.460520 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104017.19 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 10401.72 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114757.179934 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93615.47 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 9361.55 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124118.727408 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84253.93 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 8425.39 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132544.120134 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75828.53 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 7582.85 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140126.973588 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68245.68 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 6824.57 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146951.541696 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61421.11 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 6142.11 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153093.652993 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55279.00 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 5527.90 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158621.553161 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49751.10 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 4975.11 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163596.663312 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44775.99 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 4477.60 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168074.262447 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40298.39 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 4029.84 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172104.101670 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36268.55 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3626.86 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175730.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32641.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179330.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29041.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182930.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25441.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186530.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21841.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190130.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18241.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193730.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14641.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197330.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11041.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200930.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7441.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204530.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3841.70 sec) 20:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208130.956969 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241.70 sec) 20:05:00 [GPUGRID] [rr_sim] 2208372.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971195668.12G/439.78G) 20:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853713.57 20:05:00 [---] pick_jobs_to_run() start 20:05:00 [---] pick_jobs_to_run() end 20:05:00 [---] [rr_sim] end 20:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:05:00 [---] [cpu_sched_debug] preliminary job list: 20:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:05:00 [---] [cpu_sched_debug] final job list: 20:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:05:00 [---] [cpu_sched_debug] enforce_run_list: end 20:05:00 [---] choose_project(): 72300.000000 20:05:00 [WUProp@Home] doing RPC to get work 20:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363299.09 cpu 2112032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2112032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2112032.90 20:05:00 [---] Got 0 tasks 20:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:06:00 [---] [rr_sim] doing sim: CPU sched 20:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971168077.76G/439.78G) 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208309.92 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 220830.99 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220830.991779 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987478.93 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 198747.89 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419578.884380 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788731.03 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 178873.10 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598451.987720 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609857.93 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 160985.79 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759437.780727 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448872.14 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 144887.21 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904324.994433 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303984.92 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 130398.49 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034723.486768 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173586.43 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 117358.64 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152082.129870 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056227.79 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 105622.78 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257704.908662 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950605.01 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 95060.50 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352765.409574 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855544.51 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 85554.45 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438319.860396 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769990.06 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 76999.01 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515318.866135 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692991.05 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 69299.11 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584617.971300 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623691.95 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 62369.19 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646987.165949 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561322.75 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 56132.28 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703119.441133 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505190.48 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 50519.05 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753638.488798 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454671.43 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 45467.14 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799105.631697 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409204.29 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 40920.43 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1840026.060306 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368283.86 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 36828.39 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876854.446054 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331455.47 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 33145.55 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909999.993227 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298309.92 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 29830.99 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939830.985683 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268478.93 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 26847.89 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966678.878894 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241631.04 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 24163.10 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990841.982783 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217467.94 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 21746.79 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012588.776283 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195721.14 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 19572.11 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032160.890434 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176149.03 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 17614.90 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049775.793169 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158534.12 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 15853.41 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065629.205631 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142680.71 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 14268.07 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079897.276847 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128412.64 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 12841.26 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092738.540941 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115571.38 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 11557.14 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104295.678625 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104014.24 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 10401.42 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114697.102541 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93612.82 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 9361.28 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124058.384066 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84251.53 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 8425.15 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132483.537438 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75826.38 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 7582.64 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140066.175473 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68243.74 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 6824.37 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146890.549704 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61419.37 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 6141.94 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153032.486513 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55277.43 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 5527.74 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158560.229640 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49749.69 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 4974.97 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163535.198455 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44774.72 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 4477.47 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168012.670388 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40297.25 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 4029.72 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172042.395128 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36267.52 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3626.75 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175669.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32640.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179269.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29040.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182869.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25440.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186469.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21840.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190069.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18240.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193669.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14640.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197269.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11040.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200869.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7440.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204469.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3840.77 sec) 20:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208069.147394 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240.77 sec) 20:06:00 [GPUGRID] [rr_sim] 2208309.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971168077.76G/439.78G) 20:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853710.83 20:06:00 [---] pick_jobs_to_run() start 20:06:00 [---] pick_jobs_to_run() end 20:06:00 [---] [rr_sim] end 20:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:06:00 [---] [cpu_sched_debug] preliminary job list: 20:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:06:00 [---] [cpu_sched_debug] final job list: 20:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:06:00 [---] [cpu_sched_debug] enforce_run_list: end 20:06:00 [---] choose_project(): 72360.000000 20:06:00 [WUProp@Home] doing RPC to get work 20:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363239.09 cpu 2111972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111972.90 20:06:00 [---] Got 0 tasks 20:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:07:00 [---] [rr_sim] doing sim: CPU sched 20:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971140487.40G/439.78G) 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208247.18 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 220824.72 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220824.718090 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987422.46 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 198742.25 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419566.964372 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788680.22 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 178868.02 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598434.986025 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609812.19 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 160981.22 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759416.205513 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448830.98 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 144883.10 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904299.303052 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303947.88 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 130394.79 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034694.090838 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173553.09 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 117355.31 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152049.399844 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056197.78 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 105619.78 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257669.177950 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950578.00 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 95057.80 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352726.978246 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855520.20 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 85552.02 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438278.998512 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769968.18 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 76996.82 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515275.816751 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692971.36 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 69297.14 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584572.953166 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623674.23 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 62367.42 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646940.375940 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561306.80 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 56130.68 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703071.056437 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505176.12 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 50517.61 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753588.668884 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454658.51 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 45465.85 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799054.520086 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409192.66 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 40919.27 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839973.786168 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368273.39 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 36827.34 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876801.125641 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331446.06 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 33144.61 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909945.731168 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298301.45 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 29830.14 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939775.876141 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268471.30 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 26847.13 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966623.006618 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241624.17 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 24162.42 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990785.424046 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217461.76 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 21746.18 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012531.599732 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195715.58 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 19571.56 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032103.157850 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176144.02 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 17614.40 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049717.560155 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158529.62 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 15852.96 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065570.522230 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142676.66 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 14267.67 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079838.188098 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128408.99 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 12840.90 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092679.087378 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115568.09 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 11556.81 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104235.896731 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104011.28 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 10401.13 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114637.025148 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93610.16 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 9361.02 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123998.040724 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84249.14 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 8424.91 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132422.954742 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75824.23 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 7582.42 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140005.377358 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68241.80 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 6824.18 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146829.557713 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61417.62 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 6141.76 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152971.320032 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55275.86 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 5527.59 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158498.906119 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49748.27 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 4974.83 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163473.733598 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44773.45 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 4477.34 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167951.078329 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40296.10 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 4029.61 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171980.688586 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36266.49 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3626.65 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175607.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32639.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179207.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29039.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182807.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25439.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186407.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21839.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190007.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18239.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193607.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14639.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197207.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11039.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200807.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7439.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204407.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3839.84 sec) 20:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2208007.337818 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239.84 sec) 20:07:00 [GPUGRID] [rr_sim] 2208247.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971140487.40G/439.78G) 20:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853708.09 20:07:00 [---] pick_jobs_to_run() start 20:07:00 [---] pick_jobs_to_run() end 20:07:00 [---] [rr_sim] end 20:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:07:00 [---] [cpu_sched_debug] preliminary job list: 20:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:07:00 [---] [cpu_sched_debug] final job list: 20:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:07:00 [---] [cpu_sched_debug] enforce_run_list: end 20:07:00 [---] choose_project(): 72420.000000 20:07:00 [WUProp@Home] doing RPC to get work 20:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363179.09 cpu 2111912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111912.90 20:07:00 [---] Got 0 tasks 20:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:08:00 [---] [rr_sim] doing sim: CPU sched 20:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971112897.05G/439.78G) 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208184.44 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 220818.44 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220818.444402 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987366.00 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 198736.60 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419555.044364 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788629.40 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 178862.94 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598417.984330 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609766.46 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 160976.65 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759394.630299 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448789.81 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 144878.98 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904273.611672 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303910.83 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 130391.08 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034664.694907 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173519.75 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 117351.97 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1152016.669818 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056167.77 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 105616.78 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257633.447239 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950551.00 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 95055.10 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352688.546917 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855495.90 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 85549.59 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438238.136628 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769946.31 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 76994.63 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515232.767367 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692951.68 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 69295.17 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584527.935033 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623656.51 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 62365.65 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646893.585932 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561290.86 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 56129.09 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1703022.671741 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505161.77 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 50516.18 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753538.848969 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454645.60 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 45464.56 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1799003.408474 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409181.04 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 40918.10 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839921.512029 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368262.93 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 36826.29 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876747.805229 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331436.64 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 33143.66 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909891.469108 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298292.97 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 29829.30 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939720.766600 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268463.68 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 26846.37 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966567.134342 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241617.31 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 24161.73 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990728.865310 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217455.58 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 21745.56 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012474.423181 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195710.02 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 19571.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2032045.425265 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176139.02 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 17613.90 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049659.327141 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158525.12 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 15852.51 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065511.838829 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142672.61 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 14267.26 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079779.099349 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128405.34 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 12840.53 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092619.633816 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115564.81 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 11556.48 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104176.114837 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104008.33 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 10400.83 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114576.947755 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93607.50 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 9360.75 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123937.697382 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84246.75 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 8424.67 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132362.372046 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75822.07 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 7582.21 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139944.579244 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68239.86 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 6823.99 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146768.565722 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61415.88 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 6141.59 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152910.153552 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55274.29 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 5527.43 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158437.582599 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49746.86 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 4974.69 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163412.268741 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44772.18 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 4477.22 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167889.486269 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40294.96 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 4029.50 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171918.982045 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36265.46 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3626.55 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175545.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32638.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179145.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29038.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182745.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25438.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186345.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21838.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189945.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18238.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193545.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14638.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197145.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11038.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200745.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7438.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204345.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3838.92 sec) 20:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207945.528243 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 238.92 sec) 20:08:00 [GPUGRID] [rr_sim] 2208184.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971112897.05G/439.78G) 20:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853705.36 20:08:00 [---] pick_jobs_to_run() start 20:08:00 [---] pick_jobs_to_run() end 20:08:00 [---] [rr_sim] end 20:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:08:00 [---] [cpu_sched_debug] preliminary job list: 20:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:08:00 [---] [cpu_sched_debug] final job list: 20:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:08:00 [---] [cpu_sched_debug] enforce_run_list: end 20:08:00 [---] choose_project(): 72480.000000 20:08:00 [WUProp@Home] doing RPC to get work 20:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363119.09 cpu 2111852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111852.90 20:08:00 [---] Got 0 tasks 20:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:09:00 [---] [rr_sim] doing sim: CPU sched 20:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971085306.69G/439.78G) 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208121.71 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 220812.17 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220812.170714 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987309.54 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 198730.95 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419543.124357 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788578.58 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 178857.86 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598400.982635 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609720.72 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 160972.07 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759373.055086 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448748.65 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 144874.87 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904247.920291 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303873.79 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 130387.38 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034635.298976 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173486.41 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 117348.64 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151983.939793 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056137.77 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 105613.78 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257597.716527 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950523.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 95052.40 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352650.115589 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855471.59 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 85547.16 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438197.274744 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769924.43 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 76992.44 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515189.717984 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692931.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 69293.20 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584482.916899 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623638.79 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 62363.88 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646846.795923 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561274.91 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 56127.49 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702974.287045 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505147.42 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 50514.74 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753489.029055 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454632.68 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 45463.27 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798952.296863 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409169.41 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 40916.94 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839869.237891 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368252.47 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 36825.25 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876694.484816 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331427.22 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 33142.72 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909837.207048 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298284.50 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 29828.45 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939665.657058 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268456.05 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 26845.61 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966511.262066 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241610.45 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 24161.04 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990672.306573 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217449.40 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 21744.94 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012417.246630 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195704.46 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 19570.45 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031987.692681 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176134.01 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 17613.40 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049601.094127 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158520.61 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 15852.06 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065453.155428 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142668.55 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 14266.86 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079720.010600 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128401.70 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 12840.17 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092560.180254 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115561.53 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 11556.15 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104116.332942 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104005.37 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 10400.54 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114516.870362 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93604.84 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 9360.48 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123877.354040 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84244.35 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 8424.44 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132301.789350 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75819.92 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 7581.99 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139883.781129 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68237.93 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 6823.79 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146707.573730 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61414.13 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 6141.41 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152848.987071 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55272.72 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 5527.27 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158376.259078 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49745.45 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 4974.54 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163350.803884 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44770.90 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 4477.09 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167827.894210 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40293.81 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 4029.38 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171857.275503 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36264.43 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3626.44 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175483.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32637.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179083.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29037.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182683.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25437.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186283.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21837.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189883.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18237.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193483.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14637.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197083.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11037.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200683.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7437.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204283.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3837.99 sec) 20:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207883.718667 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 237.99 sec) 20:09:00 [GPUGRID] [rr_sim] 2208121.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971085306.69G/439.78G) 20:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853702.62 20:09:00 [---] pick_jobs_to_run() start 20:09:00 [---] pick_jobs_to_run() end 20:09:00 [---] [rr_sim] end 20:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:09:00 [---] [cpu_sched_debug] preliminary job list: 20:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:09:00 [---] [cpu_sched_debug] final job list: 20:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:09:00 [---] [cpu_sched_debug] enforce_run_list: end 20:09:00 [---] choose_project(): 72540.000000 20:09:00 [WUProp@Home] doing RPC to get work 20:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 363059.09 cpu 2111792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111792.90 20:09:00 [---] Got 0 tasks 20:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:10:00 [---] [rr_sim] doing sim: CPU sched 20:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971057716.33G/439.78G) 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2208058.97 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 220805.90 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220805.897026 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987253.07 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 198725.31 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419531.204349 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788527.77 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 178852.78 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598383.980940 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609674.99 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 160967.50 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759351.479872 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448707.49 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 144870.75 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904222.228911 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303836.74 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 130383.67 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034605.903045 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173453.07 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 117345.31 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151951.209767 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056107.76 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 105610.78 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257561.985816 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950496.98 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 95049.70 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352611.684260 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855447.29 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 85544.73 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438156.412860 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769902.56 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 76990.26 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515146.668600 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692912.30 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 69291.23 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584437.898766 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623621.07 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 62362.11 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646800.005915 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561258.96 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 56125.90 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702925.902349 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505133.07 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 50513.31 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753439.209140 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454619.76 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 45461.98 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798901.185252 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409157.79 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 40915.78 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839816.963753 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368242.01 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 36824.20 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876641.164403 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331417.81 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 33141.78 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909782.944989 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298276.03 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 29827.60 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939610.547516 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268448.42 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 26844.84 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966455.389790 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241603.58 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 24160.36 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990615.747837 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217443.22 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 21744.32 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012360.070079 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195698.90 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 19569.89 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031929.960097 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176129.01 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 17612.90 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049542.861113 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158516.11 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 15851.61 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065394.472028 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142664.50 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 14266.45 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079660.921851 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128398.05 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 12839.80 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092500.726691 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115558.24 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 11555.82 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104056.551048 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104002.42 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 10400.24 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114456.792969 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93602.18 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 9360.22 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123817.010698 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84241.96 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 8424.20 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132241.206654 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75817.76 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 7581.78 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139822.983015 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68235.99 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 6823.60 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146646.581739 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61412.39 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 6141.24 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152787.820591 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55271.15 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 5527.11 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158314.935558 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49744.03 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 4974.40 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163289.339028 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44769.63 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 4476.96 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167766.302151 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40292.67 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 4029.27 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171795.568962 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36263.40 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3626.34 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175421.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32637.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179021.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29037.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182621.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25437.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186221.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21837.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189821.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18237.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193421.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14637.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197021.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11037.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200621.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7437.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204221.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3837.06 sec) 20:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207821.909091 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 237.06 sec) 20:10:00 [GPUGRID] [rr_sim] 2208058.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971057716.33G/439.78G) 20:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853699.88 20:10:00 [---] pick_jobs_to_run() start 20:10:00 [---] pick_jobs_to_run() end 20:10:00 [---] [rr_sim] end 20:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:10:00 [---] [cpu_sched_debug] preliminary job list: 20:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:10:00 [---] [cpu_sched_debug] final job list: 20:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:10:00 [---] [cpu_sched_debug] enforce_run_list: end 20:10:00 [---] choose_project(): 72600.000000 20:10:00 [WUProp@Home] doing RPC to get work 20:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362999.09 cpu 2111732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111732.90 20:10:00 [---] Got 0 tasks 20:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:11:00 [---] [rr_sim] doing sim: CPU sched 20:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971030125.98G/439.78G) 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207996.23 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 220799.62 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220799.623338 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987196.61 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 198719.66 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419519.284341 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788476.95 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 178847.69 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598366.979245 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609629.25 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 160962.93 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759329.904658 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448666.33 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 144866.63 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904196.537530 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303799.70 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 130379.97 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034576.507115 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173419.73 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 117341.97 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151918.479741 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056077.75 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 105607.78 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257526.255104 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950469.98 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 95047.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352573.252931 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855422.98 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 85542.30 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438115.550976 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769880.68 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 76988.07 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515103.619216 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692892.61 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 69289.26 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584392.880632 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623603.35 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 62360.34 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646753.215906 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561243.02 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 56124.30 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702877.517653 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505118.72 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 50511.87 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753389.389226 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454606.84 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 45460.68 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798850.073641 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409146.16 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 40914.62 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839764.689614 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368231.54 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 36823.15 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876587.843990 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331408.39 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 33140.84 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909728.682929 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298267.55 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 29826.76 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939555.437974 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268440.80 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 26844.08 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966399.517514 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241596.72 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 24159.67 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990559.189100 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217437.04 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 21743.70 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012302.893528 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195693.34 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 19569.33 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031872.227513 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176124.01 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 17612.40 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049484.628099 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158511.61 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 15851.16 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065335.788627 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142660.44 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 14266.04 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079601.833102 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128394.40 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 12839.44 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092441.273129 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115554.96 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 11555.50 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103996.769154 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103999.46 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 10399.95 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114396.715576 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93599.52 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 9359.95 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123756.667356 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84239.57 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 8423.96 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132180.623958 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75815.61 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 7581.56 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139762.184900 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68234.05 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 6823.40 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146585.589748 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61410.64 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 6141.06 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152726.654110 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55269.58 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 5526.96 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158253.612037 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49742.62 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 4974.26 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163227.874171 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44768.36 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 4476.84 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167704.710091 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40291.52 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 4029.15 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171733.862420 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36262.37 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3626.24 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175360.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32636.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178960.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29036.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182560.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25436.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186160.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21836.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189760.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18236.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193360.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14636.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196960.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11036.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200560.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7436.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204160.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3836.13 sec) 20:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207760.099516 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 236.13 sec) 20:11:00 [GPUGRID] [rr_sim] 2207996.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971030125.98G/439.78G) 20:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853697.15 20:11:00 [---] pick_jobs_to_run() start 20:11:00 [---] pick_jobs_to_run() end 20:11:00 [---] [rr_sim] end 20:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:11:00 [---] [cpu_sched_debug] preliminary job list: 20:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:11:00 [---] [cpu_sched_debug] final job list: 20:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:11:00 [---] [cpu_sched_debug] enforce_run_list: end 20:11:00 [---] choose_project(): 72660.000000 20:11:00 [WUProp@Home] doing RPC to get work 20:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362939.09 cpu 2111672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111672.90 20:11:00 [---] Got 0 tasks 20:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:12:00 [---] [rr_sim] doing sim: CPU sched 20:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (971002535.62G/439.78G) 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207933.50 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 220793.35 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220793.349649 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987140.15 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 198714.01 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419507.364334 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788426.13 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 178842.61 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598349.977550 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609583.52 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 160958.35 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759308.329444 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448625.17 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 144862.52 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904170.846149 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303762.65 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 130376.27 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034547.111184 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173386.39 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 117338.64 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151885.749715 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056047.75 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 105604.77 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257490.524393 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950442.97 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 95044.30 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352534.821603 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855398.67 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 85539.87 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438074.689092 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769858.81 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 76985.88 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515060.569832 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692872.93 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 69287.29 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584347.862498 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623585.63 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 62358.56 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646706.425898 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561227.07 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 56122.71 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702829.132958 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505104.36 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 50510.44 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753339.569311 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454593.93 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 45459.39 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798798.962030 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409134.53 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 40913.45 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839712.415476 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368221.08 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 36822.11 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876534.523578 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331398.97 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 33139.90 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909674.420869 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298259.08 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 29825.91 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939500.328432 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268433.17 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 26843.32 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966343.645238 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241589.85 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 24158.99 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990502.630364 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217430.87 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 21743.09 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012245.716977 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195687.78 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 19568.78 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031814.494928 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176119.00 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 17611.90 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049426.395085 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158507.10 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 15850.71 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065277.105226 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142656.39 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 14265.64 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079542.744353 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128390.75 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 12839.08 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092381.819567 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115551.68 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 11555.17 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103936.987260 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103996.51 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 10399.65 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114336.638183 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93596.86 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 9359.69 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123696.324014 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84237.17 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 8423.72 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132120.041262 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75813.46 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 7581.35 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139701.386785 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68232.11 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 6823.21 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146524.597756 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61408.90 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 6140.89 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152665.487630 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55268.01 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 5526.80 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158192.288516 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49741.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 4974.12 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163166.409314 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44767.09 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 4476.71 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167643.118032 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40290.38 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 4029.04 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171672.155878 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36261.34 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3626.13 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175298.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32635.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178898.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29035.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182498.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25435.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186098.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21835.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189698.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18235.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193298.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14635.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196898.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11035.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200498.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7435.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204098.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3835.21 sec) 20:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207698.289940 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 235.21 sec) 20:12:00 [GPUGRID] [rr_sim] 2207933.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (971002535.62G/439.78G) 20:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853694.41 20:12:00 [---] pick_jobs_to_run() start 20:12:00 [---] pick_jobs_to_run() end 20:12:00 [---] [rr_sim] end 20:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:12:00 [---] [cpu_sched_debug] preliminary job list: 20:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:12:00 [---] [cpu_sched_debug] final job list: 20:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:12:00 [---] [cpu_sched_debug] enforce_run_list: end 20:12:00 [---] choose_project(): 72720.000000 20:12:00 [WUProp@Home] doing RPC to get work 20:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362879.09 cpu 2111612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111612.90 20:12:00 [---] Got 0 tasks 20:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:13:00 [---] [rr_sim] doing sim: CPU sched 20:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970974945.27G/439.78G) 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207870.76 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 220787.08 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220787.075961 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987083.68 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 198708.37 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419495.444326 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788375.32 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 178837.53 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598332.975855 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609537.78 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 160953.78 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759286.754231 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448584.01 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 144858.40 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904145.154769 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303725.60 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 130372.56 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034517.715253 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173353.04 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 117335.30 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151853.019689 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1056017.74 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 105601.77 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257454.793681 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950415.97 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 95041.60 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352496.390274 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855374.37 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 85537.44 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1438033.827208 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769836.93 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 76983.69 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1515017.520448 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692853.24 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 69285.32 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584302.844365 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623567.92 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 62356.79 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646659.635889 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561211.12 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 56121.11 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702780.748262 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505090.01 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 50509.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753289.749397 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454581.01 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 45458.10 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798747.850418 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409122.91 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 40912.29 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839660.141338 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368210.62 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 36821.06 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876481.203165 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331389.56 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 33138.96 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909620.158810 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298250.60 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 29825.06 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939445.218890 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268425.54 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 26842.55 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966287.772962 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241582.99 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 24158.30 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990446.071627 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217424.69 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 21742.47 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012188.540426 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195682.22 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 19568.22 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031756.762344 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176114.00 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 17611.40 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049368.162071 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158502.60 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 15850.26 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065218.421825 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142652.34 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 14265.23 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079483.655604 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128387.10 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 12838.71 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092322.366005 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115548.39 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 11554.84 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103877.205365 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103993.55 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 10399.36 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114276.560790 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93594.20 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 9359.42 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123635.980672 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84234.78 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 8423.48 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132059.458566 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75811.30 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 7581.13 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139640.588671 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68230.17 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 6823.02 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146463.605765 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61407.15 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 6140.72 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152604.321150 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55266.44 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 5526.64 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158130.964996 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49739.79 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 4973.98 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163104.944457 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44765.82 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 4476.58 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167581.525973 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40289.23 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 4028.92 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171610.449337 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36260.31 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3626.03 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175236.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32634.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178836.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29034.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182436.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25434.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186036.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21834.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189636.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18234.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193236.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14634.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196836.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11034.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200436.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7434.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204036.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3834.28 sec) 20:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207636.480364 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 234.28 sec) 20:13:00 [GPUGRID] [rr_sim] 2207870.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970974945.27G/439.78G) 20:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853691.67 20:13:00 [---] pick_jobs_to_run() start 20:13:00 [---] pick_jobs_to_run() end 20:13:00 [---] [rr_sim] end 20:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:13:00 [---] [cpu_sched_debug] preliminary job list: 20:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:13:00 [---] [cpu_sched_debug] final job list: 20:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:13:00 [---] [cpu_sched_debug] enforce_run_list: end 20:13:00 [---] choose_project(): 72780.000000 20:13:00 [WUProp@Home] doing RPC to get work 20:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362819.09 cpu 2111552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111552.90 20:13:00 [---] Got 0 tasks 20:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:14:00 [---] [rr_sim] doing sim: CPU sched 20:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970947354.91G/439.78G) 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207808.02 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 220780.80 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220780.802273 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1987027.22 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 198702.72 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419483.524319 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788324.50 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 178832.45 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598315.974160 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609492.05 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 160949.20 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759265.179017 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448542.84 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 144854.28 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904119.463388 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303688.56 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 130368.86 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034488.319322 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173319.70 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 117331.97 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151820.289663 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055987.73 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 105598.77 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257419.062970 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950388.96 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 95038.90 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352457.958946 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855350.06 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 85535.01 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437992.965324 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769815.06 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 76981.51 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514974.471065 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692833.55 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 69283.36 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584257.826231 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623550.20 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 62355.02 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646612.845881 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561195.18 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 56119.52 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702732.363566 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505075.66 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 50507.57 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753239.929482 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454568.09 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 45456.81 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798696.738807 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409111.28 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 40911.13 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839607.867199 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368200.16 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 36820.02 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876427.882752 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331380.14 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 33138.01 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909565.896750 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298242.13 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 29824.21 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939390.109348 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268417.91 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 26841.79 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966231.900686 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241576.12 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 24157.61 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990389.512891 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217418.51 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 21741.85 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012131.363874 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195676.66 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 19567.67 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031699.029760 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176108.99 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 17610.90 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049309.929057 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158498.09 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 15849.81 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065159.738424 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142648.28 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 14264.83 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079424.566855 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128383.46 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 12838.35 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092262.912442 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115545.11 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 11554.51 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103817.423471 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103990.60 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 10399.06 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114216.483397 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93591.54 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 9359.15 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123575.637330 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84232.39 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 8423.24 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131998.875870 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75809.15 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 7580.91 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139579.790556 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68228.23 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 6822.82 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146402.613773 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61405.41 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 6140.54 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152543.154669 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55264.87 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 5526.49 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158069.641475 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49738.38 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 4973.84 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163043.479601 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44764.54 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 4476.45 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167519.933914 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40288.09 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 4028.81 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171548.742795 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36259.28 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3625.93 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175174.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32633.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178774.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29033.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182374.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25433.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185974.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21833.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189574.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18233.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193174.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14633.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196774.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11033.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200374.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7433.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203974.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3833.35 sec) 20:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207574.670789 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 233.35 sec) 20:14:00 [GPUGRID] [rr_sim] 2207808.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970947354.91G/439.78G) 20:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853688.94 20:14:00 [---] pick_jobs_to_run() start 20:14:00 [---] pick_jobs_to_run() end 20:14:00 [---] [rr_sim] end 20:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:14:00 [---] [cpu_sched_debug] preliminary job list: 20:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:14:00 [---] [cpu_sched_debug] final job list: 20:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:14:00 [---] [cpu_sched_debug] enforce_run_list: end 20:14:00 [---] choose_project(): 72840.000000 20:14:00 [WUProp@Home] doing RPC to get work 20:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362759.09 cpu 2111492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111492.90 20:14:00 [---] Got 0 tasks 20:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:15:00 [---] [rr_sim] doing sim: CPU sched 20:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970919764.55G/439.78G) 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207745.29 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 220774.53 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220774.528585 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986970.76 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 198697.08 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419471.604311 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788273.68 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 178827.37 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598298.972465 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609446.31 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 160944.63 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759243.603803 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448501.68 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 144850.17 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904093.772007 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303651.51 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 130365.15 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034458.923391 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173286.36 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 117328.64 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151787.559637 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055957.73 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 105595.77 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257383.332258 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950361.95 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 95036.20 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352419.527617 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855325.76 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 85532.58 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437952.103440 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769793.18 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 76979.32 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514931.421681 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692813.86 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 69281.39 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584212.808098 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623532.48 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 62353.25 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646566.055873 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561179.23 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 56117.92 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702683.978870 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505061.31 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 50506.13 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753190.109568 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454555.18 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 45455.52 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798645.627196 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409099.66 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 40909.97 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839555.593061 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368189.69 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 36818.97 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876374.562340 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331370.72 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 33137.07 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909511.634690 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298233.65 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 29823.37 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939334.999806 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268410.29 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 26841.03 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966176.028410 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241569.26 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 24156.93 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990332.954154 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217412.33 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 21741.23 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012074.187323 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195671.10 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 19567.11 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031641.297176 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176103.99 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 17610.40 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049251.696043 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158493.59 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 15849.36 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065101.055023 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142644.23 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 14264.42 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079365.478106 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128379.81 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 12837.98 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092203.458880 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115541.83 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 11554.18 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103757.641577 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103987.64 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 10398.76 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114156.406004 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93588.88 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 9358.89 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123515.293988 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84229.99 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 8423.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131938.293174 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75806.99 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 7580.70 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139518.992442 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68226.29 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 6822.63 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146341.621782 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61403.66 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 6140.37 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152481.988189 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55263.30 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 5526.33 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158008.317955 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49736.97 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 4973.70 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162982.014744 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44763.27 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 4476.33 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167458.341854 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40286.94 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 4028.69 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171487.036254 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36258.25 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3625.82 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175112.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32632.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178712.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29032.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182312.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25432.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185912.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21832.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189512.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18232.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193112.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14632.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196712.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11032.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200312.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7432.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203912.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3832.42 sec) 20:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207512.861213 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 232.42 sec) 20:15:00 [GPUGRID] [rr_sim] 2207745.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970919764.55G/439.78G) 20:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853686.20 20:15:00 [---] pick_jobs_to_run() start 20:15:00 [---] pick_jobs_to_run() end 20:15:00 [---] [rr_sim] end 20:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:15:00 [---] [cpu_sched_debug] preliminary job list: 20:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:15:00 [---] [cpu_sched_debug] final job list: 20:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:15:00 [---] [cpu_sched_debug] enforce_run_list: end 20:15:00 [---] choose_project(): 72900.000000 20:15:00 [WUProp@Home] doing RPC to get work 20:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362699.09 cpu 2111432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111432.90 20:15:00 [---] Got 0 tasks 20:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:16:00 [---] [rr_sim] doing sim: CPU sched 20:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970892174.20G/439.78G) 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207682.55 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 220768.25 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220768.254897 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986914.29 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 198691.43 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419459.684303 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788222.86 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 178822.29 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598281.970770 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609400.58 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 160940.06 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759222.028589 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448460.52 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 144846.05 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904068.080627 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303614.47 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 130361.45 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034429.527461 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173253.02 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 117325.30 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151754.829611 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055927.72 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 105592.77 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257347.601547 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950334.95 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 95033.49 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352381.096288 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855301.45 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 85530.15 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437911.241556 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769771.31 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 76977.13 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514888.372297 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692794.18 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 69279.42 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584167.789964 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623514.76 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 62351.48 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646519.265864 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561163.28 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 56116.33 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702635.594174 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505046.95 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 50504.70 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753140.289653 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454542.26 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 45454.23 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798594.515585 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409088.03 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 40908.80 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839503.318923 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368179.23 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 36817.92 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876321.241927 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331361.31 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 33136.13 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909457.372631 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298225.18 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 29822.52 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939279.890264 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268402.66 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 26840.27 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966120.156134 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241562.39 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 24156.24 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990276.395417 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217406.15 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 21740.62 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2012017.010772 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195665.54 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 19566.55 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031583.564592 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176098.98 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 17609.90 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049193.463029 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158489.09 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 15848.91 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065042.371623 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142640.18 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 14264.02 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079306.389357 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128376.16 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 12837.62 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092144.005318 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115538.54 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 11553.85 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103697.859683 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103984.69 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 10398.47 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114096.328611 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93586.22 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 9358.62 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123454.950646 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84227.60 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 8422.76 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131877.710478 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75804.84 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 7580.48 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139458.194327 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68224.35 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 6822.44 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146280.629791 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61401.92 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 6140.19 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152420.821708 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55261.73 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 5526.17 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157946.994434 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49735.55 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 4973.56 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162920.549887 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44762.00 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 4476.20 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167396.749795 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40285.80 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 4028.58 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171425.329712 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36257.22 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3625.72 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175051.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32631.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178651.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29031.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182251.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25431.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185851.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21831.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189451.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18231.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193051.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14631.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196651.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11031.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200251.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7431.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203851.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3831.50 sec) 20:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207451.051637 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 231.50 sec) 20:16:00 [GPUGRID] [rr_sim] 2207682.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970892174.20G/439.78G) 20:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853683.46 20:16:00 [---] pick_jobs_to_run() start 20:16:00 [---] pick_jobs_to_run() end 20:16:00 [---] [rr_sim] end 20:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:16:00 [---] [cpu_sched_debug] preliminary job list: 20:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:16:00 [---] [cpu_sched_debug] final job list: 20:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:16:00 [---] [cpu_sched_debug] enforce_run_list: end 20:16:00 [---] choose_project(): 72960.000000 20:16:00 [WUProp@Home] doing RPC to get work 20:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362639.09 cpu 2111372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111372.90 20:16:00 [---] Got 0 tasks 20:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:17:00 [---] [rr_sim] doing sim: CPU sched 20:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970864583.84G/439.78G) 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207619.81 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 220761.98 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220761.981208 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986857.83 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 198685.78 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419447.764296 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788172.05 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 178817.20 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598264.969075 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609354.84 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 160935.48 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759200.453375 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448419.36 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 144841.94 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904042.389246 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303577.42 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 130357.74 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034400.131530 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173219.68 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 117321.97 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151722.099585 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055897.71 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 105589.77 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257311.870835 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950307.94 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 95030.79 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352342.664960 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855277.15 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 85527.71 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437870.379672 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769749.43 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 76974.94 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514845.322913 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692774.49 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 69277.45 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584122.771830 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623497.04 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 62349.70 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646472.475856 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561147.34 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 56114.73 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702587.209478 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505032.60 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 50503.26 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753090.469739 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454529.34 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 45452.93 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798543.403973 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409076.41 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 40907.64 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839451.044784 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368168.77 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 36816.88 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876267.921514 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331351.89 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 33135.19 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909403.110571 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298216.70 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 29821.67 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939224.780722 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268395.03 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 26839.50 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966064.283858 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241555.53 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 24155.55 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990219.836681 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217399.98 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 21740.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011959.834221 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195659.98 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 19566.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031525.832007 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176093.98 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 17609.40 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049135.230015 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158484.58 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 15848.46 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064983.688222 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142636.12 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 14263.61 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079247.300608 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128372.51 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 12837.25 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092084.551755 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115535.26 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 11553.53 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103638.077788 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103981.73 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 10398.17 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114036.251218 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93583.56 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 9358.36 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123394.607304 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84225.20 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 8422.52 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131817.127782 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75802.68 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 7580.27 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139397.396212 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68222.42 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 6822.24 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146219.637799 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61400.17 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 6140.02 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152359.655228 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55260.16 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 5526.02 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157885.670913 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49734.14 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 4973.41 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162859.085030 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44760.73 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 4476.07 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167335.157736 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40284.65 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 4028.47 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171363.623170 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36256.19 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3625.62 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174989.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32630.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178589.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29030.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182189.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25430.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185789.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21830.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189389.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18230.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192989.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14630.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196589.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11030.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200189.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7430.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203789.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3830.57 sec) 20:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207389.242062 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 230.57 sec) 20:17:00 [GPUGRID] [rr_sim] 2207619.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970864583.84G/439.78G) 20:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853680.72 20:17:00 [---] pick_jobs_to_run() start 20:17:00 [---] pick_jobs_to_run() end 20:17:00 [---] [rr_sim] end 20:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:17:00 [---] [cpu_sched_debug] preliminary job list: 20:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:17:00 [---] [cpu_sched_debug] final job list: 20:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:17:00 [---] [cpu_sched_debug] enforce_run_list: end 20:17:00 [---] choose_project(): 73020.000000 20:17:00 [WUProp@Home] doing RPC to get work 20:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362579.09 cpu 2111312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111312.90 20:17:00 [---] Got 0 tasks 20:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:18:00 [---] [rr_sim] doing sim: CPU sched 20:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970836993.48G/439.78G) 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207557.08 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 220755.71 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220755.707520 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986801.37 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 198680.14 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419435.844288 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788121.23 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 178812.12 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598247.967379 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609309.11 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 160930.91 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759178.878162 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448378.20 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 144837.82 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 904016.697866 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303540.38 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 130354.04 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034370.735599 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173186.34 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 117318.63 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151689.369559 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055867.71 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 105586.77 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257276.140124 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950280.94 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 95028.09 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352304.233631 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855252.84 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 85525.28 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437829.517788 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769727.56 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 76972.76 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514802.273530 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692754.80 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 69275.48 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584077.753697 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623479.32 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 62347.93 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646425.685847 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561131.39 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 56113.14 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702538.824783 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505018.25 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 50501.83 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1753040.649824 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454516.43 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 45451.64 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798492.292362 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409064.78 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 40906.48 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839398.770646 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368158.30 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 36815.83 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876214.601101 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331342.47 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 33134.25 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909348.848511 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298208.23 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 29820.82 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939169.671180 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268387.40 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 26838.74 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1966008.411582 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241548.66 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 24154.87 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990163.277944 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217393.80 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 21739.38 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011902.657670 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195654.42 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 19565.44 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031468.099423 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176088.98 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 17608.90 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049076.997001 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158480.08 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 15848.01 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064925.004821 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142632.07 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 14263.21 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079188.211859 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128368.86 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 12836.89 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092025.098193 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115531.98 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 11553.20 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103578.295894 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103978.78 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 10397.88 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113976.173825 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93580.90 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 9358.09 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123334.263962 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84222.81 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 8422.28 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131756.545086 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75800.53 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 7580.05 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139336.598098 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68220.48 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 6822.05 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146158.645808 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61398.43 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 6139.84 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152298.488747 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55258.59 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 5525.86 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157824.347393 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49732.73 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 4973.27 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162797.620174 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44759.46 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 4475.95 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167273.565676 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40283.51 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 4028.35 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171301.916629 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36255.16 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3625.52 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174927.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32629.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178527.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29029.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182127.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25429.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185727.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21829.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189327.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18229.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192927.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14629.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196527.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11029.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200127.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7429.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203727.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3829.64 sec) 20:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207327.432486 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 229.64 sec) 20:18:00 [GPUGRID] [rr_sim] 2207557.08: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970836993.48G/439.78G) 20:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853677.99 20:18:00 [---] pick_jobs_to_run() start 20:18:00 [---] pick_jobs_to_run() end 20:18:00 [---] [rr_sim] end 20:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:18:00 [---] [cpu_sched_debug] preliminary job list: 20:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:18:00 [---] [cpu_sched_debug] final job list: 20:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:18:00 [---] [cpu_sched_debug] enforce_run_list: end 20:18:00 [---] choose_project(): 73080.000000 20:18:00 [WUProp@Home] doing RPC to get work 20:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362519.09 cpu 2111252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111252.90 20:18:00 [---] Got 0 tasks 20:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:19:00 [---] [rr_sim] doing sim: CPU sched 20:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970809403.13G/439.78G) 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207494.34 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 220749.43 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220749.433832 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986744.90 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 198674.49 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419423.924281 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788070.41 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 178807.04 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598230.965684 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609263.37 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 160926.34 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759157.302948 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448337.04 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 144833.70 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903991.006485 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303503.33 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 130350.33 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034341.339668 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173153.00 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 117315.30 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151656.639533 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055837.70 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 105583.77 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257240.409412 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950253.93 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 95025.39 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352265.802303 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855228.54 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 85522.85 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437788.655904 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769705.68 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 76970.57 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514759.224146 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692735.11 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 69273.51 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1584032.735563 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623461.60 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 62346.16 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646378.895839 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561115.44 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 56111.54 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702490.440087 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 505003.90 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 50500.39 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752990.829910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454503.51 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 45450.35 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798441.180751 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409053.16 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 40905.32 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839346.496508 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368147.84 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 36814.78 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876161.280689 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331333.06 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 33133.31 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909294.586452 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298199.75 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 29819.98 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939114.561638 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268379.78 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 26837.98 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965952.539307 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241541.80 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 24154.18 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990106.719208 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217387.62 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 21738.76 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011845.481119 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195648.86 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 19564.89 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031410.366839 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176083.97 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 17608.40 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049018.763987 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158475.57 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 15847.56 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064866.321420 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142628.02 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 14262.80 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079129.123110 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128365.22 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 12836.52 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091965.644631 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115528.69 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 11552.87 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103518.514000 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103975.82 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 10397.58 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113916.096432 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93578.24 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 9357.82 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123273.920620 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84220.42 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 8422.04 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131695.962390 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75798.38 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 7579.84 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139275.799983 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68218.54 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 6821.85 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146097.653817 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61396.68 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 6139.67 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152237.322267 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55257.02 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 5525.70 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157763.023872 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49731.31 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 4973.13 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162736.155317 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44758.18 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 4475.82 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167211.973617 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40282.36 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 4028.24 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171240.210087 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36254.13 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3625.41 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174865.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32628.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178465.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29028.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182065.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25428.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185665.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21828.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189265.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18228.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192865.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14628.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196465.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11028.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200065.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7428.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203665.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3828.72 sec) 20:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207265.622910 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 228.72 sec) 20:19:00 [GPUGRID] [rr_sim] 2207494.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970809403.13G/439.78G) 20:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853675.25 20:19:00 [---] pick_jobs_to_run() start 20:19:00 [---] pick_jobs_to_run() end 20:19:00 [---] [rr_sim] end 20:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:19:00 [---] [cpu_sched_debug] preliminary job list: 20:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:19:00 [---] [cpu_sched_debug] final job list: 20:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:19:00 [---] [cpu_sched_debug] enforce_run_list: end 20:19:00 [---] choose_project(): 73140.000000 20:19:00 [WUProp@Home] doing RPC to get work 20:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362459.09 cpu 2111192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111192.90 20:19:00 [---] Got 0 tasks 20:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:20:00 [---] [rr_sim] doing sim: CPU sched 20:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970781812.77G/439.78G) 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207431.60 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 220743.16 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220743.160144 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986688.44 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 198668.84 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419412.004273 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1788019.60 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 178801.96 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598213.963989 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609217.64 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 160921.76 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759135.727734 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448295.87 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 144829.59 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903965.315104 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303466.29 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 130346.63 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034311.943738 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173119.66 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 117311.97 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151623.909508 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055807.69 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 105580.77 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257204.678700 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950226.92 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 95022.69 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352227.370974 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855204.23 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 85520.42 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437747.794020 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769683.81 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 76968.38 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514716.174762 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692715.43 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 69271.54 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583987.717429 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623443.88 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 62344.39 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646332.105830 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561099.50 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 56109.95 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702442.055391 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504989.55 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 50498.95 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752941.009995 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454490.59 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 45449.06 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798390.069140 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409041.53 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 40904.15 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839294.222369 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368137.38 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 36813.74 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876107.960276 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331323.64 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 33132.36 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909240.324392 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298191.28 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 29819.13 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939059.452097 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268372.15 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 26837.21 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965896.667031 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241534.93 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 24153.49 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1990050.160471 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217381.44 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 21738.14 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011788.304568 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195643.30 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 19564.33 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031352.634255 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176078.97 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 17607.90 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048960.530973 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158471.07 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 15847.11 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064807.638019 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142623.96 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 14262.40 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079070.034361 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128361.57 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 12836.16 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091906.191069 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115525.41 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 11552.54 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103458.732105 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103972.87 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 10397.29 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113856.019039 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93575.58 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 9357.56 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123213.577278 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84218.02 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 8421.80 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131635.379694 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75796.22 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 7579.62 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139215.001868 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68216.60 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 6821.66 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146036.661825 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61394.94 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 6139.49 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152176.155786 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55255.45 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 5525.54 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157701.700351 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49729.90 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 4972.99 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162674.690460 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44756.91 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 4475.69 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167150.381558 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40281.22 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 4028.12 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171178.503546 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36253.10 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3625.31 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174803.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32627.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178403.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29027.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182003.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25427.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185603.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21827.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189203.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18227.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192803.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14627.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196403.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11027.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200003.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7427.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203603.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3827.79 sec) 20:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207203.813335 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 227.79 sec) 20:20:00 [GPUGRID] [rr_sim] 2207431.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970781812.77G/439.78G) 20:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853672.51 20:20:00 [---] pick_jobs_to_run() start 20:20:00 [---] pick_jobs_to_run() end 20:20:00 [---] [rr_sim] end 20:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:20:00 [---] [cpu_sched_debug] preliminary job list: 20:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:20:00 [---] [cpu_sched_debug] final job list: 20:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:20:00 [---] [cpu_sched_debug] enforce_run_list: end 20:20:00 [---] choose_project(): 73200.000000 20:20:00 [WUProp@Home] doing RPC to get work 20:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362399.09 cpu 2111132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111132.90 20:20:00 [---] Got 0 tasks 20:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:21:00 [---] [rr_sim] doing sim: CPU sched 20:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970754222.42G/439.78G) 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207368.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 220736.89 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220736.886455 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986631.98 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 198663.20 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419400.084265 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787968.78 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 178796.88 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598196.962294 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609171.90 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 160917.19 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759114.152520 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448254.71 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 144825.47 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903939.623724 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303429.24 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 130342.92 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034282.547807 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173086.32 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 117308.63 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151591.179482 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055777.69 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 105577.77 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257168.947989 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950199.92 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 95019.99 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352188.939645 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855179.92 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 85517.99 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437706.932136 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769661.93 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 76966.19 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514673.125378 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692695.74 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 69269.57 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583942.699296 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623426.17 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 62342.62 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646285.315822 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561083.55 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 56108.35 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702393.670695 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504975.19 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 50497.52 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752891.190081 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454477.67 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 45447.77 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798338.957528 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409029.91 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 40902.99 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839241.948231 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368126.92 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 36812.69 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876054.639863 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331314.22 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 33131.42 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909186.062332 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298182.80 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 29818.28 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1939004.342555 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268364.52 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 26836.45 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965840.794755 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241528.07 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 24152.81 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989993.601735 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217375.26 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 21737.53 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011731.128017 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195637.74 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 19563.77 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031294.901670 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176073.96 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 17607.40 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048902.297959 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158466.57 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 15846.66 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064748.954618 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142619.91 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 14261.99 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079010.945612 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128357.92 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 12835.79 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091846.737506 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115522.13 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 11552.21 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103398.950211 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103969.91 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 10396.99 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113795.941645 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93572.92 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 9357.29 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123153.233936 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84215.63 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 8421.56 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131574.796998 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75794.07 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 7579.41 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139154.203754 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68214.66 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 6821.47 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145975.669834 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61393.19 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 6139.32 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152114.989306 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55253.88 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 5525.39 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157640.376831 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49728.49 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 4972.85 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162613.225603 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44755.64 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 4475.56 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167088.789498 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40280.08 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 4028.01 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171116.797004 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36252.07 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3625.21 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174742.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32626.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178342.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29026.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181942.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25426.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185542.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21826.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189142.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18226.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192742.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14626.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196342.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11026.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199942.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7426.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203542.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3826.86 sec) 20:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207142.003759 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 226.86 sec) 20:21:00 [GPUGRID] [rr_sim] 2207368.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970754222.42G/439.78G) 20:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853669.78 20:21:00 [---] pick_jobs_to_run() start 20:21:00 [---] pick_jobs_to_run() end 20:21:00 [---] [rr_sim] end 20:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:21:00 [---] [cpu_sched_debug] preliminary job list: 20:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:21:00 [---] [cpu_sched_debug] final job list: 20:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:21:00 [---] [cpu_sched_debug] enforce_run_list: end 20:21:00 [---] choose_project(): 73260.000000 20:21:00 [WUProp@Home] doing RPC to get work 20:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362339.09 cpu 2111072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111072.90 20:21:00 [---] Got 0 tasks 20:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:22:00 [---] [rr_sim] doing sim: CPU sched 20:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970726632.06G/439.78G) 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207306.13 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 220730.61 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220730.612767 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986575.51 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 198657.55 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419388.164258 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787917.96 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 178791.80 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598179.960599 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609126.17 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 160912.62 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759092.577307 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448213.55 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 144821.36 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903913.932343 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303392.20 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 130339.22 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034253.151876 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173052.98 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 117305.30 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151558.449456 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055747.68 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 105574.77 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257133.217277 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950172.91 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 95017.29 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352150.508317 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855155.62 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 85515.56 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437666.070252 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769640.06 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 76964.01 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514630.075994 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692676.05 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 69267.61 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583897.681162 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623408.45 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 62340.84 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646238.525813 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561067.60 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 56106.76 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702345.285999 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504960.84 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 50496.08 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752841.370166 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454464.76 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 45446.48 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798287.845917 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409018.28 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 40901.83 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839189.674093 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368116.45 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 36811.65 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1876001.319451 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331304.81 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 33130.48 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909131.800273 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298174.33 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 29817.43 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938949.233013 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268356.89 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 26835.69 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965784.922479 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241521.21 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 24152.12 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989937.042998 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217369.08 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 21736.91 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011673.951466 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195632.18 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 19563.22 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031237.169086 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176068.96 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 17606.90 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048844.064945 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158462.06 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 15846.21 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064690.271218 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142615.86 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 14261.59 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078951.856863 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128354.27 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 12835.43 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091787.283944 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115518.84 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 11551.88 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103339.168317 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103966.96 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 10396.70 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113735.864252 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93570.26 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 9357.03 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123092.890594 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84213.24 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 8421.32 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131514.214302 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75791.91 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 7579.19 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139093.405639 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68212.72 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 6821.27 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145914.677843 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61391.45 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 6139.14 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152053.822826 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55252.30 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 5525.23 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157579.053310 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49727.07 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 4972.71 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162551.760746 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44754.37 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 4475.44 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167027.197439 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40278.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 4027.89 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171055.090462 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36251.04 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3625.10 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174680.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32625.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178280.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29025.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181880.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25425.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185480.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21825.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189080.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18225.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192680.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14625.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196280.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11025.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199880.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7425.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203480.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3825.93 sec) 20:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207080.194183 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 225.93 sec) 20:22:00 [GPUGRID] [rr_sim] 2207306.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970726632.06G/439.78G) 20:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853667.04 20:22:00 [---] pick_jobs_to_run() start 20:22:00 [---] pick_jobs_to_run() end 20:22:00 [---] [rr_sim] end 20:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:22:00 [---] [cpu_sched_debug] preliminary job list: 20:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:22:00 [---] [cpu_sched_debug] final job list: 20:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:22:00 [---] [cpu_sched_debug] enforce_run_list: end 20:22:00 [---] choose_project(): 73320.000000 20:22:00 [WUProp@Home] doing RPC to get work 20:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362279.09 cpu 2111012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2111012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2111012.90 20:22:00 [---] Got 0 tasks 20:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:23:00 [---] [rr_sim] doing sim: CPU sched 20:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970699041.70G/439.78G) 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207243.39 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 220724.34 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220724.339079 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986519.05 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 198651.91 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419376.244250 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787867.15 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 178786.71 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598162.958904 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609080.43 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 160908.04 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759071.002093 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448172.39 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 144817.24 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903888.240963 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303355.15 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 130335.51 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034223.755945 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1173019.63 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 117301.96 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151525.719430 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055717.67 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 105571.77 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257097.486566 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950145.90 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 95014.59 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352112.076988 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855131.31 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 85513.13 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437625.208368 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769618.18 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 76961.82 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514587.026611 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692656.36 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 69265.64 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583852.663029 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623390.73 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 62339.07 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646191.735805 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561051.65 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 56105.17 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702296.901303 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504946.49 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 50494.65 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752791.550252 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454451.84 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 45445.18 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798236.734306 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 409006.66 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 40900.67 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839137.399954 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368105.99 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 36810.60 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875947.999038 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331295.39 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 33129.54 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909077.538213 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298165.85 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 29816.59 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938894.123471 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268349.27 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 26834.93 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965729.050203 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241514.34 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 24151.43 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989880.484262 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217362.91 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 21736.29 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011616.774914 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195626.62 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 19562.66 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031179.436502 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176063.95 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 17606.40 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048785.831931 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158457.56 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 15845.76 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064631.587817 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142611.80 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 14261.18 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078892.768114 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128350.62 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 12835.06 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091727.830382 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115515.56 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 11551.56 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103279.386423 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103964.00 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 10396.40 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113675.786859 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93567.60 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 9356.76 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123032.547252 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84210.84 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 8421.08 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131453.631606 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75789.76 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 7578.98 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139032.607525 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68210.78 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 6821.08 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145853.685851 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61389.70 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 6138.97 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151992.656345 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55250.73 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 5525.07 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157517.729790 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49725.66 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 4972.57 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162490.295890 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44753.09 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 4475.31 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166965.605380 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40277.79 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 4027.78 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170993.383921 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36250.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3625.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174618.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32625.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178218.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29025.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181818.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25425.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185418.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21825.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189018.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18225.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192618.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14625.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196218.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11025.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199818.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7425.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203418.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3825.01 sec) 20:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2207018.384608 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 225.01 sec) 20:23:00 [GPUGRID] [rr_sim] 2207243.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970699041.70G/439.78G) 20:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853664.30 20:23:00 [---] pick_jobs_to_run() start 20:23:00 [---] pick_jobs_to_run() end 20:23:00 [---] [rr_sim] end 20:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:23:00 [---] [cpu_sched_debug] preliminary job list: 20:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:23:00 [---] [cpu_sched_debug] final job list: 20:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:23:00 [---] [cpu_sched_debug] enforce_run_list: end 20:23:00 [---] choose_project(): 73380.000000 20:23:00 [WUProp@Home] doing RPC to get work 20:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362219.09 cpu 2110952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110952.90 20:23:00 [---] Got 0 tasks 20:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:24:00 [---] [rr_sim] doing sim: CPU sched 20:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970671451.35G/439.78G) 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207180.65 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 220718.07 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220718.065391 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986462.59 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 198646.26 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419364.324243 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787816.33 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 178781.63 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598145.957209 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1609034.70 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 160903.47 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759049.426879 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448131.23 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 144813.12 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903862.549582 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303318.10 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 130331.81 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034194.360015 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172986.29 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 117298.63 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151492.989404 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055687.66 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 105568.77 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257061.755854 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950118.90 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 95011.89 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352073.645660 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855107.01 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 85510.70 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437584.346485 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769596.31 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 76959.63 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514543.977227 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692636.68 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 69263.67 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583807.644895 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623373.01 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 62337.30 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646144.945796 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561035.71 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 56103.57 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702248.516607 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504932.14 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 50493.21 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752741.730338 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454438.92 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 45443.89 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798185.622695 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408995.03 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 40899.50 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839085.125816 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368095.53 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 36809.55 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875894.678625 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331285.98 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 33128.60 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1909023.276153 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298157.38 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 29815.74 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938839.013929 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268341.64 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 26834.16 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965673.177927 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241507.48 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 24150.75 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989823.925525 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217356.73 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 21735.67 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011559.598363 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195621.06 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 19562.11 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031121.703918 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176058.95 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 17605.89 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048727.598917 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158453.05 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 15845.31 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064572.904416 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142607.75 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 14260.77 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078833.679365 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128346.97 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 12834.70 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091668.376819 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115512.28 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 11551.23 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103219.604528 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103961.05 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 10396.10 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113615.709466 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93564.94 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 9356.49 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122972.203910 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84208.45 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 8420.84 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131393.048910 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75787.60 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 7578.76 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138971.809410 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68208.84 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 6820.88 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145792.693860 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61387.96 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 6138.80 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151931.489865 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55249.16 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 5524.92 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157456.406269 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49724.25 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 4972.42 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162428.831033 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44751.82 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 4475.18 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166904.013320 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40276.64 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 4027.66 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170931.677379 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36248.98 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3624.90 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174556.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32624.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178156.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29024.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181756.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25424.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185356.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21824.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188956.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18224.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192556.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14624.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196156.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11024.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199756.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7424.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203356.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3824.08 sec) 20:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206956.575032 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 224.08 sec) 20:24:00 [GPUGRID] [rr_sim] 2207180.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970671451.35G/439.78G) 20:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853661.57 20:24:00 [---] pick_jobs_to_run() start 20:24:00 [---] pick_jobs_to_run() end 20:24:00 [---] [rr_sim] end 20:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:24:00 [---] [cpu_sched_debug] preliminary job list: 20:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:24:00 [---] [cpu_sched_debug] final job list: 20:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:24:00 [---] [cpu_sched_debug] enforce_run_list: end 20:24:00 [---] choose_project(): 73440.000000 20:24:00 [WUProp@Home] doing RPC to get work 20:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362159.09 cpu 2110892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110892.90 20:24:00 [---] Got 0 tasks 20:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:25:00 [---] [rr_sim] doing sim: CPU sched 20:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970643860.99G/439.78G) 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207117.92 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 220711.79 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220711.791703 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986406.13 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 198640.61 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419352.404235 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787765.51 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 178776.55 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598128.955514 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608988.96 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 160898.90 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759027.851665 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448090.07 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 144809.01 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903836.858201 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303281.06 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 130328.11 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034164.964084 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172952.95 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 117295.30 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151460.259378 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055657.66 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 105565.77 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1257026.025143 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950091.89 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 95009.19 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1352035.214331 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855082.70 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 85508.27 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437543.484601 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769574.43 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 76957.44 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514500.927843 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692616.99 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 69261.70 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583762.626761 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623355.29 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 62335.53 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646098.155788 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561019.76 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 56101.98 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702200.131912 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504917.79 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 50491.78 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752691.910423 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454426.01 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 45442.60 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798134.511083 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408983.41 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 40898.34 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1839032.851678 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368085.07 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 36808.51 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875841.358212 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331276.56 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 33127.66 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908969.014094 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298148.90 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 29814.89 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938783.904387 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268334.01 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 26833.40 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965617.305651 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241500.61 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 24150.06 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989767.366788 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217350.55 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 21735.06 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011502.421812 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195615.50 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 19561.55 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031063.971334 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176053.95 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 17605.39 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048669.365903 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158448.55 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 15844.86 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064514.221015 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142603.70 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 14260.37 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078774.590616 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128343.33 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 12834.33 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091608.923257 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115508.99 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 11550.90 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103159.822634 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103958.09 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 10395.81 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113555.632073 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93562.28 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 9356.23 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122911.860568 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84206.06 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 8420.61 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131332.466214 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75785.45 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 7578.55 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138911.011295 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68206.91 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 6820.69 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145731.701868 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61386.22 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 6138.62 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151870.323384 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55247.59 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 5524.76 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157395.082748 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49722.83 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 4972.28 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162367.366176 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44750.55 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 4475.06 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166842.421261 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40275.50 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 4027.55 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170869.970838 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36247.95 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3624.79 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174494.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32623.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178094.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29023.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181694.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25423.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185294.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21823.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188894.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18223.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192494.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14623.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196094.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11023.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199694.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7423.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203294.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3823.15 sec) 20:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206894.765456 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 223.15 sec) 20:25:00 [GPUGRID] [rr_sim] 2207117.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970643860.99G/439.78G) 20:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853658.83 20:25:00 [---] pick_jobs_to_run() start 20:25:00 [---] pick_jobs_to_run() end 20:25:00 [---] [rr_sim] end 20:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:25:00 [---] [cpu_sched_debug] preliminary job list: 20:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:25:00 [---] [cpu_sched_debug] final job list: 20:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:25:00 [---] [cpu_sched_debug] enforce_run_list: end 20:25:00 [---] choose_project(): 73500.000000 20:25:00 [WUProp@Home] doing RPC to get work 20:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362099.09 cpu 2110832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110832.90 20:25:00 [---] Got 0 tasks 20:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:26:00 [---] [rr_sim] doing sim: CPU sched 20:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970616270.64G/439.78G) 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2207055.18 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 220705.52 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220705.518014 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986349.66 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 198634.97 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419340.484227 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787714.70 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 178771.47 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598111.953819 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608943.23 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 160894.32 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 759006.276451 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448048.90 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 144804.89 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903811.166821 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303244.01 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 130324.40 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034135.568153 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172919.61 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 117291.96 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151427.529352 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055627.65 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 105562.77 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256990.294431 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950064.89 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 95006.49 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351996.783002 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855058.40 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 85505.84 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437502.622717 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769552.56 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 76955.26 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514457.878459 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692597.30 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 69259.73 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583717.608628 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623337.57 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 62333.76 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646051.365779 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 561003.81 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 56100.38 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702151.747216 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504903.43 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 50490.34 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752642.090509 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454413.09 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 45441.31 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798083.399472 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408971.78 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 40897.18 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838980.577539 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368074.60 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 36807.46 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875788.037800 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331267.14 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 33126.71 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908914.752034 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298140.43 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 29814.04 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938728.794845 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268326.39 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 26832.64 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965561.433375 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241493.75 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 24149.37 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989710.808052 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217344.37 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 21734.44 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011445.245261 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195609.93 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 19560.99 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2031006.238749 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176048.94 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 17604.89 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048611.132889 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158444.05 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 15844.40 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064455.537614 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142599.64 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 14259.96 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078715.501867 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128339.68 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 12833.97 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091549.469695 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115505.71 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 11550.57 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103100.040740 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103955.14 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 10395.51 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113495.554680 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93559.63 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 9355.96 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122851.517227 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84203.66 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 8420.37 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131271.883518 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75783.30 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 7578.33 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138850.213181 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68204.97 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 6820.50 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145670.709877 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61384.47 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 6138.45 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151809.156904 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55246.02 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 5524.60 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157333.759228 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49721.42 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 4972.14 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162305.901319 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44749.28 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 4474.93 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166780.829202 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40274.35 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 4027.44 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170808.264296 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36246.92 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3624.69 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174432.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32622.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178032.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29022.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181632.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25422.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185232.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21822.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188832.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18222.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192432.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14622.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196032.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11022.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199632.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7422.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203232.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3822.22 sec) 20:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206832.955881 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 222.22 sec) 20:26:00 [GPUGRID] [rr_sim] 2207055.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970616270.64G/439.78G) 20:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853656.09 20:26:00 [---] pick_jobs_to_run() start 20:26:00 [---] pick_jobs_to_run() end 20:26:00 [---] [rr_sim] end 20:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:26:00 [---] [cpu_sched_debug] preliminary job list: 20:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:26:00 [---] [cpu_sched_debug] final job list: 20:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:26:00 [---] [cpu_sched_debug] enforce_run_list: end 20:26:00 [---] choose_project(): 73560.000000 20:26:00 [WUProp@Home] doing RPC to get work 20:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 362039.09 cpu 2110772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110772.90 20:26:00 [---] Got 0 tasks 20:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:27:00 [---] [rr_sim] doing sim: CPU sched 20:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970588680.28G/439.78G) 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206992.44 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 220699.24 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220699.244326 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986293.20 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 198629.32 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419328.564220 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787663.88 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 178766.39 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598094.952124 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608897.49 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 160889.75 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758984.701238 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1448007.74 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 144800.77 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903785.475440 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303206.97 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 130320.70 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034106.172222 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172886.27 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 117288.63 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151394.799326 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055597.64 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 105559.76 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256954.563720 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950037.88 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 95003.79 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351958.351674 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855034.09 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 85503.41 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437461.760833 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769530.68 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 76953.07 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514414.829076 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692577.61 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 69257.76 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583672.590494 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623319.85 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 62331.99 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1646004.575771 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560987.87 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 56098.79 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702103.362520 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504889.08 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 50488.91 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752592.270594 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454400.17 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 45440.02 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1798032.287861 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408960.16 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 40896.02 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838928.303401 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368064.14 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 36806.41 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875734.717387 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331257.73 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 33125.77 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908860.489974 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298131.95 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 29813.20 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938673.685303 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268318.76 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 26831.88 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965505.561099 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241486.88 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 24148.69 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989654.249315 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217338.19 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 21733.82 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011388.068710 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195604.37 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 19560.44 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030948.506165 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176043.94 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 17604.39 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048552.899875 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158439.54 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 15843.95 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064396.854213 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142595.59 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 14259.56 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078656.413118 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128336.03 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 12833.60 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091490.016133 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115502.43 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 11550.24 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103040.258845 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103952.18 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 10395.22 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113435.477287 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93556.97 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 9355.70 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122791.173885 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84201.27 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 8420.13 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131211.300822 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75781.14 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 7578.11 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138789.415066 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68203.03 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 6820.30 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145609.717886 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61382.73 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 6138.27 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151747.990423 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55244.45 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 5524.45 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157272.435707 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49720.01 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 4972.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162244.436463 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44748.01 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 4474.80 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166719.237142 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40273.21 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 4027.32 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170746.557754 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36245.89 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3624.59 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174371.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32621.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177971.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29021.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181571.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25421.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185171.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21821.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188771.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18221.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192371.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14621.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195971.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11021.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199571.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7421.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203171.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3821.30 sec) 20:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206771.146305 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 221.30 sec) 20:27:00 [GPUGRID] [rr_sim] 2206992.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970588680.28G/439.78G) 20:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853653.36 20:27:00 [---] pick_jobs_to_run() start 20:27:00 [---] pick_jobs_to_run() end 20:27:00 [---] [rr_sim] end 20:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:27:00 [---] [cpu_sched_debug] preliminary job list: 20:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:27:00 [---] [cpu_sched_debug] final job list: 20:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:27:00 [---] [cpu_sched_debug] enforce_run_list: end 20:27:00 [---] choose_project(): 73620.000000 20:27:00 [WUProp@Home] doing RPC to get work 20:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361979.09 cpu 2110712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110712.90 20:27:00 [---] Got 0 tasks 20:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:28:00 [---] [rr_sim] doing sim: CPU sched 20:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970561089.92G/439.78G) 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206929.71 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 220692.97 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220692.970638 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986236.74 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 198623.67 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419316.644212 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787613.06 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 178761.31 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598077.950429 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608851.76 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 160885.18 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758963.126024 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447966.58 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 144796.66 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903759.784059 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303169.92 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 130316.99 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034076.776291 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172852.93 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 117285.29 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151362.069300 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055567.64 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 105556.76 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256918.833008 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 950010.87 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 95001.09 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351919.920345 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 855009.79 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 85500.98 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437420.898949 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769508.81 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 76950.88 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514371.779692 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692557.93 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 69255.79 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583627.572361 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623302.13 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 62330.21 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645957.785762 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560971.92 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 56097.19 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702054.977824 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504874.73 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 50487.47 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752542.450680 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454387.26 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 45438.73 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797981.176250 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408948.53 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 40894.85 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838876.029263 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368053.68 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 36805.37 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875681.396974 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331248.31 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 33124.83 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908806.227915 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298123.48 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 29812.35 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938618.575761 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268311.13 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 26831.11 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965449.688823 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241480.02 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 24148.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989597.690579 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217332.02 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 21733.20 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011330.892159 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195598.81 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 19559.88 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030890.773581 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176038.93 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 17603.89 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048494.666861 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158435.04 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 15843.50 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064338.170813 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142591.54 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 14259.15 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078597.324369 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128332.38 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 12833.24 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091430.562570 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115499.14 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 11549.91 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102980.476951 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103949.23 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 10394.92 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113375.399894 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93554.31 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 9355.43 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122730.830543 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84198.88 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 8419.89 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131150.718126 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75778.99 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 7577.90 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138728.616952 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68201.09 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 6820.11 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145548.725894 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61380.98 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 6138.10 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151686.823943 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55242.88 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 5524.29 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157211.112187 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49718.59 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 4971.86 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162182.971606 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44746.73 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 4474.67 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166657.645083 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40272.06 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 4027.21 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170684.851213 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36244.86 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3624.49 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174309.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32620.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177909.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29020.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181509.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25420.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185109.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21820.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188709.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18220.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192309.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14620.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195909.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11020.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199509.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7420.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203109.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3820.37 sec) 20:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206709.336729 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 220.37 sec) 20:28:00 [GPUGRID] [rr_sim] 2206929.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970561089.92G/439.78G) 20:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853650.62 20:28:00 [---] pick_jobs_to_run() start 20:28:00 [---] pick_jobs_to_run() end 20:28:00 [---] [rr_sim] end 20:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:28:00 [---] [cpu_sched_debug] preliminary job list: 20:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:28:00 [---] [cpu_sched_debug] final job list: 20:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:28:00 [---] [cpu_sched_debug] enforce_run_list: end 20:28:00 [---] choose_project(): 73680.000000 20:28:00 [WUProp@Home] doing RPC to get work 20:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361919.09 cpu 2110652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110652.90 20:28:00 [---] Got 0 tasks 20:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:29:00 [---] [rr_sim] doing sim: CPU sched 20:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970533499.57G/439.78G) 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206866.97 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 220686.70 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220686.696950 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986180.27 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 198618.03 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419304.724204 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787562.25 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 178756.22 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598060.948734 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608806.02 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 160880.60 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758941.550810 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447925.42 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 144792.54 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903734.092679 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303132.88 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 130313.29 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034047.380361 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172819.59 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 117281.96 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151329.339274 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055537.63 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 105553.76 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256883.102297 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949983.87 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 94998.39 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351881.489017 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854985.48 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 85498.55 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437380.037065 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769486.93 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 76948.69 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514328.730308 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692538.24 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 69253.82 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583582.554227 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623284.42 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 62328.44 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645910.995754 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560955.97 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 56095.60 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1702006.593128 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504860.38 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 50486.04 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752492.630765 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454374.34 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 45437.43 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797930.064638 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408936.90 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 40893.69 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838823.755124 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368043.21 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 36804.32 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875628.076562 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331238.89 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 33123.89 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908751.965855 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298115.00 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 29811.50 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938563.466219 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268303.50 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 26830.35 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965393.816547 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241473.15 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 24147.32 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989541.131842 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217325.84 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 21732.58 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011273.715608 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195593.25 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 19559.33 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030833.040997 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176033.93 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 17603.39 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048436.433847 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158430.54 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 15843.05 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064279.487412 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142587.48 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 14258.75 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078538.235620 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128328.73 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 12832.87 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091371.109008 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115495.86 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 11549.59 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102920.695057 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103946.27 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 10394.63 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113315.322501 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93551.65 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 9355.16 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122670.487201 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84196.48 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 8419.65 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131090.135430 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75776.83 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 7577.68 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138667.818837 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68199.15 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 6819.92 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145487.733903 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61379.24 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 6137.92 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151625.657462 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55241.31 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 5524.13 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157149.788666 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49717.18 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 4971.72 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162121.506749 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44745.46 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 4474.55 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166596.053024 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40270.92 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 4027.09 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170623.144671 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36243.82 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3624.38 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174247.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32619.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177847.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29019.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181447.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25419.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185047.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21819.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188647.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18219.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192247.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14619.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195847.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11019.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199447.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7419.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203047.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3819.44 sec) 20:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206647.527154 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 219.44 sec) 20:29:00 [GPUGRID] [rr_sim] 2206866.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970533499.57G/439.78G) 20:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853647.88 20:29:00 [---] pick_jobs_to_run() start 20:29:00 [---] pick_jobs_to_run() end 20:29:00 [---] [rr_sim] end 20:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:29:00 [---] [cpu_sched_debug] preliminary job list: 20:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:29:00 [---] [cpu_sched_debug] final job list: 20:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:29:00 [---] [cpu_sched_debug] enforce_run_list: end 20:29:00 [---] choose_project(): 73740.000000 20:29:00 [WUProp@Home] doing RPC to get work 20:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361859.09 cpu 2110592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110592.90 20:29:00 [---] Got 0 tasks 20:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:30:00 [---] [rr_sim] doing sim: CPU sched 20:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970505909.21G/439.78G) 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206804.23 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 220680.42 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220680.423262 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986123.81 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 198612.38 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419292.804197 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787511.43 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 178751.14 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598043.947039 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608760.29 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 160876.03 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758919.975596 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447884.26 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 144788.43 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903708.401298 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303095.83 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 130309.58 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1034017.984430 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172786.25 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 117278.62 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151296.609248 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055507.62 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 105550.76 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256847.371585 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949956.86 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 94995.69 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351843.057688 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854961.17 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 85496.12 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437339.175181 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769465.06 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 76946.51 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514285.680924 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692518.55 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 69251.86 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583537.536093 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623266.70 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 62326.67 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645864.205745 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560940.03 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 56094.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701958.208432 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504846.02 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 50484.60 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752442.810851 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454361.42 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 45436.14 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797878.953027 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408925.28 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 40892.53 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838771.480986 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368032.75 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 36803.28 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875574.756149 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331229.48 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 33122.95 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908697.703795 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298106.53 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 29810.65 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938508.356677 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268295.88 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 26829.59 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965337.944271 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241466.29 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 24146.63 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989484.573106 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217319.66 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 21731.97 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011216.539057 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195587.69 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 19558.77 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030775.308412 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176028.92 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 17602.89 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048378.200833 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158426.03 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 15842.60 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064220.804011 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142583.43 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 14258.34 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078479.146871 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128325.09 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 12832.51 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091311.655446 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115492.58 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 11549.26 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102860.913163 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103943.32 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 10394.33 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113255.245108 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93548.99 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 9354.90 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122610.143859 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84194.09 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 8419.41 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131029.552734 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75774.68 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 7577.47 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138607.020722 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68197.21 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 6819.72 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145426.741912 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61377.49 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 6137.75 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151564.490982 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55239.74 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 5523.97 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157088.465145 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49715.77 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 4971.58 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162060.041892 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44744.19 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 4474.42 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166534.460965 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40269.77 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 4026.98 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170561.438130 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36242.79 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3624.28 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174185.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32618.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177785.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29018.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181385.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25418.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184985.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21818.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188585.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18218.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192185.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14618.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195785.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11018.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199385.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7418.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202985.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3818.52 sec) 20:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206585.717578 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 218.52 sec) 20:30:00 [GPUGRID] [rr_sim] 2206804.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970505909.21G/439.78G) 20:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853645.15 20:30:00 [---] pick_jobs_to_run() start 20:30:00 [---] pick_jobs_to_run() end 20:30:00 [---] [rr_sim] end 20:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:30:00 [---] [cpu_sched_debug] preliminary job list: 20:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:30:00 [---] [cpu_sched_debug] final job list: 20:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:30:00 [---] [cpu_sched_debug] enforce_run_list: end 20:30:00 [---] choose_project(): 73800.000000 20:30:00 [WUProp@Home] doing RPC to get work 20:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361799.09 cpu 2110532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110532.90 20:30:00 [---] Got 0 tasks 20:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:31:00 [---] [rr_sim] doing sim: CPU sched 20:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970478318.85G/439.78G) 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206741.50 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 220674.15 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220674.149573 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986067.35 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 198606.73 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419280.884189 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787460.61 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 178746.06 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598026.945344 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608714.55 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 160871.46 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758898.400383 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447843.10 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 144784.31 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903682.709918 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303058.79 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 130305.88 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033988.588499 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172752.91 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 117275.29 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151263.879223 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055477.62 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 105547.76 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256811.640874 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949929.85 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 94992.99 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351804.626359 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854936.87 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 85493.69 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437298.313297 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769443.18 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 76944.32 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514242.631540 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692498.86 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 69249.89 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583492.517960 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623248.98 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 62324.90 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645817.415737 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560924.08 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 56092.41 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701909.823737 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504831.67 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 50483.17 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752392.990936 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454348.50 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 45434.85 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797827.841416 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408913.65 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 40891.37 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838719.206848 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368022.29 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 36802.23 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875521.435736 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331220.06 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 33122.01 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908643.441736 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298098.05 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 29809.81 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938453.247136 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268288.25 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 26828.82 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965282.071995 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241459.42 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 24145.94 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989428.014369 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217313.48 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 21731.35 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011159.362505 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195582.13 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 19558.21 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030717.575828 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176023.92 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 17602.39 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048319.967819 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158421.53 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 15842.15 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064162.120610 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142579.38 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 14257.94 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078420.058122 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128321.44 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 12832.14 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091252.201883 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115489.29 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 11548.93 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102801.131268 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103940.36 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 10394.04 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113195.167715 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93546.33 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 9354.63 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122549.800517 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84191.70 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 8419.17 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130968.970038 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75772.53 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 7577.25 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138546.222608 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68195.27 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 6819.53 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145365.749920 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61375.75 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 6137.57 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151503.324502 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55238.17 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 5523.82 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157027.141625 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49714.35 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 4971.44 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161998.577036 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44742.92 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 4474.29 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166472.868905 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40268.63 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 4026.86 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170499.731588 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36241.76 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3624.18 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174123.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32617.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177723.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29017.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181323.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25417.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184923.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21817.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188523.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18217.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192123.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14617.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195723.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11017.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199323.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7417.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202923.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3817.59 sec) 20:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206523.908003 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217.59 sec) 20:31:00 [GPUGRID] [rr_sim] 2206741.50: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970478318.85G/439.78G) 20:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853642.41 20:31:00 [---] pick_jobs_to_run() start 20:31:00 [---] pick_jobs_to_run() end 20:31:00 [---] [rr_sim] end 20:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:31:00 [---] [cpu_sched_debug] preliminary job list: 20:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:31:00 [---] [cpu_sched_debug] final job list: 20:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:31:00 [---] [cpu_sched_debug] enforce_run_list: end 20:31:00 [---] choose_project(): 73860.000000 20:31:00 [WUProp@Home] doing RPC to get work 20:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361739.09 cpu 2110472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110472.90 20:31:00 [---] Got 0 tasks 20:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:32:00 [---] [rr_sim] doing sim: CPU sched 20:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970450728.50G/439.78G) 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206678.76 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 220667.88 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220667.875885 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1986010.88 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 198601.09 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419268.964182 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787409.79 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 178740.98 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 598009.943649 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608668.82 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 160866.88 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758876.825169 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447801.93 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 144780.19 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903657.018537 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1303021.74 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 130302.17 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033959.192568 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172719.57 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 117271.96 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151231.149197 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055447.61 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 105544.76 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256775.910162 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949902.85 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 94990.28 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351766.195031 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854912.56 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 85491.26 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437257.451413 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769421.31 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 76942.13 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514199.582157 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692479.18 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 69247.92 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583447.499826 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623231.26 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 62323.13 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645770.625729 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560908.13 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 56090.81 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701861.439041 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504817.32 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 50481.73 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752343.171022 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454335.59 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 45433.56 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797776.729805 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408902.03 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 40890.20 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838666.932709 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368011.83 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 36801.18 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875468.115323 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331210.64 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 33121.06 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908589.179676 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298089.58 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 29808.96 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938398.137594 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268280.62 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 26828.06 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965226.199719 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241452.56 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 24145.26 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989371.455632 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217307.30 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 21730.73 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011102.185954 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195576.57 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 19557.66 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030659.843244 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176018.92 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 17601.89 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048261.734805 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158417.02 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 15841.70 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064103.437209 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142575.32 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 14257.53 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078360.969373 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128317.79 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 12831.78 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091192.748321 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115486.01 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 11548.60 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102741.349374 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103937.41 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 10393.74 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113135.090322 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93543.67 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 9354.37 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122489.457175 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84189.30 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 8418.93 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130908.387342 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75770.37 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 7577.04 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138485.424493 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68193.33 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 6819.33 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145304.757929 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61374.00 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 6137.40 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151442.158021 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55236.60 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 5523.66 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156965.818104 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49712.94 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 4971.29 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161937.112179 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44741.65 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 4474.16 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166411.276846 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40267.48 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 4026.75 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170438.025046 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36240.73 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3624.07 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174062.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32616.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177662.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29016.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181262.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25416.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184862.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21816.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188462.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18216.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192062.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14616.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195662.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11016.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199262.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7416.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202862.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3816.66 sec) 20:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206462.098427 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216.66 sec) 20:32:00 [GPUGRID] [rr_sim] 2206678.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970450728.50G/439.78G) 20:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853639.67 20:32:00 [---] pick_jobs_to_run() start 20:32:00 [---] pick_jobs_to_run() end 20:32:00 [---] [rr_sim] end 20:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:32:00 [---] [cpu_sched_debug] preliminary job list: 20:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:32:00 [---] [cpu_sched_debug] final job list: 20:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:32:00 [---] [cpu_sched_debug] enforce_run_list: end 20:32:00 [---] choose_project(): 73920.000000 20:32:00 [WUProp@Home] doing RPC to get work 20:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361679.09 cpu 2110412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110412.90 20:32:00 [---] Got 0 tasks 20:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:34:00 [---] [rr_sim] doing sim: CPU sched 20:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970423138.14G/439.78G) 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206616.02 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 220661.60 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220661.602197 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985954.42 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 198595.44 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419257.044174 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787358.98 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 178735.90 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597992.941953 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608623.08 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 160862.31 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758855.249955 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447760.77 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 144776.08 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903631.327156 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302984.69 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 130298.47 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033929.796638 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172686.23 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 117268.62 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151198.419171 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055417.60 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 105541.76 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256740.179450 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949875.84 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 94987.58 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351727.763702 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854888.26 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 85488.83 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437216.589529 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769399.43 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 76939.94 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514156.532773 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692459.49 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 69245.95 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583402.481692 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623213.54 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 62321.35 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645723.835720 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560892.19 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 56089.22 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701813.054345 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504802.97 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 50480.30 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752293.351107 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454322.67 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 45432.27 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797725.618193 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408890.40 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 40889.04 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838614.658571 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 368001.36 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 36800.14 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875414.794911 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331201.23 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 33120.12 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908534.917617 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298081.10 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 29808.11 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938343.028052 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268272.99 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 26827.30 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965170.327443 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241445.69 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 24144.57 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989314.896896 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217301.13 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 21730.11 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2011045.009403 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195571.01 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 19557.10 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030602.110660 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176013.91 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 17601.39 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048203.501791 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158412.52 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 15841.25 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064044.753808 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142571.27 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 14257.13 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078301.880624 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128314.14 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 12831.41 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091133.294759 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115482.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 11548.27 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102681.567480 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103934.45 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 10393.45 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113075.012929 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93541.01 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 9354.10 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122429.113833 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84186.91 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 8418.69 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130847.804646 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75768.22 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 7576.82 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138424.626379 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68191.40 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 6819.14 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145243.765938 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61372.26 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 6137.23 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151380.991541 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55235.03 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 5523.50 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156904.494583 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49711.53 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 4971.15 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161875.647322 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44740.37 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 4474.04 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166349.684787 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40266.34 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 4026.63 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170376.318505 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36239.70 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3623.97 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174000.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32615.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177600.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29015.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181200.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25415.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184800.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21815.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188400.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18215.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192000.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14615.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195600.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11015.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199200.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7415.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202800.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3815.73 sec) 20:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206400.288851 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215.73 sec) 20:34:00 [GPUGRID] [rr_sim] 2206616.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970423138.14G/439.78G) 20:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853696.93 20:34:00 [---] pick_jobs_to_run() start 20:34:00 [---] pick_jobs_to_run() end 20:34:00 [---] [rr_sim] end 20:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:34:00 [---] [cpu_sched_debug] preliminary job list: 20:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:34:00 [---] [cpu_sched_debug] final job list: 20:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:34:00 [---] [cpu_sched_debug] enforce_run_list: end 20:34:00 [---] choose_project(): 74040.000000 20:34:00 [WUProp@Home] doing RPC to get work 20:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361559.09 cpu 2110352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110352.90 20:34:00 [---] Got 0 tasks 20:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:35:00 [---] [rr_sim] doing sim: CPU sched 20:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970395547.79G/439.78G) 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206553.29 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 220655.33 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220655.328509 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985897.96 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 198589.80 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419245.124166 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787308.16 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 178730.82 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597975.940258 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608577.34 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 160857.73 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758833.674741 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447719.61 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 144771.96 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903605.635776 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302947.65 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 130294.76 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033900.400707 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172652.88 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 117265.29 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151165.689145 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055387.60 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 105538.76 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256704.448739 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949848.84 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 94984.88 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351689.332374 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854863.95 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 85486.40 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437175.727645 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769377.56 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 76937.76 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514113.483389 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692439.80 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 69243.98 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583357.463559 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623195.82 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 62319.58 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645677.045712 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560876.24 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 56087.62 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701764.669649 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504788.62 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 50478.86 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752243.531193 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454309.75 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 45430.98 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797674.506582 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408878.78 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 40887.88 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838562.384433 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367990.90 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 36799.09 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875361.474498 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331191.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 33119.18 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908480.655557 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298072.63 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 29807.26 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938287.918510 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268265.37 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 26826.54 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965114.455167 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241438.83 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 24143.88 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989258.338159 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217294.95 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 21729.49 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010987.832852 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195565.45 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 19556.55 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030544.378076 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176008.91 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 17600.89 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048145.268777 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158408.02 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 15840.80 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063986.070408 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142567.21 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 14256.72 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078242.791875 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128310.49 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 12831.05 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091073.841197 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115479.44 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 11547.94 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102621.785586 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103931.50 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 10393.15 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113014.935536 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93538.35 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 9353.83 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122368.770491 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84184.51 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 8418.45 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130787.221950 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75766.06 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 7576.61 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138363.828264 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68189.46 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 6818.95 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145182.773946 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61370.51 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 6137.05 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151319.825060 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55233.46 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 5523.35 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156843.171063 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49710.11 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 4971.01 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161814.182465 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44739.10 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 4473.91 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166288.092727 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40265.19 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 4026.52 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170314.611963 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36238.67 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3623.87 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173938.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32614.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177538.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29014.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181138.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25414.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184738.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21814.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188338.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18214.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191938.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14614.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195538.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11014.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199138.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7414.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202738.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3814.81 sec) 20:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206338.479276 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 214.81 sec) 20:35:00 [GPUGRID] [rr_sim] 2206553.29: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970395547.79G/439.78G) 20:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853694.20 20:35:00 [---] pick_jobs_to_run() start 20:35:00 [---] pick_jobs_to_run() end 20:35:00 [---] [rr_sim] end 20:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:35:00 [---] [cpu_sched_debug] preliminary job list: 20:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:35:00 [---] [cpu_sched_debug] final job list: 20:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:35:00 [---] [cpu_sched_debug] enforce_run_list: end 20:35:00 [---] choose_project(): 74100.000000 20:35:00 [WUProp@Home] doing RPC to get work 20:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361499.09 cpu 2110292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110292.90 20:35:00 [---] Got 0 tasks 20:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:36:00 [---] [rr_sim] doing sim: CPU sched 20:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970367957.43G/439.78G) 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206490.55 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 220649.05 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220649.054820 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985841.49 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 198584.15 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419233.204159 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787257.34 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 178725.73 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597958.938563 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608531.61 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 160853.16 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758812.099527 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447678.45 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 144767.84 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903579.944395 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302910.60 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 130291.06 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033871.004776 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172619.54 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 117261.95 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151132.959119 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055357.59 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 105535.76 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256668.718027 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949821.83 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 94982.18 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351650.901045 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854839.65 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 85483.96 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437134.865761 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769355.68 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 76935.57 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514070.434005 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692420.11 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 69242.01 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583312.445425 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623178.10 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 62317.81 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645630.255703 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560860.29 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 56086.03 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701716.284953 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504774.26 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 50477.43 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752193.711278 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454296.84 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 45429.68 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797623.394971 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408867.15 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 40886.72 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838510.110294 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367980.44 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 36798.04 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875308.154085 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331182.39 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 33118.24 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908426.393497 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298064.15 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 29806.42 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938232.808968 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268257.74 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 26825.77 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965058.582892 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241431.97 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 24143.20 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989201.779423 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217288.77 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 21728.88 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010930.656301 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195559.89 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 19555.99 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030486.645491 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176003.90 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 17600.39 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048087.035763 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158403.51 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 15840.35 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063927.387007 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142563.16 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 14256.32 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078183.703127 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128306.85 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 12830.68 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091014.387634 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115476.16 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 11547.62 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102562.003691 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103928.54 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 10392.85 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112954.858143 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93535.69 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 9353.57 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122308.427149 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84182.12 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 8418.21 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130726.639254 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75763.91 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 7576.39 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138303.030149 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68187.52 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 6818.75 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145121.781955 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61368.77 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 6136.88 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151258.658580 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55231.89 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 5523.19 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156781.847542 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49708.70 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 4970.87 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161752.717608 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44737.83 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 4473.78 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166226.500668 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40264.05 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 4026.40 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170252.905422 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36237.64 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3623.76 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173876.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32613.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177476.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29013.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181076.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25413.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184676.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21813.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188276.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18213.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191876.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14613.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195476.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11013.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199076.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7413.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202676.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3813.88 sec) 20:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206276.669700 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 213.88 sec) 20:36:00 [GPUGRID] [rr_sim] 2206490.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970367957.43G/439.78G) 20:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853691.46 20:36:00 [---] pick_jobs_to_run() start 20:36:00 [---] pick_jobs_to_run() end 20:36:00 [---] [rr_sim] end 20:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:36:00 [---] [cpu_sched_debug] preliminary job list: 20:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:36:00 [---] [cpu_sched_debug] final job list: 20:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:36:00 [---] [cpu_sched_debug] enforce_run_list: end 20:36:00 [---] choose_project(): 74160.000000 20:36:00 [WUProp@Home] doing RPC to get work 20:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361439.09 cpu 2110232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110232.90 20:36:00 [---] Got 0 tasks 20:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:37:00 [---] [rr_sim] doing sim: CPU sched 20:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970340367.07G/439.78G) 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206427.81 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 220642.78 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220642.781132 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985785.03 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 198578.50 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419221.284151 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787206.53 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 178720.65 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597941.936868 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608485.87 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 160848.59 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758790.524314 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447637.29 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 144763.73 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903554.253015 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302873.56 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 130287.36 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033841.608845 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172586.20 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 117258.62 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151100.229093 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055327.58 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 105532.76 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256632.987316 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949794.82 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 94979.48 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351612.469717 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854815.34 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 85481.53 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437094.003877 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769333.81 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 76933.38 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1514027.384622 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692400.43 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 69240.04 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583267.427292 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623160.38 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 62316.04 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645583.465695 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560844.35 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 56084.43 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701667.900257 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504759.91 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 50475.99 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752143.891364 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454283.92 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 45428.39 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797572.283360 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408855.53 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 40885.55 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838457.836156 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367969.98 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 36797.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875254.833673 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331172.98 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 33117.30 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908372.131438 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298055.68 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 29805.57 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938177.699426 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268250.11 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 26825.01 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1965002.710616 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241425.10 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 24142.51 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989145.220686 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217282.59 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 21728.26 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010873.479750 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195554.33 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 19555.43 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030428.912907 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175998.90 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 17599.89 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2048028.802749 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158399.01 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 15839.90 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063868.703606 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142559.11 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 14255.91 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078124.614378 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128303.20 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 12830.32 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090954.934072 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115472.88 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 11547.29 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102502.221797 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103925.59 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 10392.56 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112894.780750 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93533.03 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 9353.30 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122248.083807 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84179.73 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 8417.97 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130666.056558 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75761.75 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 7576.18 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138242.232035 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68185.58 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 6818.56 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145060.789963 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61367.02 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 6136.70 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151197.492099 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55230.32 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 5523.03 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156720.524022 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49707.29 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 4970.73 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161691.252752 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44736.56 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 4473.66 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166164.908609 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40262.90 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 4026.29 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170191.198880 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36236.61 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3623.66 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173814.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32612.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177414.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29012.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181014.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25412.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184614.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21812.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188214.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18212.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191814.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14612.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195414.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11012.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199014.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7412.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202614.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3812.95 sec) 20:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206214.860124 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 212.95 sec) 20:37:00 [GPUGRID] [rr_sim] 2206427.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970340367.07G/439.78G) 20:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853688.72 20:37:00 [---] pick_jobs_to_run() start 20:37:00 [---] pick_jobs_to_run() end 20:37:00 [---] [rr_sim] end 20:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:37:00 [---] [cpu_sched_debug] preliminary job list: 20:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:37:00 [---] [cpu_sched_debug] final job list: 20:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:37:00 [---] [cpu_sched_debug] enforce_run_list: end 20:37:00 [---] choose_project(): 74220.000000 20:37:00 [WUProp@Home] doing RPC to get work 20:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361379.09 cpu 2110172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110172.90 20:37:00 [---] Got 0 tasks 20:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:38:00 [---] [rr_sim] doing sim: CPU sched 20:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970312776.72G/439.78G) 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206365.07 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 220636.51 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220636.507444 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985728.57 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 198572.86 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419209.364144 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787155.71 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 178715.57 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597924.935173 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608440.14 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 160844.01 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758768.949100 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447596.13 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 144759.61 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903528.561634 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302836.51 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 130283.65 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033812.212915 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172552.86 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 117255.29 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151067.499067 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055297.58 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 105529.76 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256597.256604 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949767.82 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 94976.78 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351574.038388 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854791.04 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 85479.10 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437053.141993 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769311.93 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 76931.19 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513984.335238 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692380.74 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 69238.07 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583222.409158 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623142.67 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 62314.27 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645536.675686 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560828.40 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 56082.84 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701619.515562 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504745.56 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 50474.56 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752094.071449 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454271.00 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 45427.10 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797521.171748 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408843.90 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 40884.39 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838405.562018 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367959.51 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 36795.95 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875201.513260 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331163.56 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 33116.36 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908317.869378 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298047.21 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 29804.72 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938122.589884 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268242.48 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 26824.25 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964946.838340 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241418.24 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 24141.82 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989088.661950 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217276.41 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 21727.64 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010816.303199 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195548.77 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 19554.88 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030371.180323 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175993.89 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 17599.39 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047970.569735 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158394.50 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 15839.45 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063810.020205 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142555.05 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 14255.51 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078065.525629 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128299.55 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 12829.95 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090895.480510 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115469.59 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 11546.96 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102442.439903 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103922.63 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 10392.26 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112834.703356 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93530.37 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 9353.04 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122187.740465 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84177.33 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 8417.73 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130605.473862 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75759.60 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 7575.96 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138181.433920 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68183.64 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 6818.36 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144999.797972 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61365.28 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 6136.53 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151136.325619 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55228.75 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 5522.87 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156659.200501 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49705.87 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 4970.59 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161629.787895 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44735.29 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 4473.53 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166103.316549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40261.76 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 4026.18 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170129.492338 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36235.58 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3623.56 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173753.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32612.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177353.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29012.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180953.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25412.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184553.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21812.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188153.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18212.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191753.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14612.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195353.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11012.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198953.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7412.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202553.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3812.02 sec) 20:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206153.050549 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 212.02 sec) 20:38:00 [GPUGRID] [rr_sim] 2206365.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970312776.72G/439.78G) 20:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853685.99 20:38:00 [---] pick_jobs_to_run() start 20:38:00 [---] pick_jobs_to_run() end 20:38:00 [---] [rr_sim] end 20:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:38:00 [---] [cpu_sched_debug] preliminary job list: 20:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:38:00 [---] [cpu_sched_debug] final job list: 20:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:38:00 [---] [cpu_sched_debug] enforce_run_list: end 20:38:00 [---] choose_project(): 74280.000000 20:38:00 [WUProp@Home] doing RPC to get work 20:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361319.09 cpu 2110112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110112.90 20:38:00 [---] Got 0 tasks 20:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:39:00 [---] [rr_sim] doing sim: CPU sched 20:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970285186.36G/439.78G) 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206302.34 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 220630.23 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220630.233756 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985672.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 198567.21 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419197.444136 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787104.89 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 178710.49 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597907.933478 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608394.40 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 160839.44 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758747.373886 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447554.96 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 144755.50 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903502.870253 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302799.47 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 130279.95 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033782.816984 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172519.52 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 117251.95 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151034.769041 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055267.57 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 105526.76 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256561.525893 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949740.81 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 94974.08 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351535.607059 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854766.73 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 85476.67 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1437012.280109 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769290.06 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 76929.01 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513941.285854 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692361.05 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 69236.11 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583177.391024 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623124.95 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 62312.49 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645489.885678 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560812.45 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 56081.25 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701571.130866 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504731.21 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 50473.12 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1752044.251535 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454258.09 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 45425.81 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797470.060137 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408832.28 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 40883.23 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838353.287879 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367949.05 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 36794.90 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875148.192847 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331154.14 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 33115.41 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908263.607318 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298038.73 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 29803.87 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938067.480342 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268234.86 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 26823.49 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964890.966064 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241411.37 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 24141.14 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1989032.103213 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217270.23 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 21727.02 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010759.126648 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195543.21 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 19554.32 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030313.447739 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175988.89 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 17598.89 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047912.336721 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158390.00 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 15839.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063751.336804 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142551.00 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 14255.10 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078006.436880 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128295.90 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 12829.59 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090836.026947 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115466.31 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 11546.63 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102382.658008 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103919.68 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 10391.97 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112774.625963 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93527.71 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 9352.77 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122127.397123 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84174.94 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 8417.49 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130544.891166 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75757.45 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 7575.74 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138120.635805 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68181.70 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 6818.17 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144938.805981 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61363.53 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 6136.35 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151075.159138 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55227.18 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 5522.72 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156597.876980 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49704.46 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 4970.45 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161568.323038 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44734.01 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 4473.40 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166041.724490 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40260.61 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 4026.06 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170067.785797 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36234.55 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3623.46 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173691.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32611.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177291.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29011.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180891.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25411.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184491.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21811.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188091.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18211.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191691.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14611.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195291.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11011.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198891.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7411.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202491.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3811.10 sec) 20:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206091.240973 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 211.10 sec) 20:39:00 [GPUGRID] [rr_sim] 2206302.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970285186.36G/439.78G) 20:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853683.25 20:39:00 [---] pick_jobs_to_run() start 20:39:00 [---] pick_jobs_to_run() end 20:39:00 [---] [rr_sim] end 20:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:39:00 [---] [cpu_sched_debug] preliminary job list: 20:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:39:00 [---] [cpu_sched_debug] final job list: 20:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:39:00 [---] [cpu_sched_debug] enforce_run_list: end 20:39:00 [---] choose_project(): 74340.000000 20:39:00 [WUProp@Home] doing RPC to get work 20:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361259.09 cpu 2110052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2110052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2110052.90 20:39:00 [---] Got 0 tasks 20:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:40:00 [---] [rr_sim] doing sim: CPU sched 20:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970257596.00G/439.78G) 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206239.60 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 220623.96 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220623.960068 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985615.64 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 198561.56 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419185.524128 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787054.08 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 178705.41 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597890.931783 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608348.67 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 160834.87 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758725.798672 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447513.80 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 144751.38 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903477.178873 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302762.42 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 130276.24 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033753.421053 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172486.18 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 117248.62 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1151002.039015 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055237.56 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 105523.76 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256525.795181 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949713.81 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 94971.38 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351497.175731 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854742.42 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 85474.24 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436971.418225 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769268.18 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 76926.82 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513898.236470 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692341.36 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 69234.14 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583132.372891 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623107.23 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 62310.72 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645443.095669 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560796.51 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 56079.65 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701522.746170 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504716.85 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 50471.69 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751994.431620 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454245.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 45424.52 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797418.948526 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408820.65 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 40882.07 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838301.013741 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367938.59 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 36793.86 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875094.872434 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331144.73 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 33114.47 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908209.345259 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298030.26 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 29803.03 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1938012.370800 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268227.23 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 26822.72 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964835.093788 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241404.51 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 24140.45 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988975.544477 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217264.06 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 21726.41 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010701.950096 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195537.65 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 19553.77 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030255.715154 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175983.89 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 17598.39 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047854.103707 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158385.50 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 15838.55 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063692.653403 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142546.95 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 14254.69 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077947.348131 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128292.25 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 12829.23 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090776.573385 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115463.03 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 11546.30 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102322.876114 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103916.72 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 10391.67 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112714.548570 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93525.05 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 9352.51 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122067.053781 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84172.55 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 8417.25 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130484.308470 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75755.29 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 7575.53 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138059.837691 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68179.76 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 6817.98 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144877.813989 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61361.79 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 6136.18 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151013.992658 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55225.61 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 5522.56 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156536.553460 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49703.05 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 4970.30 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161506.858181 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44732.74 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 4473.27 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165980.132431 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40259.47 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 4025.95 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170006.079255 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36233.52 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3623.35 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173629.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32610.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177229.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29010.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180829.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25410.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184429.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21810.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188029.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18210.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191629.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14610.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195229.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11010.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198829.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7410.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202429.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3810.17 sec) 20:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2206029.431397 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 210.17 sec) 20:40:00 [GPUGRID] [rr_sim] 2206239.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970257596.00G/439.78G) 20:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853680.51 20:40:00 [---] pick_jobs_to_run() start 20:40:00 [---] pick_jobs_to_run() end 20:40:00 [---] [rr_sim] end 20:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:40:00 [---] [cpu_sched_debug] preliminary job list: 20:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:40:00 [---] [cpu_sched_debug] final job list: 20:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:40:00 [---] [cpu_sched_debug] enforce_run_list: end 20:40:00 [---] choose_project(): 74400.000000 20:40:00 [WUProp@Home] doing RPC to get work 20:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361199.09 cpu 2109992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109992.90 20:40:00 [---] Got 0 tasks 20:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:41:00 [---] [rr_sim] doing sim: CPU sched 20:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970230005.65G/439.78G) 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206176.86 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 220617.69 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220617.686379 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985559.18 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 198555.92 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419173.604121 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1787003.26 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 178700.33 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597873.930088 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608302.93 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 160830.29 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758704.223459 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447472.64 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 144747.26 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903451.487492 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302725.38 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 130272.54 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033724.025122 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172452.84 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 117245.28 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150969.308989 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055207.55 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 105520.76 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256490.064470 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949686.80 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 94968.68 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351458.744402 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854718.12 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 85471.81 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436930.556341 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769246.31 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 76924.63 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513855.187086 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692321.68 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 69232.17 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583087.354757 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623089.51 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 62308.95 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645396.305661 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560780.56 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 56078.06 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701474.361474 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504702.50 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 50470.25 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751944.611706 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454232.25 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 45423.23 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797367.836915 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408809.03 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 40880.90 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838248.739603 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367928.12 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 36792.81 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1875041.552022 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331135.31 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 33113.53 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908155.083199 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298021.78 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 29802.18 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937957.261258 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268219.60 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 26821.96 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964779.221512 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241397.64 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 24139.76 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988918.985740 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217257.88 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 21725.79 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010644.773545 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195532.09 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 19553.21 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030197.982570 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175978.88 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 17597.89 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047795.870693 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158380.99 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 15838.10 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063633.970003 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142542.89 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 14254.29 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077888.259382 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128288.60 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 12828.86 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090717.119823 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115459.74 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 11545.97 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102263.094220 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103913.77 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 10391.38 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112654.471177 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93522.39 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 9352.24 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122006.710439 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84170.15 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 8417.02 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130423.725774 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75753.14 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 7575.31 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137999.039576 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68177.82 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 6817.78 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144816.821998 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61360.04 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 6136.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150952.826178 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55224.04 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 5522.40 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156475.229939 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49701.63 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 4970.16 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161445.393325 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44731.47 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 4473.15 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165918.540371 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40258.32 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 4025.83 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169944.372714 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36232.49 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3623.25 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173567.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32609.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177167.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29009.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180767.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25409.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184367.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21809.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187967.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18209.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191567.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14609.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195167.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11009.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198767.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7409.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202367.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3809.24 sec) 20:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205967.621822 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 209.24 sec) 20:41:00 [GPUGRID] [rr_sim] 2206176.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970230005.65G/439.78G) 20:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853677.78 20:41:00 [---] pick_jobs_to_run() start 20:41:00 [---] pick_jobs_to_run() end 20:41:00 [---] [rr_sim] end 20:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:41:00 [---] [cpu_sched_debug] preliminary job list: 20:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:41:00 [---] [cpu_sched_debug] final job list: 20:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:41:00 [---] [cpu_sched_debug] enforce_run_list: end 20:41:00 [---] choose_project(): 74460.000000 20:41:00 [WUProp@Home] doing RPC to get work 20:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361139.09 cpu 2109932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109932.90 20:41:00 [---] Got 0 tasks 20:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:42:00 [---] [rr_sim] doing sim: CPU sched 20:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970202415.29G/439.78G) 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206114.13 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 220611.41 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220611.412691 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985502.71 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 198550.27 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419161.684113 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786952.44 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 178695.24 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597856.928393 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608257.20 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 160825.72 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758682.648245 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447431.48 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 144743.15 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903425.796111 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302688.33 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 130268.83 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033694.629191 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172419.50 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 117241.95 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150936.578963 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055177.55 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 105517.75 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256454.333758 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949659.79 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 94965.98 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351420.313074 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854693.81 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 85469.38 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436889.694457 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769224.43 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 76922.44 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513812.137703 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692301.99 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 69230.20 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1583042.336624 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623071.79 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 62307.18 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645349.515652 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560764.61 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 56076.46 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701425.976778 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504688.15 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 50468.82 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751894.791792 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454219.34 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 45421.93 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797316.725304 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408797.40 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 40879.74 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838196.465464 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367917.66 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 36791.77 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874988.231609 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331125.90 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 33112.59 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908100.821139 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298013.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 29801.33 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937902.151716 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268211.98 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 26821.20 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964723.349236 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241390.78 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 24139.08 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988862.427003 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217251.70 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 21725.17 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010587.596994 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195526.53 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 19552.65 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030140.249986 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175973.88 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 17597.39 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047737.637678 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158376.49 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 15837.65 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063575.286602 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142538.84 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 14253.88 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077829.170633 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128284.96 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 12828.50 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090657.666261 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115456.46 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 11545.65 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102203.312326 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103910.81 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 10391.08 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112594.393784 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93519.73 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 9351.97 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121946.367097 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84167.76 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 8416.78 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130363.143078 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75750.98 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 7575.10 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137938.241462 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68175.89 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 6817.59 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144755.830007 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61358.30 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 6135.83 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150891.659697 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55222.47 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 5522.25 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156413.906419 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49700.22 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 4970.02 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161383.928468 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44730.20 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 4473.02 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165856.948312 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40257.18 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 4025.72 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169882.666172 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36231.46 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3623.15 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173505.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32608.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177105.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29008.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180705.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25408.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184305.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21808.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187905.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18208.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191505.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14608.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195105.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11008.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198705.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7408.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202305.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3808.31 sec) 20:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205905.812246 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 208.31 sec) 20:42:00 [GPUGRID] [rr_sim] 2206114.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970202415.29G/439.78G) 20:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853675.04 20:42:00 [---] pick_jobs_to_run() start 20:42:00 [---] pick_jobs_to_run() end 20:42:00 [---] [rr_sim] end 20:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:42:00 [---] [cpu_sched_debug] preliminary job list: 20:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:42:00 [---] [cpu_sched_debug] final job list: 20:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:42:00 [---] [cpu_sched_debug] enforce_run_list: end 20:42:00 [---] choose_project(): 74520.000000 20:42:00 [WUProp@Home] doing RPC to get work 20:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361079.09 cpu 2109872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109872.90 20:42:00 [---] Got 0 tasks 20:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:43:00 [---] [rr_sim] doing sim: CPU sched 20:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970174824.94G/439.78G) 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2206051.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 220605.14 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220605.139003 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985446.25 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 198544.63 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419149.764106 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786901.63 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 178690.16 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597839.926698 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608211.46 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 160821.15 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758661.073031 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447390.32 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 144739.03 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903400.104731 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302651.29 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 130265.13 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033665.233261 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172386.16 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 117238.62 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150903.848938 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055147.54 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 105514.75 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256418.603047 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949632.79 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 94963.28 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351381.881745 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854669.51 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 85466.95 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436848.832573 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769202.56 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 76920.26 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513769.088319 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692282.30 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 69228.23 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582997.318490 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623054.07 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 62305.41 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645302.725644 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560748.66 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 56074.87 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701377.592082 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504673.80 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 50467.38 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751844.971877 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454206.42 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 45420.64 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797265.613692 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408785.78 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 40878.58 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838144.191326 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367907.20 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 36790.72 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874934.911196 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331116.48 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 33111.65 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1908046.559080 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 298004.83 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 29800.48 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937847.042175 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268204.35 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 26820.43 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964667.476960 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241383.91 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 24138.39 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988805.868267 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217245.52 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 21724.55 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010530.420443 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195520.97 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 19552.10 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030082.517402 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175968.87 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 17596.89 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047679.404664 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158371.99 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 15837.20 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063516.603201 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142534.79 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 14253.48 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077770.081884 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128281.31 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 12828.13 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090598.212698 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115453.18 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 11545.32 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102143.530431 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103907.86 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 10390.79 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112534.316391 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93517.07 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 9351.71 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121886.023755 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84165.37 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 8416.54 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130302.560382 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75748.83 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 7574.88 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137877.443347 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68173.95 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 6817.39 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144694.838015 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61356.55 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 6135.66 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150830.493217 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55220.90 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 5522.09 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156352.582898 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49698.81 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 4969.88 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161322.463611 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44728.93 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 4472.89 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165795.356253 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40256.03 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 4025.60 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169820.959630 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36230.43 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3623.04 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173444.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32607.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177044.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29007.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180644.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25407.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184244.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21807.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187844.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18207.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191444.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14607.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195044.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11007.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198644.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7407.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202244.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3807.39 sec) 20:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205844.002670 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 207.39 sec) 20:43:00 [GPUGRID] [rr_sim] 2206051.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970174824.94G/439.78G) 20:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853672.30 20:43:00 [---] pick_jobs_to_run() start 20:43:00 [---] pick_jobs_to_run() end 20:43:00 [---] [rr_sim] end 20:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:43:00 [---] [cpu_sched_debug] preliminary job list: 20:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:43:00 [---] [cpu_sched_debug] final job list: 20:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:43:00 [---] [cpu_sched_debug] enforce_run_list: end 20:43:00 [---] choose_project(): 74580.000000 20:43:00 [WUProp@Home] doing RPC to get work 20:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 361019.09 cpu 2109812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109812.90 20:43:00 [---] Got 0 tasks 20:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:44:00 [---] [rr_sim] doing sim: CPU sched 20:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970147234.58G/439.78G) 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205988.65 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 220598.87 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220598.865315 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985389.79 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 198538.98 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419137.844098 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786850.81 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 178685.08 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597822.925003 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608165.73 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 160816.57 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758639.497817 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447349.16 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 144734.92 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903374.413350 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302614.24 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 130261.42 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033635.837330 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172352.82 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 117235.28 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150871.118912 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055117.53 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 105511.75 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256382.872335 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949605.78 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 94960.58 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351343.450416 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854645.20 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 85464.52 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436807.970689 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769180.68 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 76918.07 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513726.038935 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692262.61 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 69226.26 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582952.300356 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623036.35 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 62303.64 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645255.935635 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560732.72 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 56073.27 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701329.207387 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504659.45 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 50465.94 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751795.151963 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454193.50 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 45419.35 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797214.502081 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408774.15 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 40877.42 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838091.917188 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367896.74 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 36789.67 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874881.590784 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331107.06 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 33110.71 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907992.297020 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297996.36 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 29799.64 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937791.932633 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268196.72 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 26819.67 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964611.604684 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241377.05 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 24137.70 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988749.309530 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217239.34 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 21723.93 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010473.243892 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195515.41 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 19551.54 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2030024.784817 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175963.87 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 17596.39 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047621.171650 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158367.48 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 15836.75 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063457.919800 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142530.73 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 14253.07 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077710.993135 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128277.66 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 12827.77 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090538.759136 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115449.89 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 11544.99 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102083.748537 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103904.90 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 10390.49 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112474.238998 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93514.41 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 9351.44 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121825.680413 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84162.97 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 8416.30 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130241.977686 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75746.68 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 7574.67 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137816.645232 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68172.01 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 6817.20 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144633.846024 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61354.81 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 6135.48 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150769.326736 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55219.33 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 5521.93 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156291.259377 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49697.39 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 4969.74 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161260.998754 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44727.65 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 4472.77 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165733.764194 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40254.89 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 4025.49 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169759.253089 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36229.40 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3622.94 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173382.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32606.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176982.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29006.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180582.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25406.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184182.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21806.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187782.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18206.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191382.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14606.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194982.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11006.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198582.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7406.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202182.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3806.46 sec) 20:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205782.193095 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 206.46 sec) 20:44:00 [GPUGRID] [rr_sim] 2205988.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970147234.58G/439.78G) 20:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853669.57 20:44:00 [---] pick_jobs_to_run() start 20:44:00 [---] pick_jobs_to_run() end 20:44:00 [---] [rr_sim] end 20:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:44:00 [---] [cpu_sched_debug] preliminary job list: 20:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:44:00 [---] [cpu_sched_debug] final job list: 20:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:44:00 [---] [cpu_sched_debug] enforce_run_list: end 20:44:00 [---] choose_project(): 74640.000000 20:44:00 [WUProp@Home] doing RPC to get work 20:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360959.09 cpu 2109752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109752.90 20:44:00 [---] Got 0 tasks 20:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:45:00 [---] [rr_sim] doing sim: CPU sched 20:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970119644.22G/439.78G) 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205925.92 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 220592.59 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220592.591626 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985333.32 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 198533.33 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419125.924090 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786799.99 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 178680.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597805.923308 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608119.99 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 160812.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758617.922603 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447307.99 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 144730.80 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903348.721970 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302577.19 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 130257.72 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033606.441399 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172319.47 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 117231.95 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150838.388886 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055087.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 105508.75 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256347.141624 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949578.77 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 94957.88 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351305.019088 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854620.90 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 85462.09 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436767.108805 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769158.81 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 76915.88 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513682.989551 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692242.93 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 69224.29 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582907.282223 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623018.63 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 62301.86 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645209.145627 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560716.77 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 56071.68 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701280.822691 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504645.09 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 50464.51 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751745.332048 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454180.58 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 45418.06 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797163.390470 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408762.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 40876.25 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1838039.643049 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367886.27 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 36788.63 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874828.270371 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331097.65 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 33109.76 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907938.034960 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297987.88 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 29798.79 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937736.823091 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268189.09 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 26818.91 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964555.732408 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241370.18 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 24137.02 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988692.750794 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217233.17 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 21723.32 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010416.067341 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195509.85 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 19550.98 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029967.052233 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175958.86 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 17595.89 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047562.938636 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158362.98 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 15836.30 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063399.236399 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142526.68 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 14252.67 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077651.904386 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128274.01 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 12827.40 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090479.305574 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115446.61 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 11544.66 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102023.966643 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103901.95 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 10390.19 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112414.161605 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93511.75 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 9351.18 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121765.337071 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84160.58 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 8416.06 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130181.394990 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75744.52 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 7574.45 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137755.847118 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68170.07 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 6817.01 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144572.854033 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61353.06 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 6135.31 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150708.160256 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55217.76 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 5521.78 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156229.935857 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49695.98 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 4969.60 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161199.533897 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44726.38 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 4472.64 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165672.172134 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40253.74 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 4025.37 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169697.546547 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36228.37 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3622.84 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173320.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32605.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176920.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29005.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180520.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25405.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184120.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21805.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187720.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18205.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191320.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14605.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194920.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11005.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198520.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7405.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202120.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3805.53 sec) 20:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205720.383519 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 205.53 sec) 20:45:00 [GPUGRID] [rr_sim] 2205925.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970119644.22G/439.78G) 20:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853666.83 20:45:00 [---] pick_jobs_to_run() start 20:45:00 [---] pick_jobs_to_run() end 20:45:00 [---] [rr_sim] end 20:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:45:00 [---] [cpu_sched_debug] preliminary job list: 20:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:45:00 [---] [cpu_sched_debug] final job list: 20:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:45:00 [---] [cpu_sched_debug] enforce_run_list: end 20:45:00 [---] choose_project(): 74700.000000 20:45:00 [WUProp@Home] doing RPC to get work 20:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360899.09 cpu 2109692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109692.90 20:45:00 [---] Got 0 tasks 20:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:46:00 [---] [rr_sim] doing sim: CPU sched 20:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970092053.87G/439.78G) 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205863.18 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 220586.32 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220586.317938 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985276.86 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 198527.69 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419114.004083 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786749.18 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 178674.92 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597788.921613 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608074.26 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 160807.43 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758596.347390 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447266.83 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 144726.68 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903323.030589 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302540.15 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 130254.01 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033577.045468 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172286.13 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 117228.61 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150805.658860 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055057.52 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 105505.75 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256311.410912 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949551.77 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 94955.18 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351266.587759 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854596.59 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 85459.66 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436726.246921 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769136.93 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 76913.69 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513639.940168 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692223.24 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 69222.32 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582862.264089 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 623000.92 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 62300.09 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645162.355618 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560700.82 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 56070.08 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701232.437995 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504630.74 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 50463.07 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751695.512134 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454167.67 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 45416.77 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797112.278859 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408750.90 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 40875.09 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837987.368911 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367875.81 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 36787.58 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874774.949958 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331088.23 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 33108.82 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907883.772901 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297979.41 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 29797.94 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937681.713549 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268181.47 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 26818.15 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964499.860132 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241363.32 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 24136.33 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988636.192057 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217226.99 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 21722.70 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010358.890790 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195504.29 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 19550.43 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029909.319649 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175953.86 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 17595.39 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047504.705622 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158358.47 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 15835.85 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063340.552998 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142522.63 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 14252.26 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077592.815637 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128270.36 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 12827.04 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090419.852011 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115443.33 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 11544.33 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101964.184749 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103898.99 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 10389.90 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112354.084212 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93509.10 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 9350.91 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121704.993729 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84158.19 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 8415.82 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130120.812294 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75742.37 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 7574.24 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137695.049003 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68168.13 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 6816.81 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144511.862041 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61351.32 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 6135.13 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150646.993775 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55216.19 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 5521.62 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156168.612336 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49694.57 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 4969.46 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161138.069041 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44725.11 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 4472.51 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165610.580075 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40252.60 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 4025.26 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169635.840006 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36227.34 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3622.73 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173258.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32604.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176858.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29004.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180458.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25404.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184058.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21804.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187658.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18204.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191258.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14604.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194858.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11004.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198458.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7404.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202058.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3804.61 sec) 20:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205658.573943 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 204.61 sec) 20:46:00 [GPUGRID] [rr_sim] 2205863.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970092053.87G/439.78G) 20:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853664.09 20:46:00 [---] pick_jobs_to_run() start 20:46:00 [---] pick_jobs_to_run() end 20:46:00 [---] [rr_sim] end 20:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:46:00 [---] [cpu_sched_debug] preliminary job list: 20:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:46:00 [---] [cpu_sched_debug] final job list: 20:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:46:00 [---] [cpu_sched_debug] enforce_run_list: end 20:46:00 [---] choose_project(): 74760.000000 20:46:00 [WUProp@Home] doing RPC to get work 20:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360839.09 cpu 2109632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109632.90 20:46:00 [---] Got 0 tasks 20:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:47:00 [---] [rr_sim] doing sim: CPU sched 20:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970064463.51G/439.78G) 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205800.44 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 220580.04 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220580.044250 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985220.40 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 198522.04 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419102.084075 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786698.36 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 178669.84 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597771.919918 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608028.52 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 160802.85 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758574.772176 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447225.67 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 144722.57 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903297.339208 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302503.10 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 130250.31 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033547.649538 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172252.79 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 117225.28 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150772.928834 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055027.51 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 105502.75 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256275.680201 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949524.76 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 94952.48 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351228.156431 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854572.29 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 85457.23 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436685.385038 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769115.06 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 76911.51 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513596.890784 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692203.55 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 69220.36 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582817.245955 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622983.20 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 62298.32 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645115.565610 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560684.88 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 56068.49 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701184.053299 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504616.39 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 50461.64 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751645.692219 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454154.75 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 45415.48 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797061.167247 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408739.28 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 40873.93 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837935.094773 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367865.35 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 36786.53 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874721.629545 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331078.81 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 33107.88 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907829.510841 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297970.93 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 29797.09 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937626.604007 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268173.84 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 26817.38 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964443.987856 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241356.45 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 24135.65 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988579.633321 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217220.81 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 21722.08 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010301.714239 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195498.73 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 19549.87 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029851.587065 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175948.86 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 17594.89 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047446.472608 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158353.97 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 15835.40 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063281.869598 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142518.57 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 14251.86 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077533.726888 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128266.72 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 12826.67 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090360.398449 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115440.04 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 11544.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101904.402854 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103896.04 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 10389.60 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112294.006819 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93506.44 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 9350.64 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121644.650387 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84155.79 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 8415.58 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130060.229598 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75740.21 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 7574.02 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137634.250889 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68166.19 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 6816.62 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144450.870050 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61349.57 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 6134.96 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150585.827295 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55214.62 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 5521.46 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156107.288815 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49693.15 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 4969.32 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161076.604184 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44723.84 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 4472.38 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165548.988016 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40251.45 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 4025.15 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169574.133464 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36226.31 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3622.63 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173196.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32603.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176796.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29003.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180396.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25403.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183996.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21803.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187596.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18203.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191196.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14603.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194796.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11003.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198396.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7403.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201996.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3803.68 sec) 20:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205596.764368 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 203.68 sec) 20:47:00 [GPUGRID] [rr_sim] 2205800.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970064463.51G/439.78G) 20:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853661.36 20:47:00 [---] pick_jobs_to_run() start 20:47:00 [---] pick_jobs_to_run() end 20:47:00 [---] [rr_sim] end 20:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:47:00 [---] [cpu_sched_debug] preliminary job list: 20:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:47:00 [---] [cpu_sched_debug] final job list: 20:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:47:00 [---] [cpu_sched_debug] enforce_run_list: end 20:47:00 [---] choose_project(): 74820.000000 20:47:00 [WUProp@Home] doing RPC to get work 20:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360779.09 cpu 2109572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109572.90 20:47:00 [---] Got 0 tasks 20:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:48:00 [---] [rr_sim] doing sim: CPU sched 20:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970064463.51G/439.78G) 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205800.44 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 220580.04 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220580.044250 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985220.40 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 198522.04 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419102.084075 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786698.36 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 178669.84 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597771.919918 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1608028.52 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 160802.85 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758574.772176 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447225.67 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 144722.57 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903297.339208 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302503.10 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 130250.31 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033547.649538 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172252.79 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 117225.28 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150772.928834 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1055027.51 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 105502.75 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256275.680201 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949524.76 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 94952.48 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351228.156431 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854572.29 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 85457.23 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436685.385038 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769115.06 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 76911.51 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513596.890784 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692203.55 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 69220.36 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582817.245955 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622983.20 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 62298.32 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645115.565610 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560684.88 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 56068.49 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701184.053299 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504616.39 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 50461.64 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751645.692219 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454154.75 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 45415.48 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797061.167247 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408739.28 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 40873.93 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837935.094773 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367865.35 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 36786.53 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874721.629545 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331078.81 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 33107.88 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907829.510841 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297970.93 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 29797.09 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937626.604007 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268173.84 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 26817.38 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964443.987856 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241356.45 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 24135.65 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988579.633321 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217220.81 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 21722.08 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010301.714239 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195498.73 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 19549.87 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029851.587065 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175948.86 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 17594.89 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047446.472608 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158353.97 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 15835.40 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063281.869598 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142518.57 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 14251.86 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077533.726888 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128266.72 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 12826.67 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090360.398449 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115440.04 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 11544.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101904.402854 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103896.04 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 10389.60 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112294.006819 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93506.44 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 9350.64 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121644.650387 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84155.79 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 8415.58 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130060.229598 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75740.21 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 7574.02 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137634.250889 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68166.19 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 6816.62 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144450.870050 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61349.57 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 6134.96 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150585.827295 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55214.62 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 5521.46 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156107.288815 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49693.15 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 4969.32 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161076.604184 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44723.84 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 4472.38 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165548.988016 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40251.45 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 4025.15 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169574.133464 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36226.31 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3622.63 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173196.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32603.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176796.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29003.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180396.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25403.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183996.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21803.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187596.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18203.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191196.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14603.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194796.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11003.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198396.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7403.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201996.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3803.68 sec) 20:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205596.764368 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 203.68 sec) 20:48:00 [GPUGRID] [rr_sim] 2205800.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970064463.51G/439.78G) 20:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853721.36 20:48:00 [---] pick_jobs_to_run() start 20:48:00 [---] pick_jobs_to_run() end 20:48:00 [---] [rr_sim] end 20:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:48:00 [---] [cpu_sched_debug] preliminary job list: 20:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:48:00 [---] [cpu_sched_debug] final job list: 20:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:48:00 [---] [cpu_sched_debug] enforce_run_list: end 20:48:00 [---] choose_project(): 74880.000000 20:48:00 [WUProp@Home] doing RPC to get work 20:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360719.09 cpu 2109572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109572.90 20:48:00 [---] Got 0 tasks 20:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:50:00 [---] [rr_sim] doing sim: CPU sched 20:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970036873.15G/439.78G) 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205737.71 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 220573.77 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220573.770562 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985163.94 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 198516.39 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419090.164067 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786647.54 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 178664.75 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597754.918223 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607982.79 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 160798.28 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758553.196962 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447184.51 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 144718.45 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903271.647828 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302466.06 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 130246.61 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033518.253607 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172219.45 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 117221.95 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150740.198808 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054997.51 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 105499.75 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256239.949489 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949497.76 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 94949.78 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351189.725102 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854547.98 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 85454.80 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436644.523154 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769093.18 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 76909.32 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513553.841400 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692183.86 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 69218.39 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582772.227822 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622965.48 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 62296.55 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645068.775602 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560668.93 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 56066.89 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701135.668603 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504602.04 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 50460.20 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751595.872305 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454141.83 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 45414.18 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797010.055636 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408727.65 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 40872.76 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837882.820634 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367854.88 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 36785.49 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874668.309133 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331069.40 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 33106.94 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907775.248781 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297962.46 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 29796.25 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937571.494465 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268166.21 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 26816.62 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964388.115580 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241349.59 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 24134.96 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988523.074584 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217214.63 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 21721.46 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010244.537688 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195493.17 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 19549.32 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029793.854481 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175943.85 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 17594.39 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047388.239594 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158349.47 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 15834.95 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063223.186197 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142514.52 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 14251.45 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077474.638139 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128263.07 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 12826.31 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090300.944887 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115436.76 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 11543.68 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101844.620960 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103893.08 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 10389.31 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112233.929426 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93503.78 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 9350.38 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121584.307045 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84153.40 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 8415.34 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129999.646902 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75738.06 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 7573.81 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137573.452774 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68164.25 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 6816.43 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144389.878058 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61347.83 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 6134.78 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150524.660814 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55213.04 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 5521.30 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156045.965295 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49691.74 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 4969.17 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161015.139327 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44722.57 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 4472.26 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165487.395956 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40250.31 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 4025.03 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169512.426922 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36225.28 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3622.53 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173134.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32602.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176734.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29002.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180334.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25402.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183934.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21802.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187534.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18202.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191134.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14602.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194734.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11002.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198334.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7402.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201934.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3802.75 sec) 20:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205534.954792 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 202.75 sec) 20:50:00 [GPUGRID] [rr_sim] 2205737.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970036873.15G/439.78G) 20:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853778.62 20:50:00 [---] pick_jobs_to_run() start 20:50:00 [---] pick_jobs_to_run() end 20:50:00 [---] [rr_sim] end 20:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:50:00 [---] [cpu_sched_debug] preliminary job list: 20:50:00 [---] [cpu_sched_debug] final job list: 20:50:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 20:50:00 [---] [work_fetch] Request work fetch: CPUs idle 20:50:00 [---] [cpu_sched_debug] enforce_run_list: end 20:50:00 [---] choose_project(): 75000.000000 20:50:00 [WUProp@Home] doing RPC to get work 20:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360599.09 cpu 2109512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109512.90 20:50:00 [---] Got 0 tasks 20:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:51:00 [---] [rr_sim] doing sim: CPU sched 20:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970036873.15G/439.78G) 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205737.71 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 220573.77 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220573.770562 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985163.94 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 198516.39 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419090.164067 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786647.54 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 178664.75 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597754.918223 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607982.79 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 160798.28 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758553.196962 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447184.51 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 144718.45 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903271.647828 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302466.06 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 130246.61 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033518.253607 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172219.45 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 117221.95 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150740.198808 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054997.51 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 105499.75 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256239.949489 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949497.76 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 94949.78 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351189.725102 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854547.98 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 85454.80 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436644.523154 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769093.18 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 76909.32 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513553.841400 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692183.86 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 69218.39 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582772.227822 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622965.48 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 62296.55 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645068.775602 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560668.93 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 56066.89 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701135.668603 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504602.04 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 50460.20 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751595.872305 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454141.83 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 45414.18 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1797010.055636 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408727.65 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 40872.76 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837882.820634 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367854.88 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 36785.49 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874668.309133 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331069.40 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 33106.94 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907775.248781 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297962.46 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 29796.25 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937571.494465 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268166.21 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 26816.62 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964388.115580 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241349.59 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 24134.96 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988523.074584 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217214.63 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 21721.46 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010244.537688 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195493.17 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 19549.32 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029793.854481 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175943.85 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 17594.39 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047388.239594 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158349.47 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 15834.95 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063223.186197 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142514.52 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 14251.45 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077474.638139 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128263.07 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 12826.31 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090300.944887 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115436.76 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 11543.68 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101844.620960 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103893.08 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 10389.31 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112233.929426 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93503.78 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 9350.38 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121584.307045 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84153.40 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 8415.34 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129999.646902 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75738.06 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 7573.81 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137573.452774 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68164.25 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 6816.43 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144389.878058 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61347.83 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 6134.78 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150524.660814 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55213.04 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 5521.30 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156045.965295 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49691.74 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 4969.17 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161015.139327 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44722.57 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 4472.26 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165487.395956 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40250.31 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 4025.03 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169512.426922 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36225.28 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3622.53 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173134.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32602.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176734.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29002.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180334.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25402.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183934.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21802.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187534.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18202.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191134.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14602.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194734.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11002.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198334.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7402.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201934.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3802.75 sec) 20:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205534.954792 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 202.75 sec) 20:51:00 [GPUGRID] [rr_sim] 2205737.71: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970036873.15G/439.78G) 20:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853838.62 20:51:00 [---] pick_jobs_to_run() start 20:51:00 [---] pick_jobs_to_run() end 20:51:00 [---] [rr_sim] end 20:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:51:00 [---] [cpu_sched_debug] preliminary job list: 20:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:51:00 [---] [cpu_sched_debug] final job list: 20:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:51:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 970036873.15G 20:51:00 [---] [cpu_sched_debug] enforce_run_list: end 20:51:00 [---] choose_project(): 75060.000000 20:51:00 [WUProp@Home] doing RPC to get work 20:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360539.09 cpu 2109512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109512.90 20:51:00 [---] Got 0 tasks 20:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:52:00 [---] [rr_sim] doing sim: CPU sched 20:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (970009282.80G/439.78G) 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205674.97 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 220567.50 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220567.496874 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985107.47 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 198510.75 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419078.244060 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786596.72 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 178659.67 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597737.916527 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607937.05 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 160793.71 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758531.621748 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447143.35 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 144714.33 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903245.956447 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302429.01 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 130242.90 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033488.857676 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172186.11 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 117218.61 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150707.468782 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054967.50 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 105496.75 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256204.218777 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949470.75 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 94947.07 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351151.293773 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854523.67 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 85452.37 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436603.661270 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769071.31 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 76907.13 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513510.792016 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692164.18 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 69216.42 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582727.209688 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622947.76 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 62294.78 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1645021.985593 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560652.98 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 56065.30 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701087.283907 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504587.68 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 50458.77 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751546.052390 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454128.92 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 45412.89 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796958.944025 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408716.02 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 40871.60 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837830.546496 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367844.42 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 36784.44 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874614.988720 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331059.98 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 33106.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907720.986722 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297953.98 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 29795.40 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937516.384923 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268158.58 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 26815.86 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964332.243304 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241342.73 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 24134.27 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988466.515848 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217208.45 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 21720.85 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010187.361136 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195487.61 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 19548.76 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029736.121896 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175938.85 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 17593.88 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047330.006580 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158344.96 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 15834.50 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063164.502796 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142510.47 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 14251.05 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077415.549390 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128259.42 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 12825.94 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090241.491325 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115433.48 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 11543.35 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101784.839066 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103890.13 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 10389.01 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112173.852033 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93501.12 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 9350.11 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121523.963703 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84151.01 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 8415.10 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129939.064206 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75735.90 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 7573.59 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137512.654659 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68162.31 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 6816.23 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144328.886067 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61346.08 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 6134.61 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150463.494334 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55211.47 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 5521.15 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155984.641774 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49690.33 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 4969.03 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160953.674470 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44721.29 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 4472.13 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165425.803897 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40249.16 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 4024.92 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169450.720381 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36224.25 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3622.42 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173073.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32601.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176673.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29001.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180273.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25401.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183873.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21801.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187473.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18201.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191073.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14601.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194673.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11001.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198273.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7401.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201873.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3801.82 sec) 20:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205473.145216 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 201.82 sec) 20:52:00 [GPUGRID] [rr_sim] 2205674.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (970009282.80G/439.78G) 20:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853835.88 20:52:00 [---] pick_jobs_to_run() start 20:52:00 [---] pick_jobs_to_run() end 20:52:00 [---] [rr_sim] end 20:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:52:00 [---] [cpu_sched_debug] preliminary job list: 20:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 20:52:00 [---] [cpu_sched_debug] final job list: 20:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:52:00 [---] [cpu_sched_debug] enforce_run_list: end 20:52:00 [---] choose_project(): 75120.000000 20:52:00 [WUProp@Home] doing RPC to get work 20:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360479.09 cpu 2109452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109452.90 20:52:00 [---] Got 0 tasks 20:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:53:00 [---] [rr_sim] doing sim: CPU sched 20:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969981692.44G/439.78G) 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205612.23 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 220561.22 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220561.223185 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1985051.01 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 198505.10 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419066.324052 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786545.91 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 178654.59 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597720.914832 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607891.32 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 160789.13 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758510.046535 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447102.19 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 144710.22 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903220.265067 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302391.97 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 130239.20 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033459.461745 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172152.77 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 117215.28 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150674.738756 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054937.49 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 105493.75 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256168.488066 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949443.74 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 94944.37 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351112.862445 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854499.37 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 85449.94 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436562.799386 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769049.43 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 76904.94 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513467.742632 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692144.49 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 69214.45 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582682.191555 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622930.04 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 62293.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644975.195585 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560637.04 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 56063.70 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1701038.899212 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504573.33 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 50457.33 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751496.232476 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454116.00 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 45411.60 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796907.832414 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408704.40 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 40870.44 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837778.272358 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367833.96 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 36783.40 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874561.668307 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331050.56 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 33105.06 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907666.724662 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297945.51 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 29794.55 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937461.275381 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268150.96 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 26815.10 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964276.371028 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241335.86 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 24133.59 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988409.957111 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217202.27 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 21720.23 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010130.184585 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195482.05 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 19548.20 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029678.389312 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175933.84 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 17593.38 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047271.773566 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158340.46 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 15834.05 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063105.819395 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142506.41 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 14250.64 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077356.460641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128255.77 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 12825.58 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090182.037762 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115430.19 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 11543.02 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101725.057171 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103887.17 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 10388.72 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112113.774640 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93498.46 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 9349.85 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121463.620361 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84148.61 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 8414.86 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129878.481510 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75733.75 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 7573.38 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137451.856545 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68160.38 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 6816.04 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144267.894076 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61344.34 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 6134.43 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150402.327854 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55209.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 5520.99 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155923.318254 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49688.91 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 4968.89 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160892.209614 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44720.02 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 4472.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165364.211838 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40248.02 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 4024.80 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169389.013839 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36223.22 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3622.32 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173011.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32600.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176611.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29000.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180211.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25400.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183811.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21800.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187411.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18200.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191011.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14600.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194611.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11000.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198211.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7400.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201811.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3800.90 sec) 20:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205411.335641 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 200.90 sec) 20:53:00 [GPUGRID] [rr_sim] 2205612.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969981692.44G/439.78G) 20:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853833.14 20:53:00 [---] pick_jobs_to_run() start 20:53:00 [---] pick_jobs_to_run() end 20:53:00 [---] [rr_sim] end 20:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:53:00 [---] [cpu_sched_debug] preliminary job list: 20:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:53:00 [---] [cpu_sched_debug] final job list: 20:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:53:00 [---] [cpu_sched_debug] enforce_run_list: end 20:53:00 [---] choose_project(): 75180.000000 20:53:00 [WUProp@Home] doing RPC to get work 20:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360419.09 cpu 2109392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109392.90 20:53:00 [---] Got 0 tasks 20:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:54:00 [---] [rr_sim] doing sim: CPU sched 20:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969954102.09G/439.78G) 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205549.49 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 220554.95 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220554.949497 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984994.55 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 198499.45 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419054.404045 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786495.09 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 178649.51 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597703.913137 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607845.58 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 160784.56 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758488.471321 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447061.02 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 144706.10 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903194.573686 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302354.92 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 130235.49 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033430.065815 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172119.43 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 117211.94 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150642.008730 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054907.49 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 105490.75 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256132.757354 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949416.74 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 94941.67 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351074.431116 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854475.06 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 85447.51 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436521.937502 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769027.56 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 76902.76 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513424.693249 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692124.80 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 69212.48 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582637.173421 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622912.32 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 62291.23 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644928.405576 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560621.09 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 56062.11 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700990.514516 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504558.98 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 50455.90 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751446.412561 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454103.08 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 45410.31 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796856.720802 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408692.77 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 40869.28 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837725.998219 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367823.50 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 36782.35 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874508.347895 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331041.15 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 33104.11 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907612.462602 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297937.03 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 29793.70 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937406.165839 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268143.33 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 26814.33 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964220.498752 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241329.00 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 24132.90 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988353.398374 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217196.10 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 21719.61 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010073.008034 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195476.49 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 19547.65 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029620.656728 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175928.84 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 17592.88 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047213.540552 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158335.95 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 15833.60 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063047.135994 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142502.36 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 14250.24 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077297.371892 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128252.12 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 12825.21 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090122.584200 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115426.91 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 11542.69 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101665.275277 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103884.22 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 10388.42 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112053.697247 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93495.80 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 9349.58 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121403.277019 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84146.22 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 8414.62 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129817.898814 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75731.60 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 7573.16 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137391.058430 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68158.44 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 6815.84 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144206.902084 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61342.59 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 6134.26 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150341.161373 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55208.33 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 5520.83 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155861.994733 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49687.50 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 4968.75 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160830.744757 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44718.75 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 4471.88 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165302.619778 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40246.88 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 4024.69 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169327.307298 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36222.19 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3622.22 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172949.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32599.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176549.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28999.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180149.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25399.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183749.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21799.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187349.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18199.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190949.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14599.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194549.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10999.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198149.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7399.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201749.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3799.97 sec) 20:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205349.526065 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199.97 sec) 20:54:00 [GPUGRID] [rr_sim] 2205549.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969954102.09G/439.78G) 20:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853830.41 20:54:00 [---] pick_jobs_to_run() start 20:54:00 [---] pick_jobs_to_run() end 20:54:00 [---] [rr_sim] end 20:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:54:00 [---] [cpu_sched_debug] preliminary job list: 20:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:54:00 [---] [cpu_sched_debug] final job list: 20:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:54:00 [---] [cpu_sched_debug] enforce_run_list: end 20:54:00 [---] choose_project(): 75240.000000 20:54:00 [WUProp@Home] doing RPC to get work 20:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360359.09 cpu 2109332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109332.90 20:54:00 [---] Got 0 tasks 20:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:55:00 [---] [rr_sim] doing sim: CPU sched 20:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969926511.73G/439.78G) 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205486.76 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 220548.68 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220548.675809 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984938.08 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 198493.81 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419042.484037 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786444.27 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 178644.43 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597686.911442 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607799.85 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 160779.98 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758466.896107 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1447019.86 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 144701.99 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903168.882305 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302317.88 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 130231.79 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033400.669884 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172086.09 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 117208.61 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150609.278704 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054877.48 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 105487.75 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256097.026643 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949389.73 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 94938.97 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1351035.999788 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854450.76 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 85445.08 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436481.075618 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 769005.68 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 76900.57 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513381.643865 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692105.11 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 69210.51 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582592.155287 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622894.60 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 62289.46 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644881.615568 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560605.14 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 56060.51 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700942.129820 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504544.63 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 50454.46 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751396.592647 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454090.17 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 45409.02 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796805.609191 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408681.15 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 40868.11 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837673.724081 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367813.03 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 36781.30 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874455.027482 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331031.73 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 33103.17 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907558.200543 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297928.56 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 29792.86 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937351.056297 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268135.70 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 26813.57 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964164.626477 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241322.13 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 24132.21 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988296.839638 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217189.92 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 21718.99 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2010015.831483 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195470.93 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 19547.09 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029562.924144 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175923.83 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 17592.38 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047155.307538 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158331.45 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 15833.15 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062988.452593 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142498.31 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 14249.83 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077238.283143 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128248.47 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 12824.85 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090063.130638 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115423.63 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 11542.36 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101605.493383 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103881.26 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 10388.13 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111993.619854 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93493.14 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 9349.31 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121342.933677 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84143.82 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 8414.38 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129757.316118 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75729.44 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 7572.94 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137330.260316 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68156.50 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 6815.65 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144145.910093 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61340.85 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 6134.08 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150279.994893 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55206.76 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 5520.68 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155800.671212 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49686.09 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 4968.61 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160769.279900 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44717.48 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 4471.75 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165241.027719 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40245.73 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 4024.57 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169265.600756 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36221.16 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3622.12 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172887.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32599.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176487.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28999.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180087.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25399.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183687.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21799.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187287.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18199.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190887.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14599.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194487.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10999.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198087.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7399.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201687.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3799.04 sec) 20:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205287.716489 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 199.04 sec) 20:55:00 [GPUGRID] [rr_sim] 2205486.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969926511.73G/439.78G) 20:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853827.67 20:55:00 [---] pick_jobs_to_run() start 20:55:00 [---] pick_jobs_to_run() end 20:55:00 [---] [rr_sim] end 20:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:55:00 [---] [cpu_sched_debug] preliminary job list: 20:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:55:00 [---] [cpu_sched_debug] final job list: 20:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:55:00 [---] [cpu_sched_debug] enforce_run_list: end 20:55:00 [---] choose_project(): 75300.000000 20:55:00 [WUProp@Home] doing RPC to get work 20:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360299.09 cpu 2109272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109272.90 20:55:00 [---] Got 0 tasks 20:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:56:00 [---] [rr_sim] doing sim: CPU sched 20:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969898921.37G/439.78G) 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205424.02 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 220542.40 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220542.402121 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984881.62 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 198488.16 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419030.564029 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786393.46 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 178639.35 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597669.909747 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607754.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 160775.41 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758445.320893 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446978.70 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 144697.87 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903143.190925 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302280.83 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 130228.08 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033371.273953 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172052.75 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 117205.27 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150576.548678 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054847.47 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 105484.75 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256061.295931 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949362.73 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 94936.27 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350997.568459 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854426.45 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 85442.65 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436440.213734 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768983.81 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 76898.38 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513338.594481 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692085.43 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 69208.54 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582547.137154 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622876.88 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 62287.69 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644834.825559 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560589.20 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 56058.92 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700893.745124 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504530.28 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 50453.03 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751346.772732 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454077.25 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 45407.72 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796754.497580 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408669.52 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 40866.95 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837621.449943 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367802.57 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 36780.26 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874401.707069 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331022.31 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 33102.23 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907503.938483 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297920.08 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 29792.01 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937295.946755 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268128.07 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 26812.81 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964108.754201 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241315.27 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 24131.53 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988240.280901 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217183.74 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 21718.37 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009958.654932 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195465.37 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 19546.54 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029505.191559 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175918.83 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 17591.88 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047097.074524 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158326.95 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 15832.69 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062929.769193 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142494.25 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 14249.43 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077179.194394 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128244.83 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 12824.48 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090003.677075 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115420.34 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 11542.03 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101545.711489 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103878.31 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 10387.83 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111933.542461 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93490.48 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 9349.05 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121282.590335 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84141.43 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 8414.14 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129696.733422 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75727.29 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 7572.73 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137269.462201 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68154.56 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 6815.46 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144084.918102 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61339.10 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 6133.91 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150218.828412 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55205.19 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 5520.52 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155739.347692 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49684.67 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 4968.47 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160707.815043 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44716.21 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 4471.62 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165179.435660 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40244.59 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 4024.46 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169203.894214 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36220.13 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3622.01 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172825.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32598.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176425.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28998.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180025.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25398.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183625.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21798.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187225.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18198.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190825.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14598.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194425.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10998.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198025.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7398.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201625.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3798.11 sec) 20:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205225.906914 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 198.11 sec) 20:56:00 [GPUGRID] [rr_sim] 2205424.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969898921.37G/439.78G) 20:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853824.93 20:56:00 [---] pick_jobs_to_run() start 20:56:00 [---] pick_jobs_to_run() end 20:56:00 [---] [rr_sim] end 20:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:56:00 [---] [cpu_sched_debug] preliminary job list: 20:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:56:00 [---] [cpu_sched_debug] final job list: 20:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:56:00 [---] [cpu_sched_debug] enforce_run_list: end 20:56:00 [---] choose_project(): 75360.000000 20:56:00 [WUProp@Home] doing RPC to get work 20:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360239.09 cpu 2109212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109212.90 20:56:00 [---] Got 0 tasks 20:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:57:00 [---] [rr_sim] doing sim: CPU sched 20:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969871331.02G/439.78G) 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205361.28 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 220536.13 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220536.128433 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984825.16 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 198482.52 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419018.644022 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786342.64 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 178634.26 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597652.908052 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607708.38 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 160770.84 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758423.745679 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446937.54 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 144693.75 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903117.499544 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302243.78 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 130224.38 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033341.878022 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1172019.41 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 117201.94 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150543.818653 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054817.47 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 105481.75 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1256025.565220 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949335.72 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 94933.57 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350959.137130 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854402.15 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 85440.21 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436399.351850 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768961.93 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 76896.19 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513295.545097 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692065.74 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 69206.57 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582502.119020 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622859.17 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 62285.92 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644788.035551 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560573.25 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 56057.32 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700845.360428 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504515.92 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 50451.59 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751296.952818 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454064.33 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 45406.43 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796703.385969 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408657.90 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 40865.79 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837569.175804 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367792.11 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 36779.21 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874348.386656 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331012.90 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 33101.29 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907449.676423 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297911.61 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 29791.16 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937240.837213 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268120.45 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 26812.04 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1964052.881925 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241308.40 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 24130.84 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988183.722165 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217177.56 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 21717.76 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009901.478381 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195459.81 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 19545.98 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029447.458975 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175913.83 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 17591.38 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2047038.841510 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158322.44 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 15832.24 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062871.085792 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142490.20 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 14249.02 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077120.105645 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128241.18 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 12824.12 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089944.223513 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115417.06 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 11541.71 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101485.929594 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103875.35 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 10387.54 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111873.465067 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93487.82 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 9348.78 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121222.246993 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84139.04 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 8413.90 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129636.150726 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75725.13 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 7572.51 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137208.664086 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68152.62 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 6815.26 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144023.926110 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61337.36 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 6133.74 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150157.661932 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55203.62 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 5520.36 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155678.024171 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49683.26 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 4968.33 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160646.350187 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44714.93 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 4471.49 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165117.843600 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40243.44 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 4024.34 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169142.187673 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36219.10 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3621.91 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172764.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32597.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176364.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28997.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179964.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25397.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183564.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21797.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187164.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18197.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190764.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14597.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194364.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10997.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197964.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7397.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201564.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3797.19 sec) 20:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205164.097338 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 197.19 sec) 20:57:00 [GPUGRID] [rr_sim] 2205361.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969871331.02G/439.78G) 20:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853822.20 20:57:00 [---] pick_jobs_to_run() start 20:57:00 [---] pick_jobs_to_run() end 20:57:00 [---] [rr_sim] end 20:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:57:00 [---] [cpu_sched_debug] preliminary job list: 20:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:57:00 [---] [cpu_sched_debug] final job list: 20:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:57:00 [---] [cpu_sched_debug] enforce_run_list: end 20:57:00 [---] choose_project(): 75420.000000 20:57:00 [WUProp@Home] doing RPC to get work 20:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360179.09 cpu 2109152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109152.90 20:57:00 [---] Got 0 tasks 20:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:58:00 [---] [rr_sim] doing sim: CPU sched 20:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969843740.66G/439.78G) 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205298.55 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 220529.85 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220529.854744 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984768.69 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 198476.87 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 419006.724014 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786291.82 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 178629.18 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597635.906357 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607662.64 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 160766.26 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758402.170466 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446896.38 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 144689.64 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903091.808163 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302206.74 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 130220.67 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033312.482091 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171986.07 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 117198.61 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150511.088627 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054787.46 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 105478.75 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255989.834508 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949308.71 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 94930.87 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350920.705802 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854377.84 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 85437.78 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436358.489966 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768940.06 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 76894.01 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513252.495714 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692046.05 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 69204.61 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582457.100887 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622841.45 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 62284.14 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644741.245542 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560557.30 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 56055.73 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700796.975732 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504501.57 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 50450.16 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751247.132903 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454051.41 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 45405.14 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796652.274357 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408646.27 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 40864.63 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837516.901666 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367781.65 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 36778.16 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874295.066244 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 331003.48 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 33100.35 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907395.414364 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297903.13 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 29790.31 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937185.727672 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268112.82 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 26811.28 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963997.009649 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241301.54 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 24130.15 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988127.163428 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217171.38 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 21717.14 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009844.301830 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195454.25 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 19545.42 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029389.726391 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175908.82 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 17590.88 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046980.608496 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158317.94 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 15831.79 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062812.402391 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142486.15 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 14248.61 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077061.016896 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128237.53 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 12823.75 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089884.769951 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115413.78 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 11541.38 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101426.147700 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103872.40 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 10387.24 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111813.387674 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93485.16 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 9348.52 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121161.903651 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84136.64 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 8413.66 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129575.568030 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75722.98 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 7572.30 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137147.865972 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68150.68 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 6815.07 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143962.934119 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61335.61 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 6133.56 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150096.495451 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55202.05 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 5520.21 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155616.700650 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49681.85 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 4968.18 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160584.885330 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44713.66 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 4471.37 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165056.251541 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40242.30 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 4024.23 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169080.481131 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36218.07 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3621.81 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172702.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32596.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176302.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28996.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179902.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25396.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183502.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21796.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187102.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18196.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190702.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14596.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194302.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10996.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197902.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7396.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201502.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3796.26 sec) 20:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205102.287762 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 196.26 sec) 20:58:00 [GPUGRID] [rr_sim] 2205298.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969843740.66G/439.78G) 20:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853819.46 20:58:00 [---] pick_jobs_to_run() start 20:58:00 [---] pick_jobs_to_run() end 20:58:00 [---] [rr_sim] end 20:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:58:00 [---] [cpu_sched_debug] preliminary job list: 20:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:58:00 [---] [cpu_sched_debug] final job list: 20:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:58:00 [---] [cpu_sched_debug] enforce_run_list: end 20:58:00 [---] choose_project(): 75480.000000 20:58:00 [WUProp@Home] doing RPC to get work 20:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360119.09 cpu 2109092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109092.90 20:58:00 [---] Got 0 tasks 20:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 20:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 20:59:00 [---] [rr_sim] doing sim: CPU sched 20:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969816150.30G/439.78G) 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205235.81 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 220523.58 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220523.581056 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984712.23 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 198471.22 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418994.804007 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786241.01 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 178624.10 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597618.904662 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607616.91 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 160761.69 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758380.595252 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446855.22 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 144685.52 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903066.116783 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302169.69 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 130216.97 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033283.086161 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171952.72 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 117195.27 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150478.358601 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054757.45 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 105475.75 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255954.103797 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949281.71 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 94928.17 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350882.274473 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854353.54 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 85435.35 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436317.628082 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768918.18 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 76891.82 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513209.446330 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692026.36 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 69202.64 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582412.082753 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622823.73 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 62282.37 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644694.455534 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560541.36 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 56054.14 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700748.591036 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504487.22 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 50448.72 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751197.312989 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454038.50 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 45403.85 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796601.162746 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408634.65 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 40863.46 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837464.627528 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367771.18 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 36777.12 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874241.745831 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330994.06 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 33099.41 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907341.152304 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297894.66 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 29789.47 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937130.618130 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268105.19 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 26810.52 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963941.137373 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241294.67 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 24129.47 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988070.604692 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217165.21 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 21716.52 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009787.125279 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195448.69 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 19544.87 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029331.993807 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175903.82 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 17590.38 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046922.375482 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158313.44 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 15831.34 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062753.718990 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142482.09 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 14248.21 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077001.928147 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128233.88 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 12823.39 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089825.316389 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115410.49 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 11541.05 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101366.365806 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103869.44 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 10386.94 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111753.310281 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93482.50 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 9348.25 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121101.560309 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84134.25 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 8413.43 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129514.985334 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75720.83 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 7572.08 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137087.067857 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68148.74 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 6814.87 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143901.942128 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61333.87 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 6133.39 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150035.328971 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55200.48 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 5520.05 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155555.377130 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49680.43 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 4968.04 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160523.420473 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44712.39 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 4471.24 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164994.659482 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40241.15 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 4024.12 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169018.774590 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36217.04 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3621.70 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172640.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32595.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176240.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28995.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179840.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25395.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183440.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21795.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187040.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18195.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190640.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14595.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194240.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10995.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197840.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7395.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201440.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3795.33 sec) 20:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2205040.478187 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195.33 sec) 20:59:00 [GPUGRID] [rr_sim] 2205235.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969816150.30G/439.78G) 20:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853816.72 20:59:00 [---] pick_jobs_to_run() start 20:59:00 [---] pick_jobs_to_run() end 20:59:00 [---] [rr_sim] end 20:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 20:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 20:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 20:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 20:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 20:59:00 [---] [cpu_sched_debug] preliminary job list: 20:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:59:00 [---] [cpu_sched_debug] final job list: 20:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 20:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 20:59:00 [---] [cpu_sched_debug] enforce_run_list: end 20:59:00 [---] choose_project(): 75540.000000 20:59:00 [WUProp@Home] doing RPC to get work 20:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 360059.09 cpu 2109032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2109032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2109032.90 20:59:00 [---] Got 0 tasks 21:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:01:00 [---] [rr_sim] doing sim: CPU sched 21:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969788559.95G/439.78G) 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205173.07 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 220517.31 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220517.307368 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984655.77 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 198465.58 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418982.883999 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786190.19 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 178619.02 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597601.902967 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607571.17 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 160757.12 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758359.020038 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446814.05 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 144681.41 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903040.425402 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302132.65 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 130213.26 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033253.690230 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171919.38 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 117191.94 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150445.628575 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054727.45 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 105472.74 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255918.373085 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949254.70 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 94925.47 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350843.843145 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854329.23 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 85432.92 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436276.766198 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768896.31 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 76889.63 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513166.396946 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 692006.68 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 69200.67 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582367.064619 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622806.01 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 62280.60 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644647.665525 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560525.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 56052.54 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700700.206341 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504472.87 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 50447.29 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751147.493074 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454025.58 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 45402.56 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796550.051135 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408623.02 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 40862.30 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837412.353389 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367760.72 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 36776.07 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874188.425418 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330984.65 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 33098.46 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907286.890244 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297886.18 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 29788.62 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937075.508588 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268097.57 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 26809.76 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963885.265097 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241287.81 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 24128.78 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1988014.045955 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217159.03 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 21715.90 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009729.948727 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195443.12 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 19544.31 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029274.261223 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175898.81 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 17589.88 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046864.142468 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158308.93 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 15830.89 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062695.035589 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142478.04 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 14247.80 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076942.839398 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128230.23 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 12823.02 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089765.862826 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115407.21 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 11540.72 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101306.583912 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103866.49 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 10386.65 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111693.232888 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93479.84 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 9347.98 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121041.216967 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84131.86 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 8413.19 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129454.402638 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75718.67 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 7571.87 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137026.269743 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68146.80 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 6814.68 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143840.950136 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61332.12 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 6133.21 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149974.162490 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55198.91 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 5519.89 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155494.053609 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49679.02 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 4967.90 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160461.955616 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44711.12 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 4471.11 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164933.067422 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40240.01 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 4024.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168957.068048 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36216.01 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3621.60 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172578.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32594.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176178.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28994.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179778.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25394.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183378.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21794.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186978.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18194.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190578.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14594.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194178.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10994.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197778.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7394.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201378.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3794.41 sec) 21:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204978.668611 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194.41 sec) 21:01:00 [GPUGRID] [rr_sim] 2205173.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969788559.95G/439.78G) 21:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853873.99 21:01:00 [---] pick_jobs_to_run() start 21:01:00 [---] pick_jobs_to_run() end 21:01:00 [---] [rr_sim] end 21:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:01:00 [---] [cpu_sched_debug] preliminary job list: 21:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:01:00 [---] [cpu_sched_debug] final job list: 21:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:01:00 [---] [cpu_sched_debug] enforce_run_list: end 21:01:00 [---] choose_project(): 75660.000000 21:01:00 [WUProp@Home] doing RPC to get work 21:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359939.09 cpu 2108972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108972.90 21:01:00 [---] Got 0 tasks 21:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:02:00 [---] [rr_sim] doing sim: CPU sched 21:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969760969.59G/439.78G) 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205110.34 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 220511.03 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220511.033680 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984599.30 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 198459.93 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418970.963991 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786139.37 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 178613.94 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597584.901272 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607525.44 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 160752.54 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758337.444824 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446772.89 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 144677.29 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 903014.734022 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302095.60 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 130209.56 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033224.294299 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171886.04 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 117188.60 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150412.898549 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054697.44 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 105469.74 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255882.642374 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949227.69 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 94922.77 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350805.411816 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854304.92 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 85430.49 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436235.904314 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768874.43 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 76887.44 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513123.347562 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691986.99 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 69198.70 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582322.046486 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622788.29 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 62278.83 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644600.875517 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560509.46 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 56050.95 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700651.821645 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504458.52 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 50445.85 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751097.673160 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 454012.66 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 45401.27 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796498.939524 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408611.40 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 40861.14 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837360.079251 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367750.26 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 36775.03 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874135.105006 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330975.23 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 33097.52 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907232.628185 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297877.71 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 29787.77 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1937020.399046 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268089.94 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 26808.99 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963829.392821 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241280.94 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 24128.09 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987957.487219 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217152.85 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 21715.28 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009672.772176 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195437.56 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 19543.76 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029216.528638 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175893.81 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 17589.38 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046805.909454 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158304.43 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 15830.44 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062636.352188 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142473.98 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 14247.40 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076883.750649 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128226.59 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 12822.66 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089706.409264 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115403.93 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 11540.39 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101246.802017 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103863.53 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 10386.35 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111633.155495 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93477.18 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 9347.72 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120980.873625 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84129.46 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 8412.95 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129393.819942 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75716.52 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 7571.65 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136965.471628 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68144.87 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 6814.49 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143779.958145 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61330.38 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 6133.04 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149912.996010 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55197.34 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 5519.73 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155432.730089 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49677.61 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 4967.76 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160400.490759 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44709.85 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 4470.98 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164871.475363 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40238.86 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 4023.89 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168895.361507 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36214.98 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3621.50 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172516.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32593.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176116.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28993.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179716.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25393.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183316.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21793.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186916.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18193.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190516.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14593.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194116.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10993.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197716.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7393.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201316.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3793.48 sec) 21:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204916.859036 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 193.48 sec) 21:02:00 [GPUGRID] [rr_sim] 2205110.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969760969.59G/439.78G) 21:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853871.25 21:02:00 [---] pick_jobs_to_run() start 21:02:00 [---] pick_jobs_to_run() end 21:02:00 [---] [rr_sim] end 21:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:02:00 [---] [cpu_sched_debug] preliminary job list: 21:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:02:00 [---] [cpu_sched_debug] final job list: 21:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:02:00 [---] [cpu_sched_debug] enforce_run_list: end 21:02:00 [---] choose_project(): 75720.000000 21:02:00 [WUProp@Home] doing RPC to get work 21:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359879.09 cpu 2108912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108912.90 21:02:00 [---] Got 0 tasks 21:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:03:00 [---] [rr_sim] doing sim: CPU sched 21:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969733379.24G/439.78G) 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2205047.60 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 220504.76 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220504.759991 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984542.84 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 198454.28 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418959.043984 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786088.56 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 178608.86 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597567.899577 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607479.70 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 160747.97 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758315.869611 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446731.73 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 144673.17 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902989.042641 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302058.56 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 130205.86 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033194.898368 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171852.70 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 117185.27 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150380.168523 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054667.43 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 105466.74 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255846.911662 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949200.69 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 94920.07 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350766.980487 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854280.62 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 85428.06 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436195.042430 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768852.56 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 76885.26 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513080.298179 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691967.30 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 69196.73 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582277.028352 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622770.57 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 62277.06 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644554.085508 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560493.51 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 56049.35 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700603.436949 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504444.16 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 50444.42 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1751047.853246 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453999.75 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 45399.97 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796447.827912 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408599.77 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 40859.98 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837307.805113 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367739.79 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 36773.98 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874081.784593 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330965.82 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 33096.58 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907178.366125 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297869.23 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 29786.92 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936965.289504 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268082.31 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 26808.23 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963773.520545 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241274.08 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 24127.41 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987900.928482 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217146.67 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 21714.67 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009615.595625 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195432.00 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 19543.20 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029158.796054 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175888.80 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 17588.88 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046747.676440 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158299.92 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 15829.99 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062577.668788 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142469.93 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 14246.99 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076824.661900 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128222.94 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 12822.29 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089646.955702 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115400.64 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 11540.06 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101187.020123 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103860.58 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 10386.06 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111573.078102 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93474.52 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 9347.45 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120920.530283 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84127.07 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 8412.71 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129333.237246 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75714.36 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 7571.44 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136904.673513 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68142.93 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 6814.29 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143718.966153 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61328.63 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 6132.86 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149851.829530 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55195.77 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 5519.58 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155371.406568 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49676.19 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 4967.62 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160339.025903 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44708.57 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 4470.86 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164809.883304 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40237.72 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 4023.77 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168833.654965 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36213.94 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3621.39 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172455.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32592.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176055.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28992.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179655.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25392.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183255.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21792.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186855.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18192.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190455.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14592.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194055.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10992.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197655.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7392.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201255.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3792.55 sec) 21:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204855.049460 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 192.55 sec) 21:03:00 [GPUGRID] [rr_sim] 2205047.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969733379.24G/439.78G) 21:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853868.51 21:03:00 [---] pick_jobs_to_run() start 21:03:00 [---] pick_jobs_to_run() end 21:03:00 [---] [rr_sim] end 21:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:03:00 [---] [cpu_sched_debug] preliminary job list: 21:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:03:00 [---] [cpu_sched_debug] final job list: 21:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:03:00 [---] [cpu_sched_debug] enforce_run_list: end 21:03:00 [---] choose_project(): 75780.000000 21:03:00 [WUProp@Home] doing RPC to get work 21:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359819.09 cpu 2108852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108852.90 21:03:00 [---] Got 0 tasks 21:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:04:00 [---] [rr_sim] doing sim: CPU sched 21:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969705788.88G/439.78G) 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204984.86 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 220498.49 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220498.486303 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984486.38 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 198448.64 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418947.123976 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1786037.74 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 178603.77 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597550.897882 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607433.97 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 160743.40 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758294.294397 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446690.57 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 144669.06 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902963.351260 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1302021.51 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 130202.15 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033165.502438 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171819.36 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 117181.94 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150347.438497 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054637.42 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 105463.74 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255811.180951 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949173.68 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 94917.37 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350728.549159 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854256.31 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 85425.63 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436154.180546 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768830.68 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 76883.07 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1513037.248795 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691947.61 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 69194.76 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582232.010218 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622752.85 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 62275.29 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644507.295500 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560477.57 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 56047.76 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700555.052253 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504429.81 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 50442.98 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750998.033331 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453986.83 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 45398.68 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796396.716301 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408588.15 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 40858.81 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837255.530974 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367729.33 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 36772.93 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1874028.464180 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330956.40 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 33095.64 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907124.104065 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297860.76 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 29786.08 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936910.179962 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268074.68 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 26807.47 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963717.648269 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241267.21 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 24126.72 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987844.369745 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217140.49 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 21714.05 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009558.419074 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195426.44 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 19542.64 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029101.063470 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175883.80 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 17588.38 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046689.443426 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158295.42 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 15829.54 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062518.985387 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142465.88 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 14246.59 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076765.573151 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128219.29 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 12821.93 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089587.502139 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115397.36 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 11539.74 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101127.238229 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103857.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 10385.76 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111513.000709 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93471.86 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 9347.19 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120860.186941 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84124.68 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 8412.47 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129272.654550 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75712.21 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 7571.22 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136843.875399 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68140.99 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 6814.10 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143657.974162 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61326.89 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 6132.69 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149790.663049 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55194.20 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 5519.42 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155310.083047 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49674.78 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 4967.48 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160277.561046 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44707.30 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 4470.73 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164748.291245 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40236.57 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 4023.66 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168771.948423 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36212.91 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3621.29 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172393.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32591.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175993.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28991.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179593.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25391.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183193.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21791.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186793.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18191.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190393.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14591.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193993.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10991.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197593.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7391.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201193.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3791.62 sec) 21:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204793.239884 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 191.62 sec) 21:04:00 [GPUGRID] [rr_sim] 2204984.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969705788.88G/439.78G) 21:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853865.78 21:04:00 [---] pick_jobs_to_run() start 21:04:00 [---] pick_jobs_to_run() end 21:04:00 [---] [rr_sim] end 21:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:04:00 [---] [cpu_sched_debug] preliminary job list: 21:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:04:00 [---] [cpu_sched_debug] final job list: 21:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:04:00 [---] [cpu_sched_debug] enforce_run_list: end 21:04:00 [---] choose_project(): 75840.000000 21:04:00 [WUProp@Home] doing RPC to get work 21:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359759.09 cpu 2108792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108792.90 21:04:00 [---] Got 0 tasks 21:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:05:00 [---] [rr_sim] doing sim: CPU sched 21:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969678198.52G/439.78G) 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204922.13 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 220492.21 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220492.212615 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984429.91 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 198442.99 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418935.203969 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785986.92 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 178598.69 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597533.896187 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607388.23 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 160738.82 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758272.719183 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446649.41 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 144664.94 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902937.659880 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301984.47 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 130198.45 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033136.106507 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171786.02 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 117178.60 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150314.708471 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054607.42 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 105460.74 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255775.450239 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949146.68 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 94914.67 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350690.117830 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854232.01 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 85423.20 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436113.318662 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768808.81 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 76880.88 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512994.199411 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691927.93 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 69192.79 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582186.992085 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622735.13 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 62273.51 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644460.505491 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560461.62 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 56046.16 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700506.667557 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504415.46 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 50441.55 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750948.213417 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453973.91 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 45397.39 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796345.604690 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408576.52 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 40857.65 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837203.256836 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367718.87 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 36771.89 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873975.143767 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330946.98 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 33094.70 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907069.842006 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297852.28 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 29785.23 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936855.070420 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268067.06 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 26806.71 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963661.775993 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241260.35 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 24126.04 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987787.811009 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217134.32 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 21713.43 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009501.242523 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195420.88 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 19542.09 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2029043.330886 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175878.80 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 17587.88 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046631.210412 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158290.92 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 15829.09 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062460.301986 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142461.82 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 14246.18 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076706.484402 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128215.64 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 12821.56 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089528.048577 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115394.08 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 11539.41 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101067.456334 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103854.67 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 10385.47 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111452.923316 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93469.20 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 9346.92 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120799.843599 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84122.28 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 8412.23 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129212.071855 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75710.05 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 7571.01 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136783.077284 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68139.05 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 6813.90 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143596.982171 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61325.14 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 6132.51 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149729.496569 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55192.63 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 5519.26 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155248.759527 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49673.37 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 4967.34 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160216.096189 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44706.03 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 4470.60 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164686.699185 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40235.43 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 4023.54 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168710.241882 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36211.88 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3621.19 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172331.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32590.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175931.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28990.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179531.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25390.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183131.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21790.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186731.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18190.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190331.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14590.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193931.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10990.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197531.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7390.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201131.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3790.70 sec) 21:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204731.430309 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 190.70 sec) 21:05:00 [GPUGRID] [rr_sim] 2204922.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969678198.52G/439.78G) 21:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853863.04 21:05:00 [---] pick_jobs_to_run() start 21:05:00 [---] pick_jobs_to_run() end 21:05:00 [---] [rr_sim] end 21:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:05:00 [---] [cpu_sched_debug] preliminary job list: 21:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:05:00 [---] [cpu_sched_debug] final job list: 21:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:05:00 [---] [cpu_sched_debug] enforce_run_list: end 21:05:00 [---] choose_project(): 75900.000000 21:05:00 [WUProp@Home] doing RPC to get work 21:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359699.09 cpu 2108732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108732.90 21:05:00 [---] Got 0 tasks 21:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:06:00 [---] [rr_sim] doing sim: CPU sched 21:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969650608.17G/439.78G) 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204859.39 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 220485.94 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220485.938927 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984373.45 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 198437.35 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418923.283961 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785936.11 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 178593.61 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597516.894492 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607342.49 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 160734.25 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758251.143969 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446608.25 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 144660.82 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902911.968499 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301947.42 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 130194.74 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033106.710576 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171752.68 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 117175.27 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150281.978445 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054577.41 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 105457.74 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255739.719528 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949119.67 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 94911.97 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350651.686502 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854207.70 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 85420.77 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436072.456778 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768786.93 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 76878.69 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512951.150027 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691908.24 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 69190.82 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582141.973951 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622717.42 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 62271.74 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644413.715483 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560445.67 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 56044.57 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700458.282861 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504401.11 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 50440.11 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750898.393502 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453961.00 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 45396.10 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796294.493079 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408564.90 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 40856.49 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837150.982698 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367708.41 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 36770.84 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873921.823355 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330937.57 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 33093.76 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1907015.579946 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297843.81 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 29784.38 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936799.960878 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268059.43 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 26805.94 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963605.903717 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241253.49 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 24125.35 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987731.252272 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217128.14 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 21712.81 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009444.065972 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195415.32 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 19541.53 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028985.598301 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175873.79 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 17587.38 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046572.977398 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158286.41 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 15828.64 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062401.618585 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142457.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 14245.78 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076647.395653 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128211.99 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 12821.20 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089468.595015 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115390.79 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 11539.08 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101007.674440 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103851.71 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 10385.17 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111392.845923 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93466.54 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 9346.65 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120739.500257 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84119.89 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 8411.99 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129151.489159 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75707.90 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 7570.79 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136722.279169 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68137.11 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 6813.71 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143535.990179 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61323.40 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 6132.34 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149668.330088 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55191.06 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 5519.11 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155187.436006 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49671.95 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 4967.20 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160154.631332 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44704.76 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 4470.48 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164625.107126 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40234.28 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 4023.43 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168648.535340 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36210.85 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3621.09 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172269.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32589.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175869.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28989.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179469.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25389.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183069.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21789.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186669.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18189.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190269.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14589.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193869.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10989.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197469.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7389.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201069.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3789.77 sec) 21:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204669.620733 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 189.77 sec) 21:06:00 [GPUGRID] [rr_sim] 2204859.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969650608.17G/439.78G) 21:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853860.30 21:06:00 [---] pick_jobs_to_run() start 21:06:00 [---] pick_jobs_to_run() end 21:06:00 [---] [rr_sim] end 21:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:06:00 [---] [cpu_sched_debug] preliminary job list: 21:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:06:00 [---] [cpu_sched_debug] final job list: 21:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:06:00 [---] [cpu_sched_debug] enforce_run_list: end 21:06:00 [---] choose_project(): 75960.000000 21:06:00 [WUProp@Home] doing RPC to get work 21:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359639.09 cpu 2108672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108672.90 21:06:00 [---] Got 0 tasks 21:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:07:00 [---] [rr_sim] doing sim: CPU sched 21:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969623017.81G/439.78G) 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204796.65 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 220479.67 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220479.665239 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984316.99 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 198431.70 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418911.363953 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785885.29 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 178588.53 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597499.892797 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607296.76 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 160729.68 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758229.568755 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446567.08 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 144656.71 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902886.277119 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301910.38 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 130191.04 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033077.314645 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171719.34 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 117171.93 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150249.248419 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054547.40 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 105454.74 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255703.988816 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949092.66 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 94909.27 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350613.255173 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854183.40 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 85418.34 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1436031.594894 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768765.06 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 76876.51 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512908.100643 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691888.55 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 69188.86 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582096.955818 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622699.70 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 62269.97 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644366.925474 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560429.73 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 56042.97 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700409.898166 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504386.75 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 50438.68 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750848.573588 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453948.08 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 45394.81 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796243.381467 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408553.27 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 40855.33 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837098.708559 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367697.94 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 36769.79 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873868.502942 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330928.15 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 33092.81 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906961.317886 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297835.33 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 29783.53 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936744.851336 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268051.80 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 26805.18 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963550.031441 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241246.62 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 24124.66 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987674.693536 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217121.96 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 21712.20 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009386.889421 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195409.76 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 19540.98 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028927.865717 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175868.79 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 17586.88 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046514.744384 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158281.91 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 15828.19 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062342.935184 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142453.72 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 14245.37 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076588.306904 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128208.35 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 12820.83 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089409.141453 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115387.51 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 11538.75 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100947.892546 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103848.76 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 10384.88 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111332.768530 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93463.88 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 9346.39 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120679.156915 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84117.50 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 8411.75 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129090.906463 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75705.75 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 7570.57 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136661.481055 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68135.17 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 6813.52 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143474.998188 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61321.65 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 6132.17 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149607.163608 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55189.49 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 5518.95 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155126.112486 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49670.54 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 4967.05 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160093.166476 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44703.49 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 4470.35 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164563.515067 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40233.14 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 4023.31 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168586.828799 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36209.82 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3620.98 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172207.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32588.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175807.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28988.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179407.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25388.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183007.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21788.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186607.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18188.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190207.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14588.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193807.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10988.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197407.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7388.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201007.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3788.84 sec) 21:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204607.811157 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 188.84 sec) 21:07:00 [GPUGRID] [rr_sim] 2204796.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969623017.81G/439.78G) 21:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853857.57 21:07:00 [---] pick_jobs_to_run() start 21:07:00 [---] pick_jobs_to_run() end 21:07:00 [---] [rr_sim] end 21:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:07:00 [---] [cpu_sched_debug] preliminary job list: 21:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:07:00 [---] [cpu_sched_debug] final job list: 21:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:07:00 [---] [cpu_sched_debug] enforce_run_list: end 21:07:00 [---] choose_project(): 76020.000000 21:07:00 [WUProp@Home] doing RPC to get work 21:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359579.09 cpu 2108612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108612.90 21:07:00 [---] Got 0 tasks 21:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:08:00 [---] [rr_sim] doing sim: CPU sched 21:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969595427.46G/439.78G) 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204733.92 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 220473.39 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220473.391550 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984260.52 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 198426.05 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418899.443946 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785834.47 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 178583.45 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597482.891101 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607251.02 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 160725.10 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758207.993542 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446525.92 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 144652.59 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902860.585738 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301873.33 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 130187.33 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033047.918714 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171686.00 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 117168.60 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150216.518393 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054517.40 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 105451.74 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255668.258104 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949065.66 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 94906.57 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350574.823844 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854159.09 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 85415.91 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435990.733010 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768743.18 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 76874.32 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512865.051260 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691868.86 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 69186.89 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582051.937684 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622681.98 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 62268.20 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644320.135466 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560413.78 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 56041.38 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700361.513470 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504372.40 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 50437.24 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750798.753673 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453935.16 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 45393.52 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796192.269856 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408541.65 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 40854.16 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1837046.434421 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367687.48 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 36768.75 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873815.182529 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330918.73 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 33091.87 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906907.055827 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297826.86 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 29782.69 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936689.741794 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268044.17 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 26804.42 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963494.159165 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241239.76 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 24123.98 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987618.134799 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217115.78 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 21711.58 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009329.712870 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195404.20 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 19540.42 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028870.133133 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175863.78 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 17586.38 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046456.511370 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158277.40 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 15827.74 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062284.251783 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142449.66 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 14244.97 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076529.218155 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128204.70 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 12820.47 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089349.687890 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115384.23 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 11538.42 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100888.110652 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103845.80 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 10384.58 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111272.691137 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93461.22 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 9346.12 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120618.813574 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84115.10 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 8411.51 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129030.323767 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75703.59 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 7570.36 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136600.682940 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68133.23 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 6813.32 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143414.006197 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61319.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 6131.99 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149545.997127 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55187.92 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 5518.79 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155064.788965 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49669.13 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 4966.91 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160031.701619 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44702.21 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 4470.22 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164501.923007 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40231.99 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 4023.20 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168525.122257 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36208.79 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3620.88 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172146.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32587.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175746.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28987.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179346.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25387.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182946.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21787.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186546.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18187.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190146.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14587.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193746.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10987.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197346.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7387.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200946.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3787.91 sec) 21:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204546.001582 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 187.91 sec) 21:08:00 [GPUGRID] [rr_sim] 2204733.92: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969595427.46G/439.78G) 21:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853854.83 21:08:00 [---] pick_jobs_to_run() start 21:08:00 [---] pick_jobs_to_run() end 21:08:00 [---] [rr_sim] end 21:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:08:00 [---] [cpu_sched_debug] preliminary job list: 21:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:08:00 [---] [cpu_sched_debug] final job list: 21:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:08:00 [---] [cpu_sched_debug] enforce_run_list: end 21:08:00 [---] choose_project(): 76080.000000 21:08:00 [WUProp@Home] doing RPC to get work 21:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359519.09 cpu 2108552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108552.90 21:08:00 [---] Got 0 tasks 21:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:09:00 [---] [rr_sim] doing sim: CPU sched 21:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969567837.10G/439.78G) 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204671.18 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 220467.12 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220467.117862 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984204.06 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 198420.41 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418887.523938 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785783.65 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 178578.37 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597465.889406 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607205.29 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 160720.53 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758186.418328 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446484.76 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 144648.48 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902834.894357 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301836.28 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 130183.63 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1033018.522784 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171652.66 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 117165.27 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150183.788367 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054487.39 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 105448.74 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255632.527393 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949038.65 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 94903.87 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350536.392516 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854134.79 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 85413.48 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435949.871126 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768721.31 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 76872.13 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512822.001876 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691849.18 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 69184.92 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1582006.919550 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622664.26 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 62266.43 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644273.345458 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560397.83 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 56039.78 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700313.128774 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504358.05 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 50435.80 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750748.933759 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453922.24 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 45392.22 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796141.158245 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408530.02 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 40853.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836994.160283 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367677.02 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 36767.70 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873761.862117 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330909.32 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 33090.93 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906852.793767 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297818.38 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 29781.84 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936634.632252 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268036.55 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 26803.65 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963438.286889 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241232.89 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 24123.29 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987561.576063 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217109.60 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 21710.96 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009272.536318 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195398.64 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 19539.86 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028812.400549 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175858.78 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 17585.88 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046398.278356 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158272.90 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 15827.29 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062225.568383 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142445.61 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 14244.56 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076470.129406 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128201.05 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 12820.10 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089290.234328 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115380.94 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 11538.09 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100828.328757 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103842.85 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 10384.28 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111212.613744 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93458.56 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 9345.86 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120558.470232 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84112.71 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 8411.27 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128969.741071 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75701.44 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 7570.14 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136539.884826 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68131.29 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 6813.13 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143353.014205 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61318.16 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 6131.82 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149484.830647 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55186.35 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 5518.63 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155003.465444 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49667.71 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 4966.77 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159970.236762 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44700.94 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 4470.09 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164440.330948 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40230.85 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 4023.08 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168463.415715 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36207.76 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3620.78 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172084.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32586.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175684.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28986.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179284.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25386.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182884.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21786.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186484.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18186.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190084.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14586.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193684.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10986.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197284.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7386.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200884.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3786.99 sec) 21:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204484.192006 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 186.99 sec) 21:09:00 [GPUGRID] [rr_sim] 2204671.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969567837.10G/439.78G) 21:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853852.09 21:09:00 [---] pick_jobs_to_run() start 21:09:00 [---] pick_jobs_to_run() end 21:09:00 [---] [rr_sim] end 21:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:09:00 [---] [cpu_sched_debug] preliminary job list: 21:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:09:00 [---] [cpu_sched_debug] final job list: 21:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:09:00 [---] [cpu_sched_debug] enforce_run_list: end 21:09:00 [---] choose_project(): 76140.000000 21:09:00 [WUProp@Home] doing RPC to get work 21:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359459.09 cpu 2108492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108492.90 21:09:00 [---] Got 0 tasks 21:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:10:00 [---] [rr_sim] doing sim: CPU sched 21:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969540246.74G/439.78G) 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204608.44 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 220460.84 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220460.844174 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984147.60 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 198414.76 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418875.603930 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785732.84 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 178573.28 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597448.887711 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607159.55 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 160715.96 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758164.843114 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446443.60 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 144644.36 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902809.202977 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301799.24 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 130179.92 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032989.126853 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171619.31 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 117161.93 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150151.058342 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054457.38 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 105445.74 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255596.796681 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 949011.65 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 94901.16 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350497.961187 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854110.48 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 85411.05 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435909.009242 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768699.43 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 76869.94 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512778.952492 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691829.49 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 69182.95 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581961.901417 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622646.54 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 62264.65 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644226.555449 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560381.89 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 56038.19 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700264.744078 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504343.70 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 50434.37 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750699.113844 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453909.33 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 45390.93 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796090.046634 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408518.40 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 40851.84 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836941.886144 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367666.56 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 36766.66 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873708.541704 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330899.90 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 33089.99 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906798.531707 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297809.91 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 29780.99 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936579.522711 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268028.92 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 26802.89 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963382.414613 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241226.03 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 24122.60 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987505.017326 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217103.42 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 21710.34 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009215.359767 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195393.08 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 19539.31 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028754.667965 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175853.77 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 17585.38 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046340.045342 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158268.40 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 15826.84 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062166.884982 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142441.56 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 14244.16 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076411.040658 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128197.40 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 12819.74 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089230.780766 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115377.66 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 11537.77 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100768.546863 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103839.89 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 10383.99 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111152.536351 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93455.91 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 9345.59 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120498.126890 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84110.31 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 8411.03 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128909.158375 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75699.28 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 7569.93 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136479.086711 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68129.36 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 6812.94 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143292.022214 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61316.42 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 6131.64 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149423.664166 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55184.78 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 5518.48 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154942.141924 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49666.30 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 4966.63 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159908.771905 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44699.67 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 4469.97 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164378.738889 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40229.70 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 4022.97 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168401.709174 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36206.73 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3620.67 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172022.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32586.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175622.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28986.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179222.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25386.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182822.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21786.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186422.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18186.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190022.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14586.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193622.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10986.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197222.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7386.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200822.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3786.06 sec) 21:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204422.382430 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 186.06 sec) 21:10:00 [GPUGRID] [rr_sim] 2204608.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969540246.74G/439.78G) 21:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853849.35 21:10:00 [---] pick_jobs_to_run() start 21:10:00 [---] pick_jobs_to_run() end 21:10:00 [---] [rr_sim] end 21:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:10:00 [---] [cpu_sched_debug] preliminary job list: 21:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:10:00 [---] [cpu_sched_debug] final job list: 21:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:10:00 [---] [cpu_sched_debug] enforce_run_list: end 21:10:00 [---] choose_project(): 76200.000000 21:10:00 [WUProp@Home] doing RPC to get work 21:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359399.09 cpu 2108432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108432.90 21:10:00 [---] Got 0 tasks 21:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:11:00 [---] [rr_sim] doing sim: CPU sched 21:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969512656.39G/439.78G) 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204545.70 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 220454.57 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220454.570486 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984091.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 198409.11 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418863.683923 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785682.02 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 178568.20 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597431.886016 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607113.82 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 160711.38 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758143.267900 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446402.44 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 144640.24 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902783.511596 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301762.19 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 130176.22 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032959.730922 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171585.97 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 117158.60 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150118.328316 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054427.38 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 105442.74 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255561.065970 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948984.64 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 94898.46 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350459.529859 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854086.17 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 85408.62 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435868.147358 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768677.56 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 76867.76 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512735.903108 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691809.80 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 69180.98 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581916.883283 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622628.82 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 62262.88 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644179.765441 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560365.94 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 56036.59 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700216.359382 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504329.35 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 50432.93 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750649.293930 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453896.41 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 45389.64 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1796038.935022 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408506.77 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 40850.68 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836889.612006 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367656.09 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 36765.61 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873655.221291 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330890.48 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 33089.05 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906744.269648 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297801.44 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 29780.14 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936524.413169 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268021.29 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 26802.13 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963326.542337 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241219.16 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 24121.92 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987448.458589 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217097.25 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 21709.72 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009158.183216 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195387.52 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 19538.75 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028696.935380 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175848.77 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 17584.88 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046281.812328 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158263.89 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 15826.39 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062108.201581 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142437.50 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 14243.75 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076351.951909 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128193.75 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 12819.38 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089171.327203 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115374.38 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 11537.44 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100708.764969 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103836.94 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 10383.69 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111092.458958 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93453.25 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 9345.32 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120437.783548 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84107.92 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 8410.79 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128848.575679 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75697.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 7569.71 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136418.288596 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68127.42 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 6812.74 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143231.030222 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61314.67 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 6131.47 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149362.497686 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55183.21 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 5518.32 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154880.818403 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49664.89 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 4966.49 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159847.307048 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44698.40 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 4469.84 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164317.146829 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40228.56 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 4022.86 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168340.002632 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36205.70 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3620.57 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171960.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32585.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175560.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28985.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179160.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25385.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182760.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21785.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186360.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18185.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189960.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14585.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193560.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10985.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197160.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7385.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200760.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3785.13 sec) 21:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204360.572855 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 185.13 sec) 21:11:00 [GPUGRID] [rr_sim] 2204545.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969512656.39G/439.78G) 21:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853846.62 21:11:00 [---] pick_jobs_to_run() start 21:11:00 [---] pick_jobs_to_run() end 21:11:00 [---] [rr_sim] end 21:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:11:00 [---] [cpu_sched_debug] preliminary job list: 21:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:11:00 [---] [cpu_sched_debug] final job list: 21:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:11:00 [---] [cpu_sched_debug] enforce_run_list: end 21:11:00 [---] choose_project(): 76260.000000 21:11:00 [WUProp@Home] doing RPC to get work 21:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359339.09 cpu 2108372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108372.90 21:11:00 [---] Got 0 tasks 21:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:12:00 [---] [rr_sim] doing sim: CPU sched 21:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969485066.03G/439.78G) 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204482.97 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 220448.30 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220448.296798 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1984034.67 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 198403.47 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418851.763915 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785631.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 178563.12 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597414.884321 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607068.08 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 160706.81 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758121.692687 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446361.28 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 144636.13 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902757.820215 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301725.15 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 130172.51 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032930.334991 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171552.63 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 117155.26 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150085.598290 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054397.37 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 105439.74 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255525.335258 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948957.63 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 94895.76 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350421.098530 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854061.87 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 85406.19 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435827.285474 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768655.68 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 76865.57 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512692.853725 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691790.11 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 69179.01 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581871.865150 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622611.10 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 62261.11 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644132.975432 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560349.99 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 56035.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700167.974686 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504314.99 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 50431.50 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750599.474015 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453883.49 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 45388.35 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795987.823411 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408495.14 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 40849.51 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836837.337868 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367645.63 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 36764.56 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873601.900878 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330881.07 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 33088.11 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906690.007588 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297792.96 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 29779.30 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936469.303627 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268013.66 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 26801.37 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963270.670062 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241212.30 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 24121.23 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987391.899853 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217091.07 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 21709.11 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009101.006665 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195381.96 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 19538.20 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028639.202796 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175843.77 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 17584.38 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046223.579314 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158259.39 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 15825.94 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2062049.518180 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142433.45 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 14243.34 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076292.863160 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128190.10 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 12819.01 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089111.873641 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115371.09 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 11537.11 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100648.983075 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103833.98 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 10383.40 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111032.381565 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93450.59 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 9345.06 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120377.440206 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84105.53 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 8410.55 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128787.992983 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75694.97 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 7569.50 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136357.490482 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68125.48 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 6812.55 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143170.038231 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61312.93 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 6131.29 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149301.331206 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55181.64 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 5518.16 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154819.494882 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49663.47 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 4966.35 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159785.842192 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44697.13 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 4469.71 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164255.554770 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40227.41 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 4022.74 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168278.296091 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36204.67 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3620.47 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171898.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32584.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175498.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28984.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179098.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25384.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182698.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21784.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186298.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18184.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189898.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14584.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193498.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10984.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197098.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7384.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200698.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3784.20 sec) 21:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204298.763279 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 184.20 sec) 21:12:00 [GPUGRID] [rr_sim] 2204482.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969485066.03G/439.78G) 21:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853843.88 21:12:00 [---] pick_jobs_to_run() start 21:12:00 [---] pick_jobs_to_run() end 21:12:00 [---] [rr_sim] end 21:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:12:00 [---] [cpu_sched_debug] preliminary job list: 21:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:12:00 [---] [cpu_sched_debug] final job list: 21:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:12:00 [---] [cpu_sched_debug] enforce_run_list: end 21:12:00 [---] choose_project(): 76320.000000 21:12:00 [WUProp@Home] doing RPC to get work 21:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359279.09 cpu 2108312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108312.90 21:12:00 [---] Got 0 tasks 21:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:13:00 [---] [rr_sim] doing sim: CPU sched 21:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969457475.67G/439.78G) 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204420.23 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 220442.02 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220442.023109 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983978.21 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 198397.82 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418839.843908 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785580.39 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 178558.04 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597397.882626 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1607022.35 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 160702.23 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758100.117473 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446320.11 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 144632.01 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902732.128835 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301688.10 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 130168.81 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032900.939061 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171519.29 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 117151.93 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150052.868264 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054367.36 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 105436.74 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255489.604547 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948930.63 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 94893.06 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350382.667201 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854037.56 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 85403.76 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435786.423591 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768633.81 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 76863.38 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512649.804341 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691770.43 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 69177.04 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581826.847016 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622593.38 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 62259.34 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644086.185424 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560334.05 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 56033.40 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700119.589991 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504300.64 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 50430.06 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750549.654101 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453870.58 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 45387.06 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795936.711800 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408483.52 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 40848.35 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836785.063729 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367635.17 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 36763.52 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873548.580466 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330871.65 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 33087.17 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906635.745528 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297784.49 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 29778.45 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936414.194085 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 268006.04 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 26800.60 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963214.797786 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241205.43 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 24120.54 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987335.341116 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217084.89 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 21708.49 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2009043.830114 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195376.40 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 19537.64 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028581.470212 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175838.76 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 17583.88 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046165.346300 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158254.88 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 15825.49 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061990.834779 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142429.40 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 14242.94 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076233.774411 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128186.46 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 12818.65 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089052.420079 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115367.81 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 11536.78 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100589.201180 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103831.03 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 10383.10 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110972.304172 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93447.93 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 9344.79 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120317.096864 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84103.13 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 8410.31 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128727.410287 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75692.82 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 7569.28 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136296.692367 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68123.54 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 6812.35 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143109.046240 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61311.18 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 6131.12 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149240.164725 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55180.07 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 5518.01 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154758.171362 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49662.06 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 4966.21 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159724.377335 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44695.85 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 4469.59 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164193.962711 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40226.27 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 4022.63 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168216.589549 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36203.64 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3620.36 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171836.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32583.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175436.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28983.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179036.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25383.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182636.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21783.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186236.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18183.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189836.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14583.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193436.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10983.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197036.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7383.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200636.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3783.28 sec) 21:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204236.953703 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 183.28 sec) 21:13:00 [GPUGRID] [rr_sim] 2204420.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969457475.67G/439.78G) 21:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853841.14 21:13:00 [---] pick_jobs_to_run() start 21:13:00 [---] pick_jobs_to_run() end 21:13:00 [---] [rr_sim] end 21:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:13:00 [---] [cpu_sched_debug] preliminary job list: 21:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:13:00 [---] [cpu_sched_debug] final job list: 21:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:13:00 [---] [cpu_sched_debug] enforce_run_list: end 21:13:00 [---] choose_project(): 76380.000000 21:13:00 [WUProp@Home] doing RPC to get work 21:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359219.09 cpu 2108252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108252.90 21:13:00 [---] Got 0 tasks 21:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:14:00 [---] [rr_sim] doing sim: CPU sched 21:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969429885.32G/439.78G) 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204357.49 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 220435.75 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220435.749421 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983921.74 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 198392.17 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418827.923900 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785529.57 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 178552.96 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597380.880931 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606976.61 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 160697.66 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758078.542259 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446278.95 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 144627.90 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902706.437454 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301651.06 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 130165.11 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032871.543130 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171485.95 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 117148.60 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1150020.138238 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054337.36 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 105433.74 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255453.873835 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948903.62 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 94890.36 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350344.235873 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 854013.26 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 85401.33 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435745.561707 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768611.93 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 76861.19 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512606.754957 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691750.74 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 69175.07 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581781.828882 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622575.67 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 62257.57 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1644039.395415 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560318.10 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 56031.81 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700071.205295 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504286.29 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 50428.63 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750499.834186 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453857.66 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 45385.77 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795885.600189 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408471.89 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 40847.19 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836732.789591 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367624.70 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 36762.47 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873495.260053 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330862.23 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 33086.22 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906581.483469 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297776.01 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 29777.60 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936359.084543 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267998.41 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 26799.84 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963158.925510 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241198.57 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 24119.86 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987278.782380 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217078.71 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 21707.87 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008986.653563 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195370.84 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 19537.08 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028523.737628 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175833.76 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 17583.38 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046107.113286 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158250.38 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 15825.04 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061932.151378 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142425.34 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 14242.53 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076174.685662 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128182.81 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 12818.28 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088992.966517 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115364.53 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 11536.45 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100529.419286 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103828.07 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 10382.81 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110912.226778 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93445.27 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 9344.53 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120256.753522 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84100.74 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 8410.07 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128666.827591 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75690.67 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 7569.07 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136235.894253 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68121.60 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 6812.16 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143048.054248 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61309.44 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 6130.94 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149178.998245 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55178.50 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 5517.85 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154696.847841 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49660.65 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 4966.06 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159662.912478 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44694.58 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 4469.46 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164132.370651 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40225.12 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 4022.51 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168154.883007 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36202.61 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3620.26 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171775.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32582.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175375.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28982.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178975.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25382.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182575.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21782.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186175.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18182.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189775.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14582.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193375.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10982.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196975.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7382.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200575.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3782.35 sec) 21:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204175.144128 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 182.35 sec) 21:14:00 [GPUGRID] [rr_sim] 2204357.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969429885.32G/439.78G) 21:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853838.41 21:14:00 [---] pick_jobs_to_run() start 21:14:00 [---] pick_jobs_to_run() end 21:14:00 [---] [rr_sim] end 21:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:14:00 [---] [cpu_sched_debug] preliminary job list: 21:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:14:00 [---] [cpu_sched_debug] final job list: 21:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:14:00 [---] [cpu_sched_debug] enforce_run_list: end 21:14:00 [---] choose_project(): 76440.000000 21:14:00 [WUProp@Home] doing RPC to get work 21:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359159.09 cpu 2108192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108192.90 21:14:00 [---] Got 0 tasks 21:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:15:00 [---] [rr_sim] doing sim: CPU sched 21:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969402294.96G/439.78G) 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204294.76 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 220429.48 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220429.475733 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983865.28 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 198386.53 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418816.003892 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785478.75 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 178547.88 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597363.879236 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606930.88 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 160693.09 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758056.967045 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446237.79 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 144623.78 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902680.746074 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301614.01 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 130161.40 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032842.147199 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171452.61 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 117145.26 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149987.408212 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054307.35 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 105430.73 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255418.143124 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948876.61 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 94887.66 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350305.804544 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853988.95 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 85398.90 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435704.699823 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768590.06 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 76859.01 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512563.705573 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691731.05 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 69173.11 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581736.810749 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622557.95 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 62255.79 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643992.605407 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560302.15 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 56030.22 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1700022.820599 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504271.94 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 50427.19 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750450.014272 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453844.74 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 45384.47 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795834.488578 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408460.27 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 40846.03 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836680.515453 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367614.24 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 36761.42 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873441.939640 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330852.82 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 33085.28 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906527.221409 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297767.54 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 29776.75 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936303.975001 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267990.78 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 26799.08 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963103.053234 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241191.70 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 24119.17 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987222.223643 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217072.53 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 21707.25 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008929.477012 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195365.28 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 19536.53 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028466.005043 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175828.75 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 17582.88 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2046048.880272 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158245.88 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 15824.59 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061873.467978 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142421.29 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 14242.13 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076115.596913 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128179.16 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 12817.92 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088933.512954 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115361.24 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 11536.12 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100469.637392 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103825.12 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 10382.51 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110852.149385 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93442.61 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 9344.26 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120196.410180 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84098.35 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 8409.83 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128606.244895 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75688.51 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 7568.85 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136175.096138 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68119.66 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 6811.97 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142987.062257 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61307.70 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 6130.77 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149117.831764 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55176.93 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 5517.69 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154635.524321 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49659.23 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 4965.92 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159601.447621 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44693.31 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 4469.33 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164070.778592 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40223.98 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 4022.40 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168093.176466 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36201.58 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3620.16 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171713.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32581.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175313.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28981.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178913.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25381.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182513.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21781.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186113.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18181.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189713.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14581.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193313.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10981.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196913.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7381.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200513.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3781.42 sec) 21:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204113.334552 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 181.42 sec) 21:15:00 [GPUGRID] [rr_sim] 2204294.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969402294.96G/439.78G) 21:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853835.67 21:15:00 [---] pick_jobs_to_run() start 21:15:00 [---] pick_jobs_to_run() end 21:15:00 [---] [rr_sim] end 21:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:15:00 [---] [cpu_sched_debug] preliminary job list: 21:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:15:00 [---] [cpu_sched_debug] final job list: 21:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:15:00 [---] [cpu_sched_debug] enforce_run_list: end 21:15:00 [---] choose_project(): 76500.000000 21:15:00 [WUProp@Home] doing RPC to get work 21:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359099.09 cpu 2108132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108132.90 21:15:00 [---] Got 0 tasks 21:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:16:00 [---] [rr_sim] doing sim: CPU sched 21:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969374704.61G/439.78G) 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204232.02 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 220423.20 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220423.202045 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983808.82 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 198380.88 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418804.083885 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785427.94 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 178542.79 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597346.877541 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606885.14 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 160688.51 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758035.391832 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446196.63 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 144619.66 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902655.054693 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301576.97 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 130157.70 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032812.751268 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171419.27 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 117141.93 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149954.678186 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054277.34 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 105427.73 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255382.412412 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948849.61 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 94884.96 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350267.373216 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853964.65 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 85396.46 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435663.837939 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768568.18 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 76856.82 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512520.656189 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691711.36 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 69171.14 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581691.792615 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622540.23 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 62254.02 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643945.815398 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560286.21 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 56028.62 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699974.435903 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504257.58 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 50425.76 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750400.194357 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453831.83 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 45383.18 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795783.376966 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408448.64 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 40844.86 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836628.241314 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367603.78 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 36760.38 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873388.619227 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330843.40 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 33084.34 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906472.959349 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297759.06 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 29775.91 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936248.865459 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267983.15 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 26798.32 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1963047.180958 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241184.84 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 24118.48 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987165.664907 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217066.36 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 21706.64 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008872.300461 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195359.72 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 19535.97 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028408.272459 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175823.75 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 17582.37 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045990.647258 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158241.37 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 15824.14 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061814.784577 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142417.24 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 14241.72 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076056.508164 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128175.51 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 12817.55 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088874.059392 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115357.96 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 11535.80 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100409.855497 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103822.16 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 10382.22 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110792.071992 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93439.95 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 9343.99 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120136.066838 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84095.95 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 8409.60 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128545.662199 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75686.36 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 7568.64 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136114.298023 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68117.72 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 6811.77 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142926.070266 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61305.95 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 6130.60 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149056.665284 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55175.36 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 5517.54 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154574.200800 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49657.82 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 4965.78 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159539.982765 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44692.04 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 4469.20 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164009.186533 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40222.83 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 4022.28 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168031.469924 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36200.55 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3620.06 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171651.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32580.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175251.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28980.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178851.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25380.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182451.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21780.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186051.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18180.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189651.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14580.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193251.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10980.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196851.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7380.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200451.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3780.50 sec) 21:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2204051.524976 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 180.50 sec) 21:16:00 [GPUGRID] [rr_sim] 2204232.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969374704.61G/439.78G) 21:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853832.93 21:16:00 [---] pick_jobs_to_run() start 21:16:00 [---] pick_jobs_to_run() end 21:16:00 [---] [rr_sim] end 21:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:16:00 [---] [cpu_sched_debug] preliminary job list: 21:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:16:00 [---] [cpu_sched_debug] final job list: 21:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:16:00 [---] [cpu_sched_debug] enforce_run_list: end 21:16:00 [---] choose_project(): 76560.000000 21:16:00 [WUProp@Home] doing RPC to get work 21:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 359039.09 cpu 2108072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108072.90 21:16:00 [---] Got 0 tasks 21:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:17:00 [---] [rr_sim] doing sim: CPU sched 21:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969347114.25G/439.78G) 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204169.28 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 220416.93 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220416.928356 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983752.36 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 198375.24 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418792.163877 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785377.12 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 178537.71 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597329.875846 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606839.41 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 160683.94 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 758013.816618 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446155.47 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 144615.55 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902629.363312 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301539.92 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 130153.99 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032783.355338 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171385.93 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 117138.59 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149921.948160 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054247.34 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 105424.73 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255346.681701 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948822.60 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 94882.26 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350228.941887 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853940.34 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 85394.03 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435622.976055 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768546.31 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 76854.63 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512477.606806 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691691.68 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 69169.17 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581646.774482 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622522.51 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 62252.25 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643899.025390 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560270.26 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 56027.03 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699926.051207 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504243.23 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 50424.32 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750350.374443 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453818.91 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 45381.89 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795732.265355 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408437.02 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 40843.70 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836575.967176 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367593.32 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 36759.33 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873335.298815 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330833.98 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 33083.40 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906418.697290 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297750.59 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 29775.06 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936193.755917 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267975.53 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 26797.55 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962991.308682 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241177.97 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 24117.80 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987109.106170 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217060.18 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 21706.02 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008815.123910 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195354.16 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 19535.42 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028350.539875 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175818.74 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 17581.87 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045932.414244 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158236.87 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 15823.69 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061756.101176 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142413.18 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 14241.32 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075997.419415 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128171.86 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 12817.19 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088814.605830 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115354.68 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 11535.47 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100350.073603 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103819.21 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 10381.92 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110731.994599 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93437.29 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 9343.73 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120075.723496 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84093.56 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 8409.36 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128485.079503 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75684.20 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 7568.42 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136053.499909 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68115.78 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 6811.58 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142865.078274 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61304.21 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 6130.42 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148995.498803 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55173.78 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 5517.38 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154512.877279 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49656.41 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 4965.64 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159478.517908 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44690.77 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 4469.08 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163947.594473 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40221.69 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 4022.17 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167969.763383 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36199.52 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3619.95 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171589.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32579.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175189.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28979.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178789.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25379.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182389.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21779.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185989.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18179.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189589.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14579.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193189.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10979.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196789.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7379.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200389.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3779.57 sec) 21:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203989.715401 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 179.57 sec) 21:17:00 [GPUGRID] [rr_sim] 2204169.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969347114.25G/439.78G) 21:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853830.20 21:17:00 [---] pick_jobs_to_run() start 21:17:00 [---] pick_jobs_to_run() end 21:17:00 [---] [rr_sim] end 21:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:17:00 [---] [cpu_sched_debug] preliminary job list: 21:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:17:00 [---] [cpu_sched_debug] final job list: 21:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:17:00 [---] [cpu_sched_debug] enforce_run_list: end 21:17:00 [---] choose_project(): 76620.000000 21:17:00 [WUProp@Home] doing RPC to get work 21:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358979.09 cpu 2108012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2108012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2108012.90 21:17:00 [---] Got 0 tasks 21:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:18:00 [---] [rr_sim] doing sim: CPU sched 21:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969319523.89G/439.78G) 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204106.55 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 220410.65 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220410.654668 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983695.89 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 198369.59 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418780.243870 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785326.30 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 178532.63 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597312.874151 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606793.67 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 160679.37 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757992.241404 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446114.31 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 144611.43 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902603.671932 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301502.87 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 130150.29 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032753.959407 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171352.59 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 117135.26 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149889.218134 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054217.33 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 105421.73 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255310.950989 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948795.60 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 94879.56 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350190.510558 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853916.04 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 85391.60 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435582.114171 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768524.43 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 76852.44 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512434.557422 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691671.99 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 69167.20 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581601.756348 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622504.79 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 62250.48 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643852.235381 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560254.31 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 56025.43 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699877.666511 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504228.88 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 50422.89 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750300.554528 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453805.99 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 45380.60 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795681.153744 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408425.39 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 40842.54 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836523.693038 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367582.85 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 36758.29 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873281.978402 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330824.57 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 33082.46 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906364.435230 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297742.11 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 29774.21 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936138.646375 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267967.90 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 26796.79 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962935.436406 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241171.11 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 24117.11 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1987052.547434 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217054.00 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 21705.40 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008757.947358 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195348.60 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 19534.86 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028292.807291 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175813.74 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 17581.37 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045874.181230 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158232.37 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 15823.24 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061697.417775 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142409.13 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 14240.91 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075938.330666 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128168.22 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 12816.82 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088755.152267 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115351.39 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 11535.14 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100290.291709 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103816.25 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 10381.63 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110671.917206 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93434.63 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 9343.46 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120015.380154 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84091.17 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 8409.12 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128424.496807 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75682.05 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 7568.20 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135992.701794 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68113.84 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 6811.38 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142804.086283 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61302.46 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 6130.25 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148934.332323 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55172.21 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 5517.22 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154451.553759 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49654.99 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 4965.50 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159417.053051 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44689.49 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 4468.95 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163886.002414 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40220.54 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 4022.05 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167908.056841 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36198.49 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3619.85 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171527.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32578.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175127.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28978.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178727.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25378.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182327.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21778.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185927.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18178.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189527.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14578.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193127.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10978.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196727.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7378.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200327.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3778.64 sec) 21:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203927.905825 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 178.64 sec) 21:18:00 [GPUGRID] [rr_sim] 2204106.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969319523.89G/439.78G) 21:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853827.46 21:18:00 [---] pick_jobs_to_run() start 21:18:00 [---] pick_jobs_to_run() end 21:18:00 [---] [rr_sim] end 21:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:18:00 [---] [cpu_sched_debug] preliminary job list: 21:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:18:00 [---] [cpu_sched_debug] final job list: 21:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:18:00 [---] [cpu_sched_debug] enforce_run_list: end 21:18:00 [---] choose_project(): 76680.000000 21:18:00 [WUProp@Home] doing RPC to get work 21:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358919.09 cpu 2107952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107952.90 21:18:00 [---] Got 0 tasks 21:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:19:00 [---] [rr_sim] doing sim: CPU sched 21:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969291933.54G/439.78G) 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2204043.81 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 220404.38 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220404.380980 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983639.43 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 198363.94 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418768.323862 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785275.49 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 178527.55 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597295.872456 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606747.94 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 160674.79 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757970.666190 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446073.14 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 144607.31 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902577.980551 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301465.83 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 130146.58 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032724.563476 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171319.25 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 117131.92 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149856.488108 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054187.32 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 105418.73 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255275.220278 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948768.59 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 94876.86 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350152.079230 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853891.73 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 85389.17 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435541.252287 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768502.56 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 76850.26 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512391.508038 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691652.30 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 69165.23 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581556.738214 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622487.07 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 62248.71 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643805.445373 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560238.36 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 56023.84 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699829.281816 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504214.53 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 50421.45 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750250.734614 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453793.08 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 45379.31 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795630.042133 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408413.77 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 40841.38 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836471.418899 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367572.39 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 36757.24 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873228.657989 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330815.15 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 33081.52 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906310.173170 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297733.64 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 29773.36 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936083.536833 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267960.27 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 26796.03 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962879.564130 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241164.25 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 24116.42 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986995.988697 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217047.82 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 21704.78 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008700.770807 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195343.04 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 19534.30 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028235.074707 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175808.74 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 17580.87 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045815.948216 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158227.86 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 15822.79 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061638.734374 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142405.08 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 14240.51 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075879.241917 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128164.57 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 12816.46 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088695.698705 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115348.11 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 11534.81 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100230.509815 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103813.30 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 10381.33 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110611.839813 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93431.97 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 9343.20 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119955.036812 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84088.77 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 8408.88 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128363.914111 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75679.90 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 7567.99 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135931.903680 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68111.91 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 6811.19 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142743.094292 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61300.72 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 6130.07 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148873.165842 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55170.64 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 5517.06 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154390.230238 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49653.58 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 4965.36 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159355.588194 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44688.22 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 4468.82 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163824.410355 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40219.40 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 4021.94 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167846.350299 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36197.46 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3619.75 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171466.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32577.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175066.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28977.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178666.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25377.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182266.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21777.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185866.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18177.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189466.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14577.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193066.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10977.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196666.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7377.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200266.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3777.71 sec) 21:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203866.096249 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 177.71 sec) 21:19:00 [GPUGRID] [rr_sim] 2204043.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969291933.54G/439.78G) 21:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853824.72 21:19:00 [---] pick_jobs_to_run() start 21:19:00 [---] pick_jobs_to_run() end 21:19:00 [---] [rr_sim] end 21:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:19:00 [---] [cpu_sched_debug] preliminary job list: 21:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:19:00 [---] [cpu_sched_debug] final job list: 21:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:19:00 [---] [cpu_sched_debug] enforce_run_list: end 21:19:00 [---] choose_project(): 76740.000000 21:19:00 [WUProp@Home] doing RPC to get work 21:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358859.09 cpu 2107892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107892.90 21:19:00 [---] Got 0 tasks 21:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:20:00 [---] [rr_sim] doing sim: CPU sched 21:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969264343.18G/439.78G) 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203981.07 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 220398.11 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220398.107292 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983582.97 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 198358.30 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418756.403854 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785224.67 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 178522.47 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597278.870761 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606702.20 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 160670.22 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757949.090976 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1446031.98 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 144603.20 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902552.289171 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301428.78 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 130142.88 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032695.167545 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171285.91 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 117128.59 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149823.758082 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054157.31 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 105415.73 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255239.489566 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948741.58 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 94874.16 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350113.647901 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853867.43 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 85386.74 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435500.390403 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768480.68 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 76848.07 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512348.458654 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691632.61 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 69163.26 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581511.720081 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622469.35 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 62246.94 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643758.655364 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560222.42 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 56022.24 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699780.897120 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504200.18 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 50420.02 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750200.914699 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453780.16 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 45378.02 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795578.930521 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408402.14 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 40840.21 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836419.144761 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367561.93 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 36756.19 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873175.337577 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330805.74 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 33080.57 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906255.911111 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297725.16 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 29772.52 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1936028.427291 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267952.65 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 26795.26 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962823.691854 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241157.38 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 24115.74 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986939.429960 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217041.64 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 21704.16 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008643.594256 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195337.48 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 19533.75 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028177.342122 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175803.73 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 17580.37 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045757.715202 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158223.36 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 15822.34 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061580.050973 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142401.02 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 14240.10 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075820.153168 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128160.92 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 12816.09 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088636.245143 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115344.83 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 11534.48 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100170.727920 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103810.34 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 10381.03 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110551.762420 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93429.31 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 9342.93 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119894.693470 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84086.38 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 8408.64 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128303.331415 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75677.74 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 7567.77 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135871.105565 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68109.97 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 6811.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142682.102300 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61298.97 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 6129.90 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148811.999362 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55169.07 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 5516.91 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154328.906718 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49652.17 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 4965.22 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159294.123338 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44686.95 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 4468.69 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163762.818296 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40218.25 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 4021.83 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167784.643758 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36196.43 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3619.64 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171404.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32576.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175004.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28976.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178604.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25376.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182204.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21776.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185804.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18176.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189404.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14576.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193004.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10976.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196604.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7376.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200204.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3776.79 sec) 21:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203804.286674 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 176.79 sec) 21:20:00 [GPUGRID] [rr_sim] 2203981.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969264343.18G/439.78G) 21:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853821.99 21:20:00 [---] pick_jobs_to_run() start 21:20:00 [---] pick_jobs_to_run() end 21:20:00 [---] [rr_sim] end 21:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:20:00 [---] [cpu_sched_debug] preliminary job list: 21:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:20:00 [---] [cpu_sched_debug] final job list: 21:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:20:00 [---] [cpu_sched_debug] enforce_run_list: end 21:20:00 [---] choose_project(): 76800.000000 21:20:00 [WUProp@Home] doing RPC to get work 21:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358799.09 cpu 2107832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107832.90 21:20:00 [---] Got 0 tasks 21:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:21:00 [---] [rr_sim] doing sim: CPU sched 21:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969236752.82G/439.78G) 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203918.34 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 220391.83 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220391.833604 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983526.50 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 198352.65 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418744.483847 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785173.85 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 178517.39 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597261.869066 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606656.47 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 160665.65 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757927.515763 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445990.82 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 144599.08 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902526.597790 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301391.74 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 130139.17 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032665.771614 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171252.56 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 117125.26 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149791.028057 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054127.31 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 105412.73 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255203.758854 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948714.58 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 94871.46 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350075.216573 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853843.12 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 85384.31 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435459.528519 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768458.81 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 76845.88 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512305.409271 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691612.93 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 69161.29 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581466.701947 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622451.63 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 62245.16 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643711.865356 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560206.47 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 56020.65 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699732.512424 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504185.82 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 50418.58 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750151.094785 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453767.24 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 45376.72 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795527.818910 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408390.52 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 40839.05 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836366.870623 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367551.47 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 36755.15 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873122.017164 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330796.32 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 33079.63 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906201.649051 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297716.69 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 29771.67 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935973.317750 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267945.02 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 26794.50 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962767.819578 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241150.52 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 24115.05 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986882.871224 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217035.46 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 21703.55 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008586.417705 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195331.92 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 19533.19 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028119.609538 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175798.73 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 17579.87 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045699.482188 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158218.85 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 15821.89 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061521.367573 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142396.97 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 14239.70 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075761.064419 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128157.27 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 12815.73 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088576.791581 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115341.54 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 11534.15 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100110.946026 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103807.39 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 10380.74 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110491.685027 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93426.65 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 9342.67 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119834.350128 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84083.99 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 8408.40 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128242.748719 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75675.59 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 7567.56 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135810.307450 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68108.03 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 6810.80 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142621.110309 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61297.23 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 6129.72 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148750.832882 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55167.50 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 5516.75 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154267.583197 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49650.75 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 4965.08 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159232.658481 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44685.68 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 4468.57 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163701.226236 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40217.11 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 4021.71 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167722.937216 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36195.40 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3619.54 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171342.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32575.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174942.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28975.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178542.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25375.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182142.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21775.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185742.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18175.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189342.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14575.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192942.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10975.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196542.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7375.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200142.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3775.86 sec) 21:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203742.477098 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175.86 sec) 21:21:00 [GPUGRID] [rr_sim] 2203918.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969236752.82G/439.78G) 21:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853819.25 21:21:00 [---] pick_jobs_to_run() start 21:21:00 [---] pick_jobs_to_run() end 21:21:00 [---] [rr_sim] end 21:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:21:00 [---] [cpu_sched_debug] preliminary job list: 21:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:21:00 [---] [cpu_sched_debug] final job list: 21:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:21:00 [---] [cpu_sched_debug] enforce_run_list: end 21:21:00 [---] choose_project(): 76860.000000 21:21:00 [WUProp@Home] doing RPC to get work 21:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358739.09 cpu 2107772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107772.90 21:21:00 [---] Got 0 tasks 21:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:22:00 [---] [rr_sim] doing sim: CPU sched 21:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969209162.47G/439.78G) 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203855.60 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 220385.56 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220385.559915 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983470.04 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 198347.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418732.563839 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785123.04 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 178512.30 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597244.867371 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606610.73 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 160661.07 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757905.940549 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445949.66 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 144594.97 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902500.906409 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301354.69 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 130135.47 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032636.375684 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171219.22 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 117121.92 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149758.298031 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054097.30 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 105409.73 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255168.028143 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948687.57 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 94868.76 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1350036.785244 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853818.81 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 85381.88 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435418.666635 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768436.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 76843.69 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512262.359887 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691593.24 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 69159.32 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581421.683813 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622433.92 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 62243.39 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643665.075347 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560190.52 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 56019.05 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699684.127728 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504171.47 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 50417.15 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750101.274871 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453754.32 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 45375.43 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795476.707299 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408378.89 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 40837.89 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836314.596484 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367541.00 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 36754.10 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873068.696751 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330786.90 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 33078.69 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906147.386991 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297708.21 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 29770.82 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935918.208208 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267937.39 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 26793.74 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962711.947302 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241143.65 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 24114.37 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986826.312487 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217029.29 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 21702.93 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008529.241154 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195326.36 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 19532.64 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028061.876954 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175793.72 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 17579.37 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045641.249174 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158214.35 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 15821.43 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061462.684172 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142392.91 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 14239.29 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075701.975670 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128153.62 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 12815.36 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088517.338018 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115338.26 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 11533.83 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100051.164132 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103804.44 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 10380.44 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110431.607634 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93423.99 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 9342.40 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119774.006786 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84081.59 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 8408.16 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128182.166023 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75673.43 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 7567.34 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135749.509336 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68106.09 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 6810.61 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142560.118317 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61295.48 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 6129.55 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148689.666401 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55165.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 5516.59 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154206.259676 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49649.34 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 4964.93 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159171.193624 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44684.41 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 4468.44 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163639.634177 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40215.96 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 4021.60 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167661.230675 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36194.37 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3619.44 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171280.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32574.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174880.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28974.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178480.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25374.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182080.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21774.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185680.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18174.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189280.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14574.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192880.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10974.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196480.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7374.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200080.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3774.93 sec) 21:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203680.667522 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174.93 sec) 21:22:00 [GPUGRID] [rr_sim] 2203855.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969209162.47G/439.78G) 21:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853816.51 21:22:00 [---] pick_jobs_to_run() start 21:22:00 [---] pick_jobs_to_run() end 21:22:00 [---] [rr_sim] end 21:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:22:00 [---] [cpu_sched_debug] preliminary job list: 21:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:22:00 [---] [cpu_sched_debug] final job list: 21:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:22:00 [---] [cpu_sched_debug] enforce_run_list: end 21:22:00 [---] choose_project(): 76920.000000 21:22:00 [WUProp@Home] doing RPC to get work 21:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358679.09 cpu 2107712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107712.90 21:22:00 [---] Got 0 tasks 21:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:23:00 [---] [rr_sim] doing sim: CPU sched 21:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969181572.11G/439.78G) 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203792.86 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 220379.29 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220379.286227 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983413.58 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 198341.36 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418720.643832 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785072.22 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 178507.22 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597227.865675 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606565.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 160656.50 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757884.365335 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445908.50 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 144590.85 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902475.215029 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301317.65 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 130131.76 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032606.979753 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171185.88 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 117118.59 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149725.568005 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054067.29 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 105406.73 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255132.297431 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948660.56 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 94866.06 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349998.353915 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853794.51 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 85379.45 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435377.804751 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768415.06 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 76841.51 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512219.310503 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691573.55 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 69157.36 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581376.665680 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622416.20 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 62241.62 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643618.285339 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560174.58 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 56017.46 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699635.743032 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504157.12 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 50415.71 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750051.454956 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453741.41 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 45374.14 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795425.595688 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408367.27 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 40836.73 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836262.322346 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367530.54 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 36753.05 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1873015.376338 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330777.49 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 33077.75 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906093.124932 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297699.74 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 29769.97 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935863.098666 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267929.76 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 26792.98 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962656.075026 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241136.79 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 24113.68 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986769.753751 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217023.11 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 21702.31 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008472.064603 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195320.80 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 19532.08 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2028004.144370 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175788.72 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 17578.87 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045583.016160 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158209.85 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 15820.98 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061404.000771 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142388.86 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 14238.89 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075642.886921 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128149.98 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 12815.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088457.884456 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115334.98 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 11533.50 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099991.382238 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103801.48 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 10380.15 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110371.530241 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93421.33 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 9342.13 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119713.663444 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84079.20 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 8407.92 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128121.583327 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75671.28 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 7567.13 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135688.711221 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68104.15 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 6810.42 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142499.126326 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61293.74 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 6129.37 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148628.499921 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55164.36 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 5516.44 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154144.936156 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49647.93 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 4964.79 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159109.728767 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44683.13 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 4468.31 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163578.042118 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40214.82 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 4021.48 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167599.524133 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36193.34 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3619.33 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171218.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32574.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174818.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28974.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178418.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25374.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182018.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21774.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185618.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18174.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189218.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14574.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192818.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10974.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196418.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7374.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200018.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3774.00 sec) 21:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203618.857947 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174.00 sec) 21:23:00 [GPUGRID] [rr_sim] 2203792.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969181572.11G/439.78G) 21:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853813.77 21:23:00 [---] pick_jobs_to_run() start 21:23:00 [---] pick_jobs_to_run() end 21:23:00 [---] [rr_sim] end 21:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:23:00 [---] [cpu_sched_debug] preliminary job list: 21:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:23:00 [---] [cpu_sched_debug] final job list: 21:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:23:00 [---] [cpu_sched_debug] enforce_run_list: end 21:23:00 [---] choose_project(): 76980.000000 21:23:00 [WUProp@Home] doing RPC to get work 21:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358619.09 cpu 2107652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107652.90 21:23:00 [---] Got 0 tasks 21:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:24:00 [---] [rr_sim] doing sim: CPU sched 21:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969153981.76G/439.78G) 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203730.13 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 220373.01 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220373.012539 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983357.11 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 198335.71 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418708.723824 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1785021.40 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 178502.14 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597210.863980 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606519.26 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 160651.93 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757862.790121 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445867.34 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 144586.73 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902449.523648 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301280.60 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 130128.06 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032577.583822 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171152.54 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 117115.25 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149692.837979 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054037.29 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 105403.73 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255096.566720 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948633.56 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 94863.36 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349959.922587 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853770.20 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 85377.02 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435336.942867 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768393.18 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 76839.32 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512176.261119 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691553.86 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 69155.39 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581331.647546 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622398.48 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 62239.85 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643571.495330 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560158.63 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 56015.86 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699587.358336 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504142.77 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 50414.28 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1750001.635042 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453728.49 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 45372.85 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795374.484076 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408355.64 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 40835.56 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836210.048208 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367520.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 36752.01 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872962.055926 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330768.07 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 33076.81 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1906038.862872 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297691.26 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 29769.13 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935807.989124 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267922.14 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 26792.21 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962600.202750 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241129.92 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 24112.99 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986713.195014 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217016.93 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 21701.69 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008414.888052 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195315.24 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 19531.52 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027946.411785 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175783.71 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 17578.37 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045524.783146 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158205.34 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 15820.53 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061345.317370 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142384.81 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 14238.48 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075583.798172 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128146.33 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 12814.63 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088398.430894 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115331.69 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 11533.17 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099931.600343 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103798.53 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 10379.85 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110311.452848 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93418.67 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 9341.87 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119653.320102 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84076.81 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 8407.68 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128061.000631 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75669.12 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 7566.91 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135627.913106 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68102.21 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 6810.22 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142438.134335 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61291.99 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 6129.20 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148567.333440 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55162.79 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 5516.28 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154083.612635 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49646.51 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 4964.65 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159048.263910 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44681.86 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 4468.19 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163516.450058 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40213.68 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 4021.37 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167537.817591 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36192.31 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3619.23 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171157.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32573.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174757.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28973.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178357.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25373.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181957.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21773.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185557.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18173.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189157.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14573.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192757.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10973.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196357.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7373.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199957.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3773.08 sec) 21:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203557.048371 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 173.08 sec) 21:24:00 [GPUGRID] [rr_sim] 2203730.13: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969153981.76G/439.78G) 21:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853811.04 21:24:00 [---] pick_jobs_to_run() start 21:24:00 [---] pick_jobs_to_run() end 21:24:00 [---] [rr_sim] end 21:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:24:00 [---] [cpu_sched_debug] preliminary job list: 21:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:24:00 [---] [cpu_sched_debug] final job list: 21:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:24:00 [---] [cpu_sched_debug] enforce_run_list: end 21:24:00 [---] choose_project(): 77040.000000 21:24:00 [WUProp@Home] doing RPC to get work 21:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358559.09 cpu 2107592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107592.90 21:24:00 [---] Got 0 tasks 21:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:25:00 [---] [rr_sim] doing sim: CPU sched 21:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969126391.40G/439.78G) 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203667.39 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 220366.74 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220366.738851 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983300.65 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 198330.06 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418696.803816 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784970.58 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 178497.06 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597193.862285 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606473.53 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 160647.35 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757841.214908 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445826.17 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 144582.62 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902423.832267 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301243.56 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 130124.36 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032548.187891 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171119.20 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 117111.92 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149660.107953 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1054007.28 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 105400.73 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255060.836008 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948606.55 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 94860.66 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349921.491258 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853745.90 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 85374.59 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435296.080983 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768371.31 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 76837.13 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512133.211735 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691534.18 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 69153.42 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581286.629413 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622380.76 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 62238.08 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643524.705322 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560142.68 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 56014.27 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699538.973641 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504128.41 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 50412.84 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749951.815127 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453715.57 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 45371.56 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795323.372465 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408344.02 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 40834.40 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836157.774069 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367509.61 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 36750.96 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872908.735513 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330758.65 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 33075.87 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905984.600812 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297682.79 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 29768.28 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935752.879582 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267914.51 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 26791.45 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962544.330474 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241123.06 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 24112.31 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986656.636278 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217010.75 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 21701.08 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008357.711501 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195309.68 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 19530.97 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027888.679201 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175778.71 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 17577.87 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045466.550132 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158200.84 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 15820.08 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061286.633969 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142380.75 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 14238.08 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075524.709423 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128142.68 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 12814.27 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088338.977331 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115328.41 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 11532.84 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099871.818449 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103795.57 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 10379.56 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110251.375455 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93416.01 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 9341.60 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119592.976760 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84074.41 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 8407.44 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128000.417935 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75666.97 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 7566.70 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135567.114992 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68100.27 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 6810.03 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142377.142343 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61290.25 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 6129.02 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148506.166960 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55161.22 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 5516.12 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154022.289114 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49645.10 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 4964.51 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158986.799054 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44680.59 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 4468.06 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163454.857999 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40212.53 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 4021.25 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167476.111050 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36191.28 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3619.13 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171095.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32572.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174695.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28972.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178295.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25372.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181895.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21772.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185495.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18172.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189095.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14572.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192695.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10972.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196295.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7372.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199895.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3772.15 sec) 21:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203495.238796 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 172.15 sec) 21:25:00 [GPUGRID] [rr_sim] 2203667.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969126391.40G/439.78G) 21:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853808.30 21:25:00 [---] pick_jobs_to_run() start 21:25:00 [---] pick_jobs_to_run() end 21:25:00 [---] [rr_sim] end 21:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:25:00 [---] [cpu_sched_debug] preliminary job list: 21:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:25:00 [---] [cpu_sched_debug] final job list: 21:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:25:00 [---] [cpu_sched_debug] enforce_run_list: end 21:25:00 [---] choose_project(): 77100.000000 21:25:00 [WUProp@Home] doing RPC to get work 21:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358499.09 cpu 2107532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107532.90 21:25:00 [---] Got 0 tasks 21:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:26:00 [---] [rr_sim] doing sim: CPU sched 21:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969098801.04G/439.78G) 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203604.65 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 220360.47 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220360.465162 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983244.19 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 198324.42 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418684.883809 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784919.77 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 178491.98 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597176.860590 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606427.79 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 160642.78 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757819.639694 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445785.01 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 144578.50 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902398.140887 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301206.51 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 130120.65 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032518.791961 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171085.86 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 117108.59 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149627.377927 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053977.27 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 105397.73 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1255025.105297 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948579.55 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 94857.95 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349883.059930 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853721.59 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 85372.16 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435255.219099 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768349.43 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 76834.94 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512090.162352 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691514.49 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 69151.45 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581241.611279 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622363.04 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 62236.30 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643477.915314 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560126.74 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 56012.67 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699490.588945 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504114.06 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 50411.41 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749901.995213 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453702.66 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 45370.27 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795272.260854 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408332.39 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 40833.24 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836105.499931 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367499.15 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 36749.92 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872855.415100 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330749.24 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 33074.92 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905930.338753 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297674.31 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 29767.43 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935697.770040 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267906.88 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 26790.69 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962488.458198 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241116.19 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 24111.62 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986600.077541 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 217004.57 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 21700.46 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008300.534949 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195304.12 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 19530.41 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027830.946617 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175773.71 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 17577.37 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045408.317118 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158196.33 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 15819.63 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061227.950568 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142376.70 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 14237.67 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075465.620674 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128139.03 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 12813.90 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088279.523769 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115325.13 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 11532.51 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099812.036555 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103792.62 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 10379.26 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110191.298062 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93413.35 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 9341.34 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119532.633418 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84072.02 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 8407.20 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127939.835239 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75664.82 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 7566.48 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135506.316877 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68098.33 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 6809.83 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142316.150352 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61288.50 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 6128.85 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148445.000479 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55159.65 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 5515.97 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153960.965594 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49643.69 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 4964.37 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158925.334197 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44679.32 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 4467.93 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163393.265940 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40211.39 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 4021.14 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167414.404508 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36190.25 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3619.02 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171033.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32571.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174633.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28971.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178233.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25371.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181833.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21771.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185433.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18171.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189033.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14571.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192633.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10971.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196233.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7371.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199833.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3771.22 sec) 21:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203433.429220 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 171.22 sec) 21:26:00 [GPUGRID] [rr_sim] 2203604.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969098801.04G/439.78G) 21:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853805.56 21:26:00 [---] pick_jobs_to_run() start 21:26:00 [---] pick_jobs_to_run() end 21:26:00 [---] [rr_sim] end 21:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:26:00 [---] [cpu_sched_debug] preliminary job list: 21:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:26:00 [---] [cpu_sched_debug] final job list: 21:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:26:00 [---] [cpu_sched_debug] enforce_run_list: end 21:26:00 [---] choose_project(): 77160.000000 21:26:00 [WUProp@Home] doing RPC to get work 21:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358439.09 cpu 2107472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107472.90 21:26:00 [---] Got 0 tasks 21:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:27:00 [---] [rr_sim] doing sim: CPU sched 21:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969071210.69G/439.78G) 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203541.91 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 220354.19 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220354.191474 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983187.72 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 198318.77 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418672.963801 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784868.95 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 178486.90 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597159.858895 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606382.06 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 160638.21 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757798.064480 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445743.85 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 144574.39 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902372.449506 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301169.47 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 130116.95 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032489.396030 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171052.52 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 117105.25 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149594.647901 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053947.27 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 105394.73 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254989.374585 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948552.54 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 94855.25 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349844.628601 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853697.29 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 85369.73 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435214.357215 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768327.56 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 76832.76 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512047.112968 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691494.80 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 69149.48 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581196.593145 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622345.32 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 62234.53 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643431.125305 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560110.79 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 56011.08 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699442.204249 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504099.71 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 50409.97 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749852.175298 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453689.74 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 45368.97 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795221.149243 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408320.77 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 40832.08 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836053.225793 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367488.69 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 36748.87 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872802.094688 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330739.82 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 33073.98 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905876.076693 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297665.84 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 29766.58 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935642.660498 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267899.25 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 26789.93 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962432.585923 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241109.33 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 24110.93 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986543.518805 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216998.40 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 21699.84 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008243.358398 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195298.56 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 19529.86 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027773.214033 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175768.70 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 17576.87 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045350.084104 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158191.83 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 15819.18 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061169.267168 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142372.65 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 14237.26 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075406.531925 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128135.38 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 12813.54 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088220.070207 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115321.84 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 11532.18 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099752.254660 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103789.66 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 10378.97 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110131.220669 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93410.69 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 9341.07 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119472.290076 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84069.62 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 8406.96 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127879.252543 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75662.66 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 7566.27 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135445.518763 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68096.40 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 6809.64 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142255.158361 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61286.76 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 6128.68 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148383.833999 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55158.08 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 5515.81 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153899.642073 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49642.27 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 4964.23 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158863.869340 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44678.05 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 4467.80 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163331.673880 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40210.24 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 4021.02 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167352.697967 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36189.22 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3618.92 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170971.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32570.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174571.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28970.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178171.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25370.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181771.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21770.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185371.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18170.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188971.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14570.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192571.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10970.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196171.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7370.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199771.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3770.30 sec) 21:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203371.619644 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 170.30 sec) 21:27:00 [GPUGRID] [rr_sim] 2203541.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969071210.69G/439.78G) 21:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853802.83 21:27:00 [---] pick_jobs_to_run() start 21:27:00 [---] pick_jobs_to_run() end 21:27:00 [---] [rr_sim] end 21:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:27:00 [---] [cpu_sched_debug] preliminary job list: 21:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:27:00 [---] [cpu_sched_debug] final job list: 21:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:27:00 [---] [cpu_sched_debug] enforce_run_list: end 21:27:00 [---] choose_project(): 77220.000000 21:27:00 [WUProp@Home] doing RPC to get work 21:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358379.09 cpu 2107412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107412.90 21:27:00 [---] Got 0 tasks 21:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:28:00 [---] [rr_sim] doing sim: CPU sched 21:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969071210.69G/439.78G) 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203541.91 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 220354.19 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220354.191474 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983187.72 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 198318.77 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418672.963801 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784868.95 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 178486.90 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597159.858895 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606382.06 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 160638.21 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757798.064480 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445743.85 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 144574.39 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902372.449506 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301169.47 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 130116.95 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032489.396030 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171052.52 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 117105.25 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149594.647901 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053947.27 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 105394.73 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254989.374585 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948552.54 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 94855.25 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349844.628601 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853697.29 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 85369.73 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435214.357215 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768327.56 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 76832.76 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512047.112968 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691494.80 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 69149.48 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581196.593145 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622345.32 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 62234.53 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643431.125305 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560110.79 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 56011.08 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699442.204249 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504099.71 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 50409.97 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749852.175298 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453689.74 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 45368.97 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795221.149243 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408320.77 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 40832.08 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836053.225793 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367488.69 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 36748.87 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872802.094688 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330739.82 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 33073.98 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905876.076693 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297665.84 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 29766.58 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935642.660498 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267899.25 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 26789.93 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962432.585923 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241109.33 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 24110.93 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986543.518805 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216998.40 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 21699.84 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008243.358398 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195298.56 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 19529.86 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027773.214033 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175768.70 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 17576.87 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045350.084104 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158191.83 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 15819.18 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061169.267168 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142372.65 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 14237.26 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075406.531925 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128135.38 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 12813.54 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088220.070207 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115321.84 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 11532.18 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099752.254660 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103789.66 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 10378.97 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110131.220669 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93410.69 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 9341.07 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119472.290076 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84069.62 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 8406.96 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127879.252543 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75662.66 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 7566.27 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135445.518763 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68096.40 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 6809.64 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142255.158361 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61286.76 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 6128.68 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148383.833999 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55158.08 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 5515.81 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153899.642073 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49642.27 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 4964.23 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158863.869340 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44678.05 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 4467.80 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163331.673880 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40210.24 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 4021.02 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167352.697967 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36189.22 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3618.92 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170971.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32570.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174571.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28970.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178171.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25370.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181771.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21770.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185371.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18170.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188971.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14570.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192571.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10970.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196171.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7370.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199771.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3770.30 sec) 21:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203371.619644 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 170.30 sec) 21:28:00 [GPUGRID] [rr_sim] 2203541.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969071210.69G/439.78G) 21:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853862.83 21:28:00 [---] pick_jobs_to_run() start 21:28:00 [---] pick_jobs_to_run() end 21:28:00 [---] [rr_sim] end 21:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:28:00 [---] [cpu_sched_debug] preliminary job list: 21:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:28:00 [---] [cpu_sched_debug] final job list: 21:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:28:00 [---] [cpu_sched_debug] enforce_run_list: end 21:28:00 [---] choose_project(): 77280.000000 21:28:00 [WUProp@Home] doing RPC to get work 21:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358319.09 cpu 2107412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107412.90 21:28:00 [---] Got 0 tasks 21:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:29:00 [---] [rr_sim] doing sim: CPU sched 21:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969043620.33G/439.78G) 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203479.18 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 220347.92 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220347.917786 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983131.26 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 198313.13 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418661.043793 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784818.13 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 178481.81 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597142.857200 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606336.32 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 160633.63 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757776.489266 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445702.69 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 144570.27 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902346.758126 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301132.42 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 130113.24 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032460.000099 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171019.18 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 117101.92 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149561.917875 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053917.26 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 105391.73 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254953.643874 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948525.53 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 94852.55 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349806.197272 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853672.98 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 85367.30 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435173.495331 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768305.68 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 76830.57 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512004.063584 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691475.11 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 69147.51 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581151.575012 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622327.60 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 62232.76 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643384.335297 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560094.84 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 56009.48 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699393.819553 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504085.36 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 50408.54 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749802.355384 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453676.82 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 45367.68 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795170.037631 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408309.14 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 40830.91 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836000.951654 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367478.23 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 36747.82 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872748.774275 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330730.40 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 33073.04 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905821.814633 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297657.36 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 29765.74 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935587.550956 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267891.63 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 26789.16 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962376.713647 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241102.46 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 24110.25 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986486.960068 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216992.22 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 21699.22 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008186.181847 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195293.00 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 19529.30 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027715.481449 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175763.70 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 17576.37 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045291.851090 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158187.33 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 15818.73 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061110.583767 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142368.59 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 14236.86 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075347.443176 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128131.73 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 12813.17 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088160.616645 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115318.56 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 11531.86 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099692.472766 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103786.71 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 10378.67 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110071.143276 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93408.03 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 9340.80 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119411.946734 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84067.23 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 8406.72 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127818.669847 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75660.51 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 7566.05 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135384.720648 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68094.46 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 6809.45 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142194.166369 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61285.01 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 6128.50 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148322.667518 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55156.51 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 5515.65 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153838.318553 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49640.86 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 4964.09 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158802.404483 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44676.77 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 4467.68 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163270.081821 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40209.10 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 4020.91 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167290.991425 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36188.19 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3618.82 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170909.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32569.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174509.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28969.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178109.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25369.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181709.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21769.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185309.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18169.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188909.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14569.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192509.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10969.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196109.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7369.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199709.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3769.37 sec) 21:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203309.810069 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 169.37 sec) 21:29:00 [GPUGRID] [rr_sim] 2203479.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969043620.33G/439.78G) 21:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853860.09 21:29:00 [---] pick_jobs_to_run() start 21:29:00 [---] pick_jobs_to_run() end 21:29:00 [---] [rr_sim] end 21:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:29:00 [---] [cpu_sched_debug] preliminary job list: 21:29:00 [---] [cpu_sched_debug] final job list: 21:29:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 21:29:00 [---] [work_fetch] Request work fetch: CPUs idle 21:29:00 [---] [cpu_sched_debug] enforce_run_list: end 21:29:00 [---] choose_project(): 77340.000000 21:29:00 [WUProp@Home] doing RPC to get work 21:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358259.09 cpu 2107352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107352.90 21:29:00 [---] Got 0 tasks 21:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:30:00 [---] [rr_sim] doing sim: CPU sched 21:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969043620.33G/439.78G) 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203479.18 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 220347.92 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220347.917786 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983131.26 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 198313.13 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418661.043793 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784818.13 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 178481.81 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597142.857200 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606336.32 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 160633.63 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757776.489266 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445702.69 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 144570.27 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902346.758126 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301132.42 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 130113.24 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032460.000099 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1171019.18 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 117101.92 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149561.917875 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053917.26 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 105391.73 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254953.643874 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948525.53 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 94852.55 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349806.197272 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853672.98 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 85367.30 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435173.495331 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768305.68 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 76830.57 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1512004.063584 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691475.11 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 69147.51 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581151.575012 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622327.60 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 62232.76 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643384.335297 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560094.84 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 56009.48 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699393.819553 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504085.36 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 50408.54 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749802.355384 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453676.82 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 45367.68 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795170.037631 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408309.14 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 40830.91 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1836000.951654 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367478.23 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 36747.82 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872748.774275 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330730.40 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 33073.04 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905821.814633 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297657.36 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 29765.74 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935587.550956 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267891.63 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 26789.16 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962376.713647 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241102.46 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 24110.25 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986486.960068 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216992.22 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 21699.22 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008186.181847 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195293.00 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 19529.30 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027715.481449 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175763.70 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 17576.37 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045291.851090 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158187.33 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 15818.73 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061110.583767 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142368.59 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 14236.86 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075347.443176 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128131.73 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 12813.17 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088160.616645 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115318.56 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 11531.86 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099692.472766 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103786.71 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 10378.67 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110071.143276 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93408.03 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 9340.80 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119411.946734 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84067.23 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 8406.72 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127818.669847 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75660.51 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 7566.05 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135384.720648 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68094.46 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 6809.45 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142194.166369 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61285.01 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 6128.50 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148322.667518 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55156.51 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 5515.65 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153838.318553 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49640.86 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 4964.09 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158802.404483 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44676.77 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 4467.68 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163270.081821 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40209.10 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 4020.91 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167290.991425 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36188.19 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3618.82 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170909.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32569.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174509.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28969.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178109.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25369.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181709.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21769.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185309.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18169.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188909.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14569.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192509.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10969.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196109.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7369.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199709.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3769.37 sec) 21:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203309.810069 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 169.37 sec) 21:30:00 [GPUGRID] [rr_sim] 2203479.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969043620.33G/439.78G) 21:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853920.09 21:30:00 [---] pick_jobs_to_run() start 21:30:00 [---] pick_jobs_to_run() end 21:30:00 [---] [rr_sim] end 21:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:30:00 [---] [cpu_sched_debug] preliminary job list: 21:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:30:00 [---] [cpu_sched_debug] final job list: 21:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 21:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:30:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 969043620.33G 21:30:00 [---] [cpu_sched_debug] enforce_run_list: end 21:30:00 [---] choose_project(): 77400.000000 21:30:00 [WUProp@Home] doing RPC to get work 21:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358199.09 cpu 2107352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107352.90 21:30:00 [---] Got 0 tasks 21:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:31:00 [---] [rr_sim] doing sim: CPU sched 21:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (969016029.97G/439.78G) 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203416.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 220341.64 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220341.644098 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983074.80 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 198307.48 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418649.123786 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784767.32 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 178476.73 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597125.855505 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606290.59 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 160629.06 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757754.914052 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445661.53 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 144566.15 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902321.066745 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301095.37 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 130109.54 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032430.604168 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170985.84 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 117098.58 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149529.187849 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053887.25 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 105388.73 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254917.913162 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948498.53 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 94849.85 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349767.765944 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853648.68 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 85364.87 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435132.633447 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768283.81 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 76828.38 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511961.014200 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691455.43 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 69145.54 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581106.556878 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622309.88 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 62230.99 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643337.545288 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560078.90 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 56007.89 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699345.434857 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504071.01 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 50407.10 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749752.535469 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453663.91 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 45366.39 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795118.926020 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408297.51 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 40829.75 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835948.677516 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367467.76 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 36746.78 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872695.453862 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330720.99 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 33072.10 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905767.552574 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297648.89 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 29764.89 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935532.441414 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267884.00 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 26788.40 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962320.841371 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241095.60 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 24109.56 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986430.401331 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216986.04 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 21698.60 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008129.005296 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195287.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 19528.74 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027657.748864 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175758.69 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 17575.87 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045233.618076 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158182.82 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 15818.28 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2061051.900366 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142364.54 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 14236.45 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075288.354427 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128128.09 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 12812.81 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088101.163082 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115315.28 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 11531.53 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099632.690872 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103783.75 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 10378.38 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110011.065882 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93405.38 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 9340.54 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119351.603392 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84064.84 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 8406.48 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127758.087151 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75658.35 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 7565.84 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135323.922533 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68092.52 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 6809.25 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142133.174378 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61283.27 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 6128.33 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148261.501038 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55154.94 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 5515.49 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153776.995032 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49639.45 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 4963.94 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158740.939627 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44675.50 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 4467.55 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163208.489762 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40207.95 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 4020.80 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167229.284883 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36187.16 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3618.72 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170848.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32568.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174448.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28968.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178048.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25368.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181648.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21768.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185248.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18168.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188848.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14568.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192448.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10968.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196048.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7368.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199648.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3768.44 sec) 21:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203248.000493 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 168.44 sec) 21:31:00 [GPUGRID] [rr_sim] 2203416.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (969016029.97G/439.78G) 21:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853917.35 21:31:00 [---] pick_jobs_to_run() start 21:31:00 [---] pick_jobs_to_run() end 21:31:00 [---] [rr_sim] end 21:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:31:00 [---] [cpu_sched_debug] preliminary job list: 21:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 21:31:00 [---] [cpu_sched_debug] final job list: 21:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:31:00 [---] [cpu_sched_debug] enforce_run_list: end 21:31:00 [---] choose_project(): 77460.000000 21:31:00 [WUProp@Home] doing RPC to get work 21:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358139.09 cpu 2107292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107292.90 21:31:00 [---] Got 0 tasks 21:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:32:00 [---] [rr_sim] doing sim: CPU sched 21:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968988439.62G/439.78G) 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203353.70 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 220335.37 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220335.370410 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1983018.33 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 198301.83 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418637.203778 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784716.50 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 178471.65 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597108.853810 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606244.85 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 160624.49 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757733.338839 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445620.37 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 144562.04 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902295.375364 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301058.33 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 130105.83 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032401.208238 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170952.50 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 117095.25 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149496.457823 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053857.25 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 105385.72 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254882.182451 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948471.52 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 94847.15 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349729.334615 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853624.37 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 85362.44 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435091.771563 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768261.93 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 76826.19 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511917.964817 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691435.74 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 69143.57 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581061.538745 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622292.17 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 62229.22 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643290.755280 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560062.95 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 56006.29 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699297.050161 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504056.65 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 50405.67 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749702.715555 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453650.99 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 45365.10 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795067.814409 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408285.89 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 40828.59 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835896.403378 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367457.30 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 36745.73 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872642.133449 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330711.57 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 33071.16 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905713.290514 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297640.41 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 29764.04 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935477.331872 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267876.37 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 26787.64 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962264.969095 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241088.74 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 24108.87 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986373.842595 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216979.86 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 21697.99 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008071.828745 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195281.88 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 19528.19 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027600.016280 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175753.69 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 17575.37 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045175.385062 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158178.32 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 15817.83 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060993.216965 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142360.49 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 14236.05 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075229.265678 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128124.44 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 12812.44 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088041.709520 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115311.99 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 11531.20 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099572.908978 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103780.80 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 10378.08 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109950.988489 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93402.72 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 9340.27 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119291.260050 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84062.44 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 8406.24 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127697.504455 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75656.20 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 7565.62 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135263.124419 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68090.58 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 6809.06 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142072.182387 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61281.52 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 6128.15 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148200.334558 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55153.37 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 5515.34 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153715.671511 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49638.03 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 4963.80 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158679.474770 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44674.23 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 4467.42 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163146.897702 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40206.81 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 4020.68 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167167.578342 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36186.13 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3618.61 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170786.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32567.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174386.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28967.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177986.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25367.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181586.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21767.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185186.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18167.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188786.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14567.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192386.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10967.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195986.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7367.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199586.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3767.51 sec) 21:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203186.190917 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 167.51 sec) 21:32:00 [GPUGRID] [rr_sim] 2203353.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968988439.62G/439.78G) 21:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853914.62 21:32:00 [---] pick_jobs_to_run() start 21:32:00 [---] pick_jobs_to_run() end 21:32:00 [---] [rr_sim] end 21:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:32:00 [---] [cpu_sched_debug] preliminary job list: 21:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:32:00 [---] [cpu_sched_debug] final job list: 21:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:32:00 [---] [cpu_sched_debug] enforce_run_list: end 21:32:00 [---] choose_project(): 77520.000000 21:32:00 [WUProp@Home] doing RPC to get work 21:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358079.09 cpu 2107232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107232.90 21:32:00 [---] Got 0 tasks 21:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:33:00 [---] [rr_sim] doing sim: CPU sched 21:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968960849.26G/439.78G) 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203290.97 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 220329.10 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220329.096721 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982961.87 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 198296.19 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418625.283771 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784665.68 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 178466.57 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597091.852115 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606199.12 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 160619.91 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757711.763625 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445579.20 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 144557.92 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902269.683984 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1301021.28 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 130102.13 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032371.812307 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170919.15 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 117091.92 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149463.727797 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053827.24 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 105382.72 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254846.451739 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948444.52 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 94844.45 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349690.903287 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853600.06 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 85360.01 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435050.909679 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768240.06 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 76824.01 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511874.915433 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691416.05 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 69141.61 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1581016.520611 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622274.45 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 62227.44 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643243.965271 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560047.00 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 56004.70 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699248.665465 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504042.30 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 50404.23 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749652.895640 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453638.07 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 45363.81 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1795016.702798 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408274.26 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 40827.43 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835844.129239 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367446.84 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 36744.68 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872588.813037 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330702.15 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 33070.22 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905659.028454 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297631.94 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 29763.19 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935422.222330 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267868.74 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 26786.87 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962209.096819 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241081.87 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 24108.19 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986317.283858 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216973.68 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 21697.37 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2008014.652194 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195276.32 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 19527.63 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027542.283696 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175748.68 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 17574.87 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045117.152048 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158173.82 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 15817.38 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060934.533564 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142356.43 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 14235.64 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075170.176929 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128120.79 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 12812.08 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087982.255958 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115308.71 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 11530.87 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099513.127083 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103777.84 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 10377.78 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109890.911096 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93400.06 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 9340.01 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119230.916708 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84060.05 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 8406.01 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127636.921759 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75654.05 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 7565.40 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135202.326304 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68088.64 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 6808.86 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142011.190395 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61279.78 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 6127.98 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148139.168077 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55151.80 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 5515.18 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153654.347991 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49636.62 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 4963.66 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158618.009913 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44672.96 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 4467.30 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163085.305643 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40205.66 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 4020.57 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167105.871800 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36185.10 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3618.51 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170724.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32566.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174324.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28966.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177924.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25366.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181524.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21766.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185124.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18166.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188724.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14566.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192324.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10966.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195924.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7366.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199524.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3766.59 sec) 21:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203124.381342 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 166.59 sec) 21:33:00 [GPUGRID] [rr_sim] 2203290.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968960849.26G/439.78G) 21:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853911.88 21:33:00 [---] pick_jobs_to_run() start 21:33:00 [---] pick_jobs_to_run() end 21:33:00 [---] [rr_sim] end 21:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:33:00 [---] [cpu_sched_debug] preliminary job list: 21:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:33:00 [---] [cpu_sched_debug] final job list: 21:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:33:00 [---] [cpu_sched_debug] enforce_run_list: end 21:33:00 [---] choose_project(): 77580.000000 21:33:00 [WUProp@Home] doing RPC to get work 21:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 358019.09 cpu 2107172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107172.90 21:33:00 [---] Got 0 tasks 21:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:34:00 [---] [rr_sim] doing sim: CPU sched 21:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968933258.91G/439.78G) 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203228.23 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 220322.82 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220322.823033 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982905.41 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 198290.54 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418613.363763 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784614.87 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 178461.49 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597074.850420 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606153.38 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 160615.34 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757690.188411 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445538.04 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 144553.80 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902243.992603 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300984.24 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 130098.42 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032342.416376 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170885.81 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 117088.58 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149430.997772 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053797.23 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 105379.72 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254810.721028 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948417.51 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 94841.75 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349652.471958 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853575.76 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 85357.58 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1435010.047795 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768218.18 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 76821.82 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511831.866049 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691396.36 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 69139.64 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580971.502477 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622256.73 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 62225.67 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643197.175263 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560031.06 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 56003.11 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699200.280770 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504027.95 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 50402.79 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749603.075726 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453625.15 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 45362.52 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794965.591186 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408262.64 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 40826.26 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835791.855101 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367436.38 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 36743.64 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872535.492624 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330692.74 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 33069.27 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905604.766395 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297623.46 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 29762.35 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935367.112788 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267861.12 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 26786.11 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962153.224543 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241075.01 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 24107.50 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986260.725122 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216967.51 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 21696.75 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007957.475643 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195270.75 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 19527.08 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027484.551112 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175743.68 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 17574.37 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045058.919034 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158169.31 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 15816.93 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060875.850163 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142352.38 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 14235.24 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075111.088180 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128117.14 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 12811.71 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087922.802395 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115305.43 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 11530.54 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099453.345189 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103774.89 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 10377.49 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109830.833703 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93397.40 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 9339.74 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119170.573366 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84057.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 8405.77 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127576.339063 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75651.89 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 7565.19 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135141.528190 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68086.70 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 6808.67 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141950.198404 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61278.03 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 6127.80 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148078.001597 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55150.23 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 5515.02 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153593.024470 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49635.21 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 4963.52 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158556.545056 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44671.69 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 4467.17 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163023.713584 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40204.52 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 4020.45 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167044.165259 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36184.07 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3618.41 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170662.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32565.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174262.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28965.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177862.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25365.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181462.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21765.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185062.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18165.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188662.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14565.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192262.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10965.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195862.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7365.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199462.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3765.66 sec) 21:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203062.571766 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 165.66 sec) 21:34:00 [GPUGRID] [rr_sim] 2203228.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968933258.91G/439.78G) 21:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853909.14 21:34:00 [---] pick_jobs_to_run() start 21:34:00 [---] pick_jobs_to_run() end 21:34:00 [---] [rr_sim] end 21:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:34:00 [---] [cpu_sched_debug] preliminary job list: 21:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:34:00 [---] [cpu_sched_debug] final job list: 21:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:34:00 [---] [cpu_sched_debug] enforce_run_list: end 21:34:00 [---] choose_project(): 77640.000000 21:34:00 [WUProp@Home] doing RPC to get work 21:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357959.09 cpu 2107112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107112.90 21:34:00 [---] Got 0 tasks 21:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:35:00 [---] [rr_sim] doing sim: CPU sched 21:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968905668.55G/439.78G) 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203165.49 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 220316.55 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220316.549345 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982848.94 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 198284.89 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418601.443755 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784564.05 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 178456.40 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597057.848725 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606107.64 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 160610.76 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757668.613197 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445496.88 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 144549.69 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902218.301223 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300947.19 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 130094.72 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032313.020445 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170852.47 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 117085.25 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149398.267746 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053767.23 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 105376.72 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254774.990316 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948390.50 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 94839.05 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349614.040629 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853551.45 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 85355.15 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434969.185911 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768196.31 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 76819.63 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511788.816665 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691376.68 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 69137.67 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580926.484344 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622239.01 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 62223.90 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643150.385254 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 560015.11 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 56001.51 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699151.896074 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 504013.60 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 50401.36 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749553.255811 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453612.24 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 45361.22 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794914.479575 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408251.01 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 40825.10 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835739.580963 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367425.91 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 36742.59 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872482.172211 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330683.32 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 33068.33 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905550.504335 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297614.99 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 29761.50 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935312.003247 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267853.49 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 26785.35 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962097.352267 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241068.14 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 24106.81 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986204.166385 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216961.33 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 21696.13 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007900.299092 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195265.19 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 19526.52 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027426.818527 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175738.67 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 17573.87 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2045000.686020 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158164.81 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 15816.48 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060817.166763 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142348.33 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 14234.83 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2075051.999431 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128113.49 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 12811.35 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087863.348833 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115302.14 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 11530.21 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099393.563295 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103771.93 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 10377.19 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109770.756310 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93394.74 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 9339.47 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119110.230024 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84055.26 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 8405.53 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127515.756367 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75649.74 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 7564.97 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135080.730075 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68084.76 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 6808.48 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141889.206412 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61276.29 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 6127.63 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148016.835116 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55148.66 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 5514.87 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153531.700950 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49633.79 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 4963.38 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158495.080200 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44670.41 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 4467.04 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162962.121525 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40203.37 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 4020.34 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166982.458717 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36183.03 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3618.30 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170600.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32564.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174200.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28964.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177800.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25364.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181400.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21764.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185000.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18164.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188600.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14564.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192200.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10964.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195800.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7364.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199400.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3764.73 sec) 21:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2203000.762190 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 164.73 sec) 21:35:00 [GPUGRID] [rr_sim] 2203165.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968905668.55G/439.78G) 21:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853906.41 21:35:00 [---] pick_jobs_to_run() start 21:35:00 [---] pick_jobs_to_run() end 21:35:00 [---] [rr_sim] end 21:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:35:00 [---] [cpu_sched_debug] preliminary job list: 21:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:35:00 [---] [cpu_sched_debug] final job list: 21:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:35:00 [---] [cpu_sched_debug] enforce_run_list: end 21:35:00 [---] choose_project(): 77700.000000 21:35:00 [WUProp@Home] doing RPC to get work 21:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357899.09 cpu 2107052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2107052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2107052.90 21:35:00 [---] Got 0 tasks 21:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:36:00 [---] [rr_sim] doing sim: CPU sched 21:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968878078.19G/439.78G) 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203102.76 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 220310.28 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220310.275657 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982792.48 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 198279.25 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418589.523748 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784513.23 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 178451.32 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597040.847030 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606061.91 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 160606.19 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757647.037984 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445455.72 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 144545.57 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902192.609842 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300910.15 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 130091.01 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032283.624514 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170819.13 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 117081.91 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149365.537720 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053737.22 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 105373.72 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254739.259605 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948363.50 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 94836.35 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349575.609301 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853527.15 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 85352.71 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434928.324027 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768174.43 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 76817.44 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511745.767281 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691356.99 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 69135.70 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580881.466210 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622221.29 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 62222.13 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643103.595246 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559999.16 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 55999.92 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699103.511378 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503999.25 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 50399.92 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749503.435897 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453599.32 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 45359.93 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794863.367964 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408239.39 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 40823.94 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835687.306824 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367415.45 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 36741.54 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872428.851799 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330673.90 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 33067.39 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905496.242275 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297606.51 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 29760.65 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935256.893705 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267845.86 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 26784.59 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1962041.479991 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241061.28 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 24106.13 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986147.607649 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216955.15 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 21695.51 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007843.122540 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195259.63 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 19525.96 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027369.085943 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175733.67 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 17573.37 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044942.453006 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158160.30 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 15816.03 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060758.483362 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142344.27 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 14234.43 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074992.910682 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128109.85 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 12810.98 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087803.895271 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115298.86 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 11529.89 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099333.781400 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103768.98 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 10376.90 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109710.678917 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93392.08 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 9339.21 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119049.886682 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84052.87 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 8405.29 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127455.173671 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75647.58 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 7564.76 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135019.931960 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68082.82 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 6808.28 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141828.214421 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61274.54 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 6127.45 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147955.668636 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55147.09 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 5514.71 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153470.377429 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49632.38 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 4963.24 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158433.615343 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44669.14 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 4466.91 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162900.529465 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40202.23 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 4020.22 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166920.752175 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36182.00 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3618.20 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170538.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32563.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174138.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28963.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177738.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25363.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181338.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21763.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184938.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18163.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188538.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14563.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192138.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10963.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195738.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7363.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199338.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3763.80 sec) 21:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202938.952615 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 163.80 sec) 21:36:00 [GPUGRID] [rr_sim] 2203102.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968878078.19G/439.78G) 21:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853903.67 21:36:00 [---] pick_jobs_to_run() start 21:36:00 [---] pick_jobs_to_run() end 21:36:00 [---] [rr_sim] end 21:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:36:00 [---] [cpu_sched_debug] preliminary job list: 21:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:36:00 [---] [cpu_sched_debug] final job list: 21:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:36:00 [---] [cpu_sched_debug] enforce_run_list: end 21:36:00 [---] choose_project(): 77760.000000 21:36:00 [WUProp@Home] doing RPC to get work 21:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357839.09 cpu 2106992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106992.90 21:36:00 [---] Got 0 tasks 21:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:37:00 [---] [rr_sim] doing sim: CPU sched 21:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968850487.84G/439.78G) 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2203040.02 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 220304.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220304.001969 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982736.02 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 198273.60 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418577.603740 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784462.42 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 178446.24 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597023.845335 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1606016.17 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 160601.62 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757625.462770 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445414.56 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 144541.46 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902166.918461 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300873.10 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 130087.31 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032254.228584 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170785.79 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 117078.58 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149332.807694 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053707.21 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 105370.72 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254703.528893 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948336.49 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 94833.65 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349537.177972 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853502.84 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 85350.28 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434887.462144 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768152.56 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 76815.26 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511702.717898 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691337.30 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 69133.73 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580836.448076 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622203.57 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 62220.36 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643056.805237 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559983.21 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 55998.32 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699055.126682 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503984.89 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 50398.49 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749453.615982 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453586.40 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 45358.64 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794812.256353 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408227.76 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 40822.78 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835635.032686 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367404.99 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 36740.50 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872375.531386 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330664.49 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 33066.45 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905441.980216 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297598.04 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 29759.80 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935201.784163 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267838.24 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 26783.82 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961985.607715 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241054.41 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 24105.44 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986091.048912 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216948.97 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 21694.90 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007785.945989 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195254.07 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 19525.41 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027311.353359 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175728.67 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 17572.87 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044884.219992 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158155.80 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 15815.58 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060699.799961 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142340.22 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 14234.02 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074933.821933 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128106.20 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 12810.62 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087744.441709 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115295.58 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 11529.56 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099273.999506 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103766.02 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 10376.60 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109650.601524 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93389.42 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 9338.94 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118989.543340 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84050.48 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 8405.05 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127394.590975 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75645.43 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 7564.54 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134959.133846 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68080.89 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 6808.09 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141767.222430 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61272.80 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 6127.28 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147894.502155 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55145.52 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 5514.55 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153409.053908 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49630.97 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 4963.10 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158372.150486 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44667.87 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 4466.79 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162838.937406 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40201.08 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 4020.11 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166859.045634 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36180.97 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3618.10 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170477.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32562.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174077.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28962.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177677.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25362.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181277.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21762.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184877.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18162.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188477.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14562.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192077.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10962.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195677.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7362.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199277.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3762.88 sec) 21:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202877.143039 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 162.88 sec) 21:37:00 [GPUGRID] [rr_sim] 2203040.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968850487.84G/439.78G) 21:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853900.93 21:37:00 [---] pick_jobs_to_run() start 21:37:00 [---] pick_jobs_to_run() end 21:37:00 [---] [rr_sim] end 21:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:37:00 [---] [cpu_sched_debug] preliminary job list: 21:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:37:00 [---] [cpu_sched_debug] final job list: 21:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:37:00 [---] [cpu_sched_debug] enforce_run_list: end 21:37:00 [---] choose_project(): 77820.000000 21:37:00 [WUProp@Home] doing RPC to get work 21:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357779.09 cpu 2106932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106932.90 21:37:00 [---] Got 0 tasks 21:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:38:00 [---] [rr_sim] doing sim: CPU sched 21:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968822897.48G/439.78G) 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202977.28 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 220297.73 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220297.728280 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982679.55 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 198267.96 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418565.683733 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784411.60 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 178441.16 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 597006.843640 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605970.44 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 160597.04 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757603.887556 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445373.40 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 144537.34 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902141.227081 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300836.06 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 130083.61 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032224.832653 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170752.45 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 117075.25 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149300.077668 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053677.21 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 105367.72 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254667.798181 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948309.48 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 94830.95 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349498.746644 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853478.54 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 85347.85 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434846.600260 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768130.68 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 76813.07 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511659.668514 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691317.61 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 69131.76 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580791.429943 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622185.85 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 62218.59 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1643010.015229 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559967.27 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 55996.73 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1699006.741986 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503970.54 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 50397.05 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749403.796068 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453573.49 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 45357.35 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794761.144741 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408216.14 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 40821.61 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835582.758548 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367394.52 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 36739.45 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872322.210973 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330655.07 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 33065.51 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905387.718156 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297589.56 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 29758.96 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935146.674621 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267830.61 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 26783.06 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961929.735439 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241047.55 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 24104.75 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1986034.490175 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216942.79 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 21694.28 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007728.769438 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195248.51 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 19524.85 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027253.620775 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175723.66 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 17572.37 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044825.986978 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158151.30 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 15815.13 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060641.116560 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142336.17 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 14233.62 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074874.733184 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128102.55 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 12810.25 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087684.988146 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115292.29 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 11529.23 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099214.217612 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103763.07 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 10376.31 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109590.524131 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93386.76 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 9338.68 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118929.199998 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84048.08 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 8404.81 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127334.008279 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75643.27 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 7564.33 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134898.335731 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68078.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 6807.89 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141706.230438 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61271.05 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 6127.11 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147833.335675 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55143.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 5514.39 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153347.730388 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49629.55 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 4962.96 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158310.685629 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44666.60 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 4466.66 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162777.345347 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40199.94 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 4019.99 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166797.339092 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36179.94 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3617.99 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170415.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32561.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174015.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28961.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177615.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25361.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181215.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21761.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184815.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18161.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188415.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14561.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192015.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10961.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195615.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7361.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199215.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3761.95 sec) 21:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202815.333463 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161.95 sec) 21:38:00 [GPUGRID] [rr_sim] 2202977.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968822897.48G/439.78G) 21:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853898.20 21:38:00 [---] pick_jobs_to_run() start 21:38:00 [---] pick_jobs_to_run() end 21:38:00 [---] [rr_sim] end 21:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:38:00 [---] [cpu_sched_debug] preliminary job list: 21:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:38:00 [---] [cpu_sched_debug] final job list: 21:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:38:00 [---] [cpu_sched_debug] enforce_run_list: end 21:38:00 [---] choose_project(): 77880.000000 21:38:00 [WUProp@Home] doing RPC to get work 21:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357719.09 cpu 2106872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106872.90 21:38:00 [---] Got 0 tasks 21:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:39:00 [---] [rr_sim] doing sim: CPU sched 21:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968795307.13G/439.78G) 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202914.55 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 220291.45 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220291.454592 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982623.09 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 198262.31 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418553.763725 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784360.78 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 178436.08 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596989.841945 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605924.70 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 160592.47 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757582.312342 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445332.23 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 144533.22 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902115.535700 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300799.01 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 130079.90 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032195.436722 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170719.11 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 117071.91 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149267.347642 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053647.20 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 105364.72 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254632.067470 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948282.48 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 94828.25 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349460.315315 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853454.23 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 85345.42 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434805.738376 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768108.81 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 76810.88 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511616.619130 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691297.93 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 69129.79 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580746.411809 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622168.13 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 62216.81 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642963.225220 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559951.32 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 55995.13 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698958.357290 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503956.19 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 50395.62 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749353.976153 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453560.57 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 45356.06 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794710.033130 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408204.51 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 40820.45 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835530.484409 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367384.06 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 36738.41 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872268.890560 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330645.66 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 33064.57 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905333.456096 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297581.09 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 29758.11 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935091.565079 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267822.98 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 26782.30 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961873.863163 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241040.68 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 24104.07 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985977.931439 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216936.61 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 21693.66 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007671.592887 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195242.95 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 19524.30 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027195.888190 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175718.66 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 17571.87 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044767.753964 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158146.79 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 15814.68 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060582.433159 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142332.11 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 14233.21 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074815.644435 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128098.90 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 12809.89 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087625.534584 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115289.01 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 11528.90 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099154.435718 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103760.11 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 10376.01 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109530.446738 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93384.10 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 9338.41 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118868.856656 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84045.69 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 8404.57 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127273.425583 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75641.12 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 7564.11 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134837.537617 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68077.01 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 6807.70 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141645.238447 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61269.31 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 6126.93 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147772.169194 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55142.38 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 5514.24 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153286.406867 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49628.14 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 4962.81 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158249.220772 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44665.33 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 4466.53 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162715.753287 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40198.79 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 4019.88 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166735.632551 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36178.91 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3617.89 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170353.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32561.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173953.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28961.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177553.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25361.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181153.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21761.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184753.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18161.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188353.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14561.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191953.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10961.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195553.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7361.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199153.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3761.02 sec) 21:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202753.523888 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 161.02 sec) 21:39:00 [GPUGRID] [rr_sim] 2202914.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968795307.13G/439.78G) 21:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853895.46 21:39:00 [---] pick_jobs_to_run() start 21:39:00 [---] pick_jobs_to_run() end 21:39:00 [---] [rr_sim] end 21:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:39:00 [---] [cpu_sched_debug] preliminary job list: 21:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:39:00 [---] [cpu_sched_debug] final job list: 21:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:39:00 [---] [cpu_sched_debug] enforce_run_list: end 21:39:00 [---] choose_project(): 77940.000000 21:39:00 [WUProp@Home] doing RPC to get work 21:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357659.09 cpu 2106812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106812.90 21:39:00 [---] Got 0 tasks 21:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:40:00 [---] [rr_sim] doing sim: CPU sched 21:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968767716.77G/439.78G) 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202851.81 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 220285.18 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220285.180904 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982566.63 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 198256.66 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418541.843717 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784309.97 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 178431.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596972.840249 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605878.97 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 160587.90 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757560.737128 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445291.07 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 144529.11 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902089.844319 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300761.96 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 130076.20 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032166.040791 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170685.77 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 117068.58 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149234.617616 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053617.19 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 105361.72 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254596.336758 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948255.47 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 94825.55 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349421.883986 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853429.93 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 85342.99 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434764.876492 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768086.93 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 76808.69 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511573.569746 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691278.24 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 69127.82 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580701.393676 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622150.42 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 62215.04 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642916.435212 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559935.37 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 55993.54 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698909.972595 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503941.84 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 50394.18 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749304.156239 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453547.65 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 45354.77 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794658.921519 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408192.89 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 40819.29 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835478.210271 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367373.60 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 36737.36 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872215.570148 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330636.24 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 33063.62 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905279.194037 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297572.62 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 29757.26 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1935036.455537 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267815.35 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 26781.54 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961817.990887 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241033.82 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 24103.38 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985921.372702 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216930.44 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 21693.04 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007614.416336 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195237.39 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 19523.74 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027138.155606 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175713.65 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 17571.37 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044709.520950 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158142.29 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 15814.23 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060523.749758 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142328.06 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 14232.81 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074756.555686 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128095.25 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 12809.53 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087566.081022 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115285.73 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 11528.57 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099094.653823 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103757.16 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 10375.72 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109470.369345 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93381.44 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 9338.14 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118808.513314 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84043.30 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 8404.33 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127212.842887 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75638.97 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 7563.90 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134776.739502 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68075.07 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 6807.51 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141584.246456 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61267.56 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 6126.76 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147711.002714 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55140.81 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 5514.08 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153225.083346 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49626.73 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 4962.67 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158187.755916 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44664.05 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 4466.41 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162654.161228 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40197.65 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 4019.76 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166673.926009 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36177.88 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3617.79 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170291.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32560.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173891.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28960.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177491.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25360.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181091.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21760.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184691.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18160.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188291.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14560.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191891.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10960.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195491.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7360.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199091.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3760.09 sec) 21:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202691.714312 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 160.09 sec) 21:40:00 [GPUGRID] [rr_sim] 2202851.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968767716.77G/439.78G) 21:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853892.72 21:40:00 [---] pick_jobs_to_run() start 21:40:00 [---] pick_jobs_to_run() end 21:40:00 [---] [rr_sim] end 21:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:40:00 [---] [cpu_sched_debug] preliminary job list: 21:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:40:00 [---] [cpu_sched_debug] final job list: 21:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:40:00 [---] [cpu_sched_debug] enforce_run_list: end 21:40:00 [---] choose_project(): 78000.000000 21:40:00 [WUProp@Home] doing RPC to get work 21:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357599.09 cpu 2106752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106752.90 21:40:00 [---] Got 0 tasks 21:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:41:00 [---] [rr_sim] doing sim: CPU sched 21:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968740126.41G/439.78G) 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202789.07 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 220278.91 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220278.907216 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982510.16 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 198251.02 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418529.923710 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784259.15 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 178425.91 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596955.838554 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605833.23 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 160583.32 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757539.161915 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445249.91 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 144524.99 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902064.152939 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300724.92 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 130072.49 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032136.644861 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170652.43 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 117065.24 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149201.887590 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053587.18 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 105358.72 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254560.606047 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948228.47 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 94822.85 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349383.452658 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853405.62 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 85340.56 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434724.014608 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768065.06 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 76806.51 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511530.520363 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691258.55 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 69125.86 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580656.375542 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622132.70 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 62213.27 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642869.645203 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559919.43 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 55991.94 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698861.587899 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503927.48 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 50392.75 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749254.336325 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453534.74 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 45353.47 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794607.809908 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408181.26 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 40818.13 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835425.936133 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367363.14 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 36736.31 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872162.249735 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330626.82 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 33062.68 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905224.931977 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297564.14 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 29756.41 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934981.345995 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267807.73 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 26780.77 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961762.118611 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241026.95 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 24102.70 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985864.813966 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216924.26 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 21692.43 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007557.239785 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195231.83 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 19523.18 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027080.423022 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175708.65 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 17570.86 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044651.287935 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158137.78 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 15813.78 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060465.066358 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142324.01 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 14232.40 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074697.466937 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128091.61 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 12809.16 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087506.627459 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115282.44 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 11528.24 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099034.871929 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103754.20 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 10375.42 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109410.291952 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93378.78 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 9337.88 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118748.169972 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84040.90 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 8404.09 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127152.260191 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75636.81 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 7563.68 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134715.941387 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68073.13 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 6807.31 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141523.254464 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61265.82 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 6126.58 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147649.836233 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55139.24 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 5513.92 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153163.759826 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49625.31 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 4962.53 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158126.291059 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44662.78 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 4466.28 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162592.569169 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40196.50 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 4019.65 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166612.219467 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36176.85 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3617.69 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170229.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32559.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173829.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28959.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177429.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25359.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181029.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21759.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184629.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18159.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188229.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14559.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191829.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10959.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195429.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7359.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199029.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3759.17 sec) 21:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202629.904736 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 159.17 sec) 21:41:00 [GPUGRID] [rr_sim] 2202789.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968740126.41G/439.78G) 21:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853889.98 21:41:00 [---] pick_jobs_to_run() start 21:41:00 [---] pick_jobs_to_run() end 21:41:00 [---] [rr_sim] end 21:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:41:00 [---] [cpu_sched_debug] preliminary job list: 21:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:41:00 [---] [cpu_sched_debug] final job list: 21:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:41:00 [---] [cpu_sched_debug] enforce_run_list: end 21:41:00 [---] choose_project(): 78060.000000 21:41:00 [WUProp@Home] doing RPC to get work 21:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357539.09 cpu 2106692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106692.90 21:41:00 [---] Got 0 tasks 21:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:42:00 [---] [rr_sim] doing sim: CPU sched 21:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968712536.06G/439.78G) 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202726.34 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 220272.63 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220272.633527 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982453.70 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 198245.37 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418518.003702 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784208.33 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 178420.83 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596938.836859 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605787.50 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 160578.75 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757517.586701 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445208.75 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 144520.87 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902038.461558 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300687.87 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 130068.79 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032107.248930 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170619.09 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 117061.91 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149169.157564 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053557.18 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 105355.72 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254524.875335 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948201.46 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 94820.15 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349345.021329 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853381.31 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 85338.13 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434683.152724 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768043.18 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 76804.32 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511487.470979 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691238.86 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 69123.89 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580611.357408 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622114.98 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 62211.50 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642822.855195 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559903.48 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 55990.35 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698813.203203 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503913.13 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 50391.31 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749204.516410 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453521.82 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 45352.18 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794556.698296 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408169.64 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 40816.96 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835373.661994 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367352.67 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 36735.27 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872108.929322 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330617.41 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 33061.74 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905170.669918 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297555.67 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 29755.57 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934926.236453 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267800.10 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 26780.01 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961706.246335 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241020.09 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 24102.01 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985808.255229 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216918.08 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 21691.81 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007500.063234 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195226.27 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 19522.63 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2027022.690438 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175703.64 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 17570.36 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044593.054921 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158133.28 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 15813.33 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060406.382957 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142319.95 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 14232.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074638.378189 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128087.96 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 12808.80 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087447.173897 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115279.16 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 11527.92 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098975.090035 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103751.25 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 10375.12 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109350.214559 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93376.12 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 9337.61 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118687.826630 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84038.51 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 8403.85 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127091.677495 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75634.66 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 7563.47 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134655.143273 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68071.19 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 6807.12 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141462.262473 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61264.07 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 6126.41 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147588.669753 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55137.67 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 5513.77 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153102.436305 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49623.90 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 4962.39 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158064.826202 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44661.51 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 4466.15 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162530.977109 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40195.36 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 4019.54 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166550.512926 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36175.82 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3617.58 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170168.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32558.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173768.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28958.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177368.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25358.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180968.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21758.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184568.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18158.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188168.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14558.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191768.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10958.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195368.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7358.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198968.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3758.24 sec) 21:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202568.095161 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158.24 sec) 21:42:00 [GPUGRID] [rr_sim] 2202726.34: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968712536.06G/439.78G) 21:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853887.25 21:42:00 [---] pick_jobs_to_run() start 21:42:00 [---] pick_jobs_to_run() end 21:42:00 [---] [rr_sim] end 21:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:42:00 [---] [cpu_sched_debug] preliminary job list: 21:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:42:00 [---] [cpu_sched_debug] final job list: 21:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:42:00 [---] [cpu_sched_debug] enforce_run_list: end 21:42:00 [---] choose_project(): 78120.000000 21:42:00 [WUProp@Home] doing RPC to get work 21:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357479.09 cpu 2106632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106632.90 21:42:00 [---] Got 0 tasks 21:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:43:00 [---] [rr_sim] doing sim: CPU sched 21:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968684945.70G/439.78G) 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202663.60 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 220266.36 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220266.359839 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982397.24 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 198239.72 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418506.083695 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784157.51 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 178415.75 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596921.835164 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605741.76 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 160574.18 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757496.011487 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445167.59 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 144516.76 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 902012.770178 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300650.83 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 130065.08 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032077.852999 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170585.75 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 117058.57 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149136.427538 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053527.17 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 105352.72 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254489.144624 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948174.45 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 94817.45 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349306.590001 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853357.01 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 85335.70 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434642.290840 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 768021.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 76802.13 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511444.421595 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691219.18 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 69121.92 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580566.339275 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622097.26 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 62209.73 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642776.065187 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559887.53 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 55988.75 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698764.818507 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503898.78 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 50389.88 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749154.696496 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453508.90 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 45350.89 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794505.586685 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408158.01 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 40815.80 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835321.387856 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367342.21 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 36734.22 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872055.608910 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330607.99 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 33060.80 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905116.407858 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297547.19 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 29754.72 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934871.126911 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267792.47 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 26779.25 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961650.374059 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241013.22 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 24101.32 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985751.696493 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216911.90 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 21691.19 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007442.886683 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195220.71 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 19522.07 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026964.957854 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175698.64 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 17569.86 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044534.821907 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158128.78 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 15812.88 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060347.699556 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142315.90 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 14231.59 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074579.289440 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128084.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 12808.43 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087387.720335 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115275.88 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 11527.59 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098915.308141 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103748.29 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 10374.83 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109290.137166 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93373.46 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 9337.35 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118627.483288 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84036.12 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 8403.61 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127031.094799 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75632.50 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 7563.25 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134594.345158 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68069.25 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 6806.93 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141401.270482 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61262.33 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 6126.23 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147527.503273 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55136.10 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 5513.61 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153041.112785 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49622.49 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 4962.25 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158003.361345 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44660.24 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 4466.02 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162469.385050 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40194.21 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 4019.42 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166488.806384 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36174.79 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3617.48 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170106.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32557.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173706.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28957.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177306.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25357.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180906.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21757.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184506.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18157.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188106.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14557.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191706.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10957.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195306.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7357.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198906.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3757.31 sec) 21:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202506.285585 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157.31 sec) 21:43:00 [GPUGRID] [rr_sim] 2202663.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968684945.70G/439.78G) 21:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853884.51 21:43:00 [---] pick_jobs_to_run() start 21:43:00 [---] pick_jobs_to_run() end 21:43:00 [---] [rr_sim] end 21:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:43:00 [---] [cpu_sched_debug] preliminary job list: 21:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:43:00 [---] [cpu_sched_debug] final job list: 21:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:43:00 [---] [cpu_sched_debug] enforce_run_list: end 21:43:00 [---] choose_project(): 78180.000000 21:43:00 [WUProp@Home] doing RPC to get work 21:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357419.09 cpu 2106572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106572.90 21:43:00 [---] Got 0 tasks 21:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:44:00 [---] [rr_sim] doing sim: CPU sched 21:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968657355.34G/439.78G) 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202600.86 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 220260.09 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220260.086151 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982340.78 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 198234.08 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418494.163687 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784106.70 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 178410.67 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596904.833469 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605696.03 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 160569.60 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757474.436273 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445126.43 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 144512.64 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901987.078797 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300613.78 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 130061.38 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032048.457068 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170552.40 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 117055.24 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149103.697512 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053497.16 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 105349.72 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254453.413912 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948147.45 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 94814.74 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349268.158672 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853332.70 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 85333.27 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434601.428956 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767999.43 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 76799.94 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511401.372211 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691199.49 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 69119.95 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580521.321141 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622079.54 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 62207.95 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642729.275178 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559871.59 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 55987.16 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698716.433811 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503884.43 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 50388.44 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749104.876581 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453495.98 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 45349.60 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794454.475074 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408146.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 40814.64 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835269.113718 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367331.75 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 36733.17 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1872002.288497 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330598.57 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 33059.86 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905062.145798 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297538.72 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 29753.87 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934816.017369 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267784.84 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 26778.48 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961594.501783 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 241006.36 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 24100.64 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985695.137756 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216905.72 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 21690.57 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007385.710132 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195215.15 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 19521.52 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026907.225269 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175693.64 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 17569.36 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044476.588893 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158124.27 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 15812.43 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060289.016155 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142311.85 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 14231.18 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074520.200691 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128080.66 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 12808.07 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087328.266773 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115272.59 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 11527.26 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098855.526246 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103745.34 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 10374.53 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109230.059773 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93370.80 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 9337.08 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118567.139946 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84033.72 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 8403.37 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126970.512103 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75630.35 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 7563.03 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134533.547044 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68067.31 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 6806.73 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141340.278490 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61260.58 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 6126.06 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147466.336792 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55134.52 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 5513.45 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152979.789264 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49621.07 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 4962.11 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157941.896489 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44658.97 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 4465.90 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162407.792991 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40193.07 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 4019.31 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166427.099843 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36173.76 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3617.38 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170044.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32556.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173644.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28956.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177244.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25356.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180844.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21756.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184444.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18156.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188044.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14556.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191644.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10956.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195244.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7356.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198844.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3756.39 sec) 21:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202444.476009 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 156.39 sec) 21:44:00 [GPUGRID] [rr_sim] 2202600.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968657355.34G/439.78G) 21:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853881.77 21:44:00 [---] pick_jobs_to_run() start 21:44:00 [---] pick_jobs_to_run() end 21:44:00 [---] [rr_sim] end 21:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:44:00 [---] [cpu_sched_debug] preliminary job list: 21:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:44:00 [---] [cpu_sched_debug] final job list: 21:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:44:00 [---] [cpu_sched_debug] enforce_run_list: end 21:44:00 [---] choose_project(): 78240.000000 21:44:00 [WUProp@Home] doing RPC to get work 21:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357359.09 cpu 2106512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106512.90 21:44:00 [---] Got 0 tasks 21:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:45:00 [---] [rr_sim] doing sim: CPU sched 21:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968629764.99G/439.78G) 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202538.12 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 220253.81 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220253.812463 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982284.31 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 198228.43 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418482.243679 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784055.88 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 178405.59 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596887.831774 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605650.29 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 160565.03 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757452.861060 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445085.26 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 144508.53 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901961.387416 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300576.74 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 130057.67 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1032019.061138 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170519.06 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 117051.91 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149070.967487 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053467.16 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 105346.72 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254417.683201 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948120.44 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 94812.04 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349229.727343 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853308.40 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 85330.84 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434560.567072 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767977.56 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 76797.76 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511358.322827 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691179.80 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 69117.98 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580476.303008 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622061.82 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 62206.18 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642682.485170 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559855.64 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 55985.56 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698668.049115 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503870.08 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 50387.01 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749055.056667 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453483.07 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 45348.31 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794403.363463 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408134.76 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 40813.48 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835216.839579 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367321.29 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 36732.13 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871948.968084 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330589.16 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 33058.92 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1905007.883739 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297530.24 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 29753.02 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934760.907827 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267777.22 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 26777.72 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961538.629508 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240999.50 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 24099.95 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985638.579020 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216899.55 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 21689.95 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007328.533580 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195209.59 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 19520.96 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026849.492685 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175688.63 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 17568.86 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044418.355879 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158119.77 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 15811.98 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060230.332754 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142307.79 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 14230.78 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074461.111942 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128077.01 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 12807.70 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087268.813210 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115269.31 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 11526.93 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098795.744352 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103742.38 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 10374.24 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109169.982380 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93368.14 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 9336.81 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118506.796604 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84031.33 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 8403.13 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126909.929407 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75628.20 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 7562.82 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134472.748929 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68065.38 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 6806.54 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141279.286499 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61258.84 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 6125.88 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147405.170312 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55132.95 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 5513.30 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152918.465743 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49619.66 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 4961.97 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157880.431632 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44657.69 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 4465.77 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162346.200931 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40191.92 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 4019.19 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166365.393301 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36172.73 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3617.27 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169982.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32555.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173582.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28955.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177182.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25355.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180782.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21755.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184382.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18155.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187982.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14555.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191582.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10955.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195182.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7355.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198782.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3755.46 sec) 21:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202382.666434 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 155.46 sec) 21:45:00 [GPUGRID] [rr_sim] 2202538.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968629764.99G/439.78G) 21:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853879.04 21:45:00 [---] pick_jobs_to_run() start 21:45:00 [---] pick_jobs_to_run() end 21:45:00 [---] [rr_sim] end 21:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:45:00 [---] [cpu_sched_debug] preliminary job list: 21:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:45:00 [---] [cpu_sched_debug] final job list: 21:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:45:00 [---] [cpu_sched_debug] enforce_run_list: end 21:45:00 [---] choose_project(): 78300.000000 21:45:00 [WUProp@Home] doing RPC to get work 21:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357299.09 cpu 2106452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106452.90 21:45:00 [---] Got 0 tasks 21:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:46:00 [---] [rr_sim] doing sim: CPU sched 21:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968602174.63G/439.78G) 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202475.39 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 220247.54 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220247.538775 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982227.85 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 198222.78 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418470.323672 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1784005.06 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 178400.51 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596870.830079 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605604.56 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 160560.46 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757431.285846 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445044.10 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 144504.41 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901935.696036 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300539.69 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 130053.97 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031989.665207 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170485.72 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 117048.57 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149038.237461 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053437.15 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 105343.72 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254381.952489 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948093.44 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 94809.34 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349191.296015 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853284.09 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 85328.41 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434519.705188 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767955.68 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 76795.57 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511315.273444 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691160.11 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 69116.01 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580431.284874 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622044.10 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 62204.41 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642635.695161 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559839.69 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 55983.97 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698619.664420 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503855.72 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 50385.57 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1749005.236752 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453470.15 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 45347.02 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794352.251852 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408123.14 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 40812.31 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835164.565441 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367310.82 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 36731.08 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871895.647671 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330579.74 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 33057.97 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904953.621679 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297521.77 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 29752.18 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934705.798286 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267769.59 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 26776.96 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961482.757232 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240992.63 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 24099.26 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985582.020283 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216893.37 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 21689.34 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007271.357029 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195204.03 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 19520.40 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026791.760101 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175683.63 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 17568.36 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044360.122865 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158115.26 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 15811.53 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060171.649353 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142303.74 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 14230.37 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074402.023193 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128073.36 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 12807.34 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087209.359648 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115266.03 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 11526.60 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098735.962458 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103739.43 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 10373.94 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109109.904987 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93365.48 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 9336.55 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118446.453262 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84028.93 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 8402.89 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126849.346711 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75626.04 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 7562.60 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134411.950814 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68063.44 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 6806.34 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141218.294507 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61257.09 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 6125.71 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147344.003831 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55131.38 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 5513.14 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152857.142223 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49618.25 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 4961.82 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157818.966775 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44656.42 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 4465.64 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162284.608872 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40190.78 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 4019.08 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166303.686759 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36171.70 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3617.17 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169920.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32554.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173520.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28954.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177120.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25354.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180720.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21754.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184320.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18154.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187920.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14554.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191520.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10954.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195120.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7354.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198720.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3754.53 sec) 21:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202320.856858 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 154.53 sec) 21:46:00 [GPUGRID] [rr_sim] 2202475.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968602174.63G/439.78G) 21:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853876.30 21:46:00 [---] pick_jobs_to_run() start 21:46:00 [---] pick_jobs_to_run() end 21:46:00 [---] [rr_sim] end 21:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:46:00 [---] [cpu_sched_debug] preliminary job list: 21:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:46:00 [---] [cpu_sched_debug] final job list: 21:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:46:00 [---] [cpu_sched_debug] enforce_run_list: end 21:46:00 [---] choose_project(): 78360.000000 21:46:00 [WUProp@Home] doing RPC to get work 21:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357239.09 cpu 2106392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106392.90 21:46:00 [---] Got 0 tasks 21:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:47:00 [---] [rr_sim] doing sim: CPU sched 21:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968574584.28G/439.78G) 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202412.65 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 220241.27 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220241.265086 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982171.39 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 198217.14 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418458.403664 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783954.25 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 178395.42 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596853.828384 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605558.82 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 160555.88 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757409.710632 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1445002.94 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 144500.29 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901910.004655 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300502.65 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 130050.26 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031960.269276 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170452.38 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 117045.24 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1149005.507435 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053407.14 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 105340.71 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254346.221778 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948066.43 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 94806.64 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349152.864686 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853259.79 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 85325.98 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434478.843304 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767933.81 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 76793.38 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511272.224060 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691140.43 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 69114.04 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580386.266740 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622026.38 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 62202.64 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642588.905153 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559823.75 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 55982.37 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698571.279724 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503841.37 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 50384.14 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748955.416838 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453457.23 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 45345.72 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794301.140240 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408111.51 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 40811.15 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835112.291303 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367300.36 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 36730.04 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871842.327259 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330570.32 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 33057.03 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904899.359619 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297513.29 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 29751.33 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934650.688744 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267761.96 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 26776.20 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961426.884956 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240985.77 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 24098.58 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985525.461546 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216887.19 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 21688.72 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007214.180478 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195198.47 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 19519.85 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026734.027517 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175678.62 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 17567.86 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044301.889851 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158110.76 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 15811.08 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060112.965953 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142299.68 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 14229.97 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074342.934444 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128069.72 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 12806.97 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087149.906086 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115262.74 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 11526.27 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098676.180563 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103736.47 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 10373.65 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109049.827593 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93362.82 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 9336.28 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118386.109920 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84026.54 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 8402.65 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126788.764015 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75623.89 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 7562.39 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134351.152700 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68061.50 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 6806.15 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141157.302516 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61255.35 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 6125.53 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147282.837351 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55129.81 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 5512.98 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152795.818702 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49616.83 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 4961.68 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157757.501918 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44655.15 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 4465.51 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162223.016813 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40189.63 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 4018.96 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166241.980218 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36170.67 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3617.07 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169859.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32553.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173459.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28953.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177059.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25353.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180659.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21753.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184259.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18153.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187859.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14553.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191459.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10953.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195059.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7353.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198659.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3753.60 sec) 21:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202259.047282 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 153.60 sec) 21:47:00 [GPUGRID] [rr_sim] 2202412.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968574584.28G/439.78G) 21:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853873.56 21:47:00 [---] pick_jobs_to_run() start 21:47:00 [---] pick_jobs_to_run() end 21:47:00 [---] [rr_sim] end 21:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:47:00 [---] [cpu_sched_debug] preliminary job list: 21:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:47:00 [---] [cpu_sched_debug] final job list: 21:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:47:00 [---] [cpu_sched_debug] enforce_run_list: end 21:47:00 [---] choose_project(): 78420.000000 21:47:00 [WUProp@Home] doing RPC to get work 21:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357179.09 cpu 2106332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106332.90 21:47:00 [---] Got 0 tasks 21:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:48:00 [---] [rr_sim] doing sim: CPU sched 21:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968546993.92G/439.78G) 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202349.91 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 220234.99 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220234.991398 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982114.92 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 198211.49 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418446.483656 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783903.43 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 178390.34 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596836.826689 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605513.09 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 160551.31 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757388.135418 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444961.78 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 144496.18 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901884.313275 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300465.60 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 130046.56 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031930.873345 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170419.04 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 117041.90 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148972.777409 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053377.14 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 105337.71 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254310.491066 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948039.42 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 94803.94 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349114.433358 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853235.48 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 85323.55 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434437.981420 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767911.93 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 76791.19 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511229.174676 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691120.74 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 69112.07 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580341.248607 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 622008.67 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 62200.87 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642542.115144 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559807.80 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 55980.78 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698522.895028 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503827.02 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 50382.70 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748905.596923 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453444.32 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 45344.43 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794250.028629 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408099.89 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 40809.99 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835060.017164 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367289.90 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 36728.99 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871789.006846 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330560.91 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 33056.09 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904845.097560 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297504.82 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 29750.48 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934595.579202 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267754.33 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 26775.43 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961371.012680 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240978.90 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 24097.89 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985468.902810 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216881.01 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 21688.10 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007157.003927 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195192.91 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 19519.29 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026676.294932 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175673.62 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 17567.36 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044243.656837 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158106.26 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 15810.63 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2060054.282552 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142295.63 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 14229.56 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074283.845695 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128066.07 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 12806.61 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087090.452523 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115259.46 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 11525.95 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098616.398669 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103733.52 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 10373.35 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108989.750200 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93360.16 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 9336.02 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118325.766579 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84024.15 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 8402.41 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126728.181319 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75621.73 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 7562.17 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134290.354585 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68059.56 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 6805.96 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141096.310525 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61253.60 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 6125.36 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147221.670870 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55128.24 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 5512.82 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152734.495181 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49615.42 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 4961.54 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157696.037061 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44653.88 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 4465.39 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162161.424753 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40188.49 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 4018.85 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166180.273676 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36169.64 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3616.96 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169797.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32552.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173397.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28952.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176997.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25352.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180597.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21752.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184197.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18152.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187797.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14552.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191397.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10952.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194997.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7352.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198597.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3752.68 sec) 21:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202197.237707 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 152.68 sec) 21:48:00 [GPUGRID] [rr_sim] 2202349.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968546993.92G/439.78G) 21:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853870.83 21:48:00 [---] pick_jobs_to_run() start 21:48:00 [---] pick_jobs_to_run() end 21:48:00 [---] [rr_sim] end 21:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:48:00 [---] [cpu_sched_debug] preliminary job list: 21:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:48:00 [---] [cpu_sched_debug] final job list: 21:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:48:00 [---] [cpu_sched_debug] enforce_run_list: end 21:48:00 [---] choose_project(): 78480.000000 21:48:00 [WUProp@Home] doing RPC to get work 21:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357119.09 cpu 2106272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106272.90 21:48:00 [---] Got 0 tasks 21:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:49:00 [---] [rr_sim] doing sim: CPU sched 21:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968519403.56G/439.78G) 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202287.18 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 220228.72 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220228.717710 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982058.46 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 198205.85 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418434.563649 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783852.61 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 178385.26 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596819.824994 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605467.35 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 160546.74 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757366.560204 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444920.62 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 144492.06 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901858.621894 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300428.56 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 130042.86 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031901.477414 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170385.70 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 117038.57 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148940.047383 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053347.13 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 105334.71 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254274.760355 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 948012.42 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 94801.24 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349076.002029 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853211.18 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 85321.12 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434397.119536 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767890.06 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 76789.01 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511186.125292 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691101.05 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 69110.11 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580296.230473 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621990.95 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 62199.09 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642495.325136 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559791.85 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 55979.19 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698474.510332 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503812.67 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 50381.27 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748855.777009 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453431.40 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 45343.14 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794198.917018 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408088.26 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 40808.83 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1835007.743026 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367279.43 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 36727.94 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871735.686433 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330551.49 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 33055.15 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904790.835500 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297496.34 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 29749.63 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934540.469660 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267746.71 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 26774.67 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961315.140404 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240972.04 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 24097.20 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985412.344073 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216874.83 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 21687.48 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007099.827376 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195187.35 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 19518.73 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026618.562348 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175668.61 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 17566.86 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044185.423823 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158101.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 15810.18 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059995.599151 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142291.58 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 14229.16 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074224.756946 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128062.42 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 12806.24 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087030.998961 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115256.18 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 11525.62 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098556.616775 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103730.56 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 10373.06 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108929.672807 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93357.50 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 9335.75 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118265.423237 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84021.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 8402.18 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126667.598623 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75619.58 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 7561.96 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134229.556470 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68057.62 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 6805.76 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141035.318533 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61251.86 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 6125.19 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147160.504390 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55126.67 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 5512.67 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152673.171661 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49614.01 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 4961.40 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157634.572205 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44652.60 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 4465.26 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162099.832694 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40187.34 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 4018.73 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166118.567135 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36168.61 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3616.86 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169735.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32551.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173335.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28951.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176935.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25351.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180535.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21751.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184135.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18151.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187735.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14551.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191335.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10951.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194935.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7351.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198535.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3751.75 sec) 21:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202135.428131 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 151.75 sec) 21:49:00 [GPUGRID] [rr_sim] 2202287.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968519403.56G/439.78G) 21:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853868.09 21:49:00 [---] pick_jobs_to_run() start 21:49:00 [---] pick_jobs_to_run() end 21:49:00 [---] [rr_sim] end 21:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:49:00 [---] [cpu_sched_debug] preliminary job list: 21:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:49:00 [---] [cpu_sched_debug] final job list: 21:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:49:00 [---] [cpu_sched_debug] enforce_run_list: end 21:49:00 [---] choose_project(): 78540.000000 21:49:00 [WUProp@Home] doing RPC to get work 21:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 357059.09 cpu 2106212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106212.90 21:49:00 [---] Got 0 tasks 21:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:50:00 [---] [rr_sim] doing sim: CPU sched 21:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968491813.21G/439.78G) 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202224.44 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 220222.44 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220222.444022 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1982002.00 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 198200.20 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418422.643641 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783801.80 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 178380.18 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596802.823299 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605421.62 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 160542.16 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757344.984991 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444879.46 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 144487.95 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901832.930513 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300391.51 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 130039.15 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031872.081484 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170352.36 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 117035.24 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148907.317357 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053317.12 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 105331.71 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254239.029643 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947985.41 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 94798.54 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1349037.570700 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853186.87 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 85318.69 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434356.257652 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767868.18 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 76786.82 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511143.075909 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691081.36 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 69108.14 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580251.212339 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621973.23 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 62197.32 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642448.535127 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559775.91 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 55977.59 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698426.125636 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503798.31 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 50379.83 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748805.957094 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453418.48 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 45341.85 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794147.805407 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408076.63 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 40807.66 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834955.468888 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367268.97 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 36726.90 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871682.366021 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330542.07 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 33054.21 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904736.573440 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297487.87 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 29748.79 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934485.360118 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267739.08 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 26773.91 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961259.268128 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240965.17 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 24096.52 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985355.785337 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216868.65 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 21686.87 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2007042.650825 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195181.79 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 19518.18 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026560.829764 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175663.61 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 17566.36 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044127.190809 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158097.25 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 15809.72 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059936.915750 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142287.52 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 14228.75 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074165.668197 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128058.77 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 12805.88 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086971.545399 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115252.89 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 11525.29 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098496.834881 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103727.61 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 10372.76 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108869.595414 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93354.84 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 9335.48 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118205.079895 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84019.36 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 8401.94 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126607.015927 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75617.42 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 7561.74 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134168.758356 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68055.68 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 6805.57 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140974.326542 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61250.11 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 6125.01 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147099.337909 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55125.10 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 5512.51 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152611.848140 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49612.59 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 4961.26 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157573.107348 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44651.33 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 4465.13 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162038.240635 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40186.20 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 4018.62 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166056.860593 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36167.58 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3616.76 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169673.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32550.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173273.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28950.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176873.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25350.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180473.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21750.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184073.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18150.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187673.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14550.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191273.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10950.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194873.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7350.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198473.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3750.82 sec) 21:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202073.618555 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 150.82 sec) 21:50:00 [GPUGRID] [rr_sim] 2202224.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968491813.21G/439.78G) 21:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853865.35 21:50:00 [---] pick_jobs_to_run() start 21:50:00 [---] pick_jobs_to_run() end 21:50:00 [---] [rr_sim] end 21:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:50:00 [---] [cpu_sched_debug] preliminary job list: 21:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:50:00 [---] [cpu_sched_debug] final job list: 21:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:50:00 [---] [cpu_sched_debug] enforce_run_list: end 21:50:00 [---] choose_project(): 78600.000000 21:50:00 [WUProp@Home] doing RPC to get work 21:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356999.09 cpu 2106152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106152.90 21:50:00 [---] Got 0 tasks 21:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:51:00 [---] [rr_sim] doing sim: CPU sched 21:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968464222.85G/439.78G) 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202161.70 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 220216.17 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220216.170333 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981945.53 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 198194.55 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418410.723634 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783750.98 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 178375.10 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596785.821604 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605375.88 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 160537.59 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757323.409777 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444838.29 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 144483.83 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901807.239133 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300354.46 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 130035.45 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031842.685553 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170319.02 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 117031.90 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148874.587331 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053287.12 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 105328.71 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254203.298931 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947958.40 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 94795.84 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348999.139372 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853162.56 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 85316.26 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434315.395768 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767846.31 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 76784.63 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511100.026525 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691061.68 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 69106.17 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580206.194206 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621955.51 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 62195.55 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642401.745119 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559759.96 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 55976.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698377.740940 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503783.96 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 50378.40 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748756.137180 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453405.57 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 45340.56 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794096.693795 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408065.01 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 40806.50 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834903.194749 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367258.51 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 36725.85 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871629.045608 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330532.66 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 33053.27 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904682.311381 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297479.39 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 29747.94 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934430.250576 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267731.45 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 26773.15 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961203.395852 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240958.31 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 24095.83 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985299.226600 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216862.48 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 21686.25 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006985.474274 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195176.23 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 19517.62 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026503.097180 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175658.61 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 17565.86 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044068.957795 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158092.75 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 15809.27 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059878.232349 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142283.47 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 14228.35 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074106.579448 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128055.12 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 12805.51 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086912.091837 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115249.61 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 11524.96 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098437.052986 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103724.65 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 10372.47 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108809.518021 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93352.19 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 9335.22 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118144.736553 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84016.97 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 8401.70 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126546.433231 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75615.27 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 7561.53 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134107.960241 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68053.74 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 6805.37 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140913.334551 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61248.37 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 6124.84 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147038.171429 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55123.53 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 5512.35 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152550.524620 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49611.18 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 4961.12 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157511.642491 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44650.06 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 4465.01 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161976.648576 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40185.05 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 4018.51 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165995.154051 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36166.55 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3616.65 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169611.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32549.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173211.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28949.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176811.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25349.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180411.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21749.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184011.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18149.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187611.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14549.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191211.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10949.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194811.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7349.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198411.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3749.89 sec) 21:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2202011.808980 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 149.89 sec) 21:51:00 [GPUGRID] [rr_sim] 2202161.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968464222.85G/439.78G) 21:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853862.62 21:51:00 [---] pick_jobs_to_run() start 21:51:00 [---] pick_jobs_to_run() end 21:51:00 [---] [rr_sim] end 21:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:51:00 [---] [cpu_sched_debug] preliminary job list: 21:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:51:00 [---] [cpu_sched_debug] final job list: 21:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:51:00 [---] [cpu_sched_debug] enforce_run_list: end 21:51:00 [---] choose_project(): 78660.000000 21:51:00 [WUProp@Home] doing RPC to get work 21:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356939.09 cpu 2106092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106092.90 21:51:00 [---] Got 0 tasks 21:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:52:00 [---] [rr_sim] doing sim: CPU sched 21:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968436632.49G/439.78G) 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202098.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 220209.90 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220209.896645 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981889.07 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 198188.91 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418398.803626 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783700.16 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 178370.02 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596768.819909 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605330.15 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 160533.01 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757301.834563 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444797.13 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 144479.71 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901781.547752 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300317.42 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 130031.74 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031813.289622 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170285.68 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 117028.57 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148841.857305 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053257.11 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 105325.71 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254167.568220 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947931.40 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 94793.14 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348960.708043 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853138.26 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 85313.83 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434274.533884 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767824.43 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 76782.44 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511056.977141 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691041.99 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 69104.20 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580161.176072 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621937.79 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 62193.78 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642354.955110 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559744.01 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 55974.40 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698329.356245 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503769.61 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 50376.96 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748706.317265 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453392.65 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 45339.26 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1794045.582184 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408053.38 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 40805.34 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834850.920611 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367248.05 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 36724.80 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871575.725195 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330523.24 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 33052.32 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904628.049321 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297470.92 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 29747.09 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934375.141034 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267723.83 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 26772.38 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961147.523576 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240951.44 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 24095.14 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985242.667864 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216856.30 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 21685.63 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006928.297723 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195170.67 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 19517.07 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026445.364596 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175653.60 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 17565.36 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2044010.724781 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158088.24 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 15808.82 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059819.548948 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142279.42 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 14227.94 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2074047.490699 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128051.48 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 12805.15 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086852.638274 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115246.33 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 11524.63 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098377.271092 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103721.70 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 10372.17 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108749.440628 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93349.53 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 9334.95 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118084.393211 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84014.57 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 8401.46 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126485.850535 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75613.12 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 7561.31 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134047.162127 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68051.80 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 6805.18 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140852.342559 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61246.62 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 6124.66 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146977.004949 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55121.96 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 5512.20 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152489.201099 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49609.77 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 4960.98 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157450.177634 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44648.79 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 4464.88 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161915.056516 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40183.91 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 4018.39 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165933.447510 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36165.52 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3616.55 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169549.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32548.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173149.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28948.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176749.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25348.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180349.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21748.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183949.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18148.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187549.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14548.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191149.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10948.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194749.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7348.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198349.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3748.97 sec) 21:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201949.999404 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 148.97 sec) 21:52:00 [GPUGRID] [rr_sim] 2202098.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968436632.49G/439.78G) 21:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853859.88 21:52:00 [---] pick_jobs_to_run() start 21:52:00 [---] pick_jobs_to_run() end 21:52:00 [---] [rr_sim] end 21:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:52:00 [---] [cpu_sched_debug] preliminary job list: 21:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:52:00 [---] [cpu_sched_debug] final job list: 21:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:52:00 [---] [cpu_sched_debug] enforce_run_list: end 21:52:00 [---] choose_project(): 78720.000000 21:52:00 [WUProp@Home] doing RPC to get work 21:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356879.09 cpu 2106032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2106032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2106032.90 21:52:00 [---] Got 0 tasks 21:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:53:00 [---] [rr_sim] doing sim: CPU sched 21:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968409042.14G/439.78G) 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2202036.23 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 220203.62 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220203.622957 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981832.61 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 198183.26 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418386.883618 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783649.35 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 178364.93 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596751.818214 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605284.41 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 160528.44 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757280.259349 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444755.97 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 144475.60 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901755.856371 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300280.37 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 130028.04 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031783.893691 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170252.34 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 117025.23 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148809.127279 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053227.10 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 105322.71 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254131.837508 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947904.39 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 94790.44 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348922.276715 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853113.95 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 85311.40 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434233.672000 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767802.56 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 76780.26 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1511013.927757 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691022.30 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 69102.23 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580116.157939 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621920.07 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 62192.01 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642308.165102 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559728.06 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 55972.81 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698280.971549 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503755.26 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 50375.53 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748656.497351 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453379.73 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 45337.97 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793994.470573 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408041.76 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 40804.18 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834798.646473 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367237.58 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 36723.76 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871522.404782 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330513.82 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 33051.38 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904573.787261 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297462.44 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 29746.24 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934320.031492 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267716.20 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 26771.62 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961091.651300 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240944.58 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 24094.46 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985186.109127 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216850.12 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 21685.01 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006871.121171 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195165.11 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 19516.51 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026387.632011 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175648.60 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 17564.86 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043952.491767 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158083.74 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 15808.37 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059760.865548 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142275.36 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 14227.54 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073988.401950 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128047.83 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 12804.78 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086793.184712 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115243.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 11524.30 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098317.489198 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103718.74 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 10371.87 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108689.363235 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93346.87 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 9334.69 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118024.049869 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84012.18 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 8401.22 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126425.267839 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75610.96 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 7561.10 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133986.364012 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68049.87 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 6804.99 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140791.350568 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61244.88 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 6124.49 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146915.838468 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55120.39 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 5512.04 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152427.877578 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49608.35 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 4960.84 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157388.712778 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44647.52 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 4464.75 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161853.464457 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40182.77 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 4018.28 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165871.740968 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36164.49 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3616.45 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169488.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32548.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173088.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28948.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176688.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25348.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180288.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21748.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183888.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18148.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187488.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14548.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191088.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10948.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194688.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7348.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198288.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3748.04 sec) 21:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201888.189829 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 148.04 sec) 21:53:00 [GPUGRID] [rr_sim] 2202036.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968409042.14G/439.78G) 21:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853857.14 21:53:00 [---] pick_jobs_to_run() start 21:53:00 [---] pick_jobs_to_run() end 21:53:00 [---] [rr_sim] end 21:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:53:00 [---] [cpu_sched_debug] preliminary job list: 21:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:53:00 [---] [cpu_sched_debug] final job list: 21:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:53:00 [---] [cpu_sched_debug] enforce_run_list: end 21:53:00 [---] choose_project(): 78780.000000 21:53:00 [WUProp@Home] doing RPC to get work 21:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356819.09 cpu 2105972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105972.90 21:53:00 [---] Got 0 tasks 21:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:54:00 [---] [rr_sim] doing sim: CPU sched 21:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968381451.78G/439.78G) 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201973.49 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 220197.35 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220197.349269 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981776.14 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 198177.61 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418374.963611 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783598.53 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 178359.85 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596734.816519 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605238.68 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 160523.87 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757258.684136 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444714.81 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 144471.48 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901730.164991 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300243.33 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 130024.33 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031754.497761 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170218.99 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 117021.90 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148776.397253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053197.10 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 105319.71 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254096.106797 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947877.39 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 94787.74 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348883.845386 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853089.65 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 85308.96 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434192.810116 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767780.68 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 76778.07 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510970.878373 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 691002.61 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 69100.26 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580071.139805 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621902.35 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 62190.24 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642261.375093 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559712.12 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 55971.21 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698232.586853 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503740.91 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 50374.09 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748606.677436 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453366.82 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 45336.68 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793943.358962 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408030.13 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 40803.01 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834746.372334 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367227.12 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 36722.71 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871469.084370 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330504.41 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 33050.44 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904519.525202 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297453.97 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 29745.40 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934264.921950 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267708.57 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 26770.86 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1961035.779024 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240937.71 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 24093.77 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985129.550391 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216843.94 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 21684.39 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006813.944620 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195159.55 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 19515.95 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026329.899427 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175643.59 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 17564.36 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043894.258753 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158079.23 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 15807.92 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059702.182147 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142271.31 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 14227.13 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073929.313201 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128044.18 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 12804.42 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086733.731150 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115239.76 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 11523.98 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098257.707304 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103715.79 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 10371.58 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108629.285842 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93344.21 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 9334.42 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117963.706527 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84009.79 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 8400.98 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126364.685143 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75608.81 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 7560.88 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133925.565897 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68047.93 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 6804.79 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140730.358577 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61243.13 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 6124.31 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146854.671988 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55118.82 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 5511.88 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152366.554058 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49606.94 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 4960.69 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157327.247921 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44646.24 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 4464.62 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161791.872398 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40181.62 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 4018.16 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165810.034427 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36163.46 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3616.35 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169426.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32547.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173026.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28947.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176626.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25347.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180226.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21747.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183826.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18147.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187426.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14547.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191026.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10947.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194626.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7347.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198226.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3747.11 sec) 21:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201826.380253 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 147.11 sec) 21:54:00 [GPUGRID] [rr_sim] 2201973.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968381451.78G/439.78G) 21:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853854.41 21:54:00 [---] pick_jobs_to_run() start 21:54:00 [---] pick_jobs_to_run() end 21:54:00 [---] [rr_sim] end 21:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:54:00 [---] [cpu_sched_debug] preliminary job list: 21:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:54:00 [---] [cpu_sched_debug] final job list: 21:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:54:00 [---] [cpu_sched_debug] enforce_run_list: end 21:54:00 [---] choose_project(): 78840.000000 21:54:00 [WUProp@Home] doing RPC to get work 21:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356759.09 cpu 2105912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105912.90 21:54:00 [---] Got 0 tasks 21:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:55:00 [---] [rr_sim] doing sim: CPU sched 21:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968353861.43G/439.78G) 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201910.76 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 220191.08 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220191.075581 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981719.68 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 198171.97 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418363.043603 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783547.71 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 178354.77 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596717.814823 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605192.94 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 160519.29 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757237.108922 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444673.65 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 144467.36 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901704.473610 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300206.28 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 130020.63 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031725.101830 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170185.65 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 117018.57 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148743.667227 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053167.09 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 105316.71 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254060.376085 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947850.38 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 94785.04 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348845.414057 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853065.34 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 85306.53 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434151.948232 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767758.81 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 76775.88 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510927.828990 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690982.93 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 69098.29 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1580026.121671 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621884.63 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 62188.46 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642214.585085 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559696.17 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 55969.62 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698184.202157 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503726.55 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 50372.66 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748556.857522 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453353.90 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 45335.39 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793892.247350 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408018.51 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 40801.85 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834694.098196 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367216.66 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 36721.67 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871415.763957 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330494.99 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 33049.50 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904465.263142 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297445.49 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 29744.55 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934209.812408 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267700.94 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 26770.09 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960979.906748 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240930.85 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 24093.08 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985072.991654 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216837.76 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 21683.78 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006756.768069 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195153.99 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 19515.40 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026272.166843 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175638.59 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 17563.86 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043836.025739 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158074.73 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 15807.47 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059643.498746 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142267.26 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 14226.73 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073870.224452 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128040.53 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 12804.05 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086674.277587 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115236.48 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 11523.65 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098197.925409 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103712.83 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 10371.28 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108569.208449 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93341.55 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 9334.15 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117903.363185 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84007.39 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 8400.74 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126304.102447 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75606.65 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 7560.67 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133864.767783 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68045.99 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 6804.60 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140669.366585 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61241.39 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 6124.14 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146793.505507 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55117.25 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 5511.73 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152305.230537 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49605.53 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 4960.55 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157265.783064 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44644.97 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 4464.50 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161730.280338 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40180.48 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 4018.05 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165748.327885 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36162.43 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3616.24 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169364.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32546.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172964.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28946.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176564.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25346.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180164.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21746.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183764.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18146.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187364.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14546.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190964.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10946.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194564.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7346.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198164.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3746.19 sec) 21:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201764.570677 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 146.19 sec) 21:55:00 [GPUGRID] [rr_sim] 2201910.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968353861.43G/439.78G) 21:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853851.67 21:55:00 [---] pick_jobs_to_run() start 21:55:00 [---] pick_jobs_to_run() end 21:55:00 [---] [rr_sim] end 21:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:55:00 [---] [cpu_sched_debug] preliminary job list: 21:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:55:00 [---] [cpu_sched_debug] final job list: 21:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:55:00 [---] [cpu_sched_debug] enforce_run_list: end 21:55:00 [---] choose_project(): 78900.000000 21:55:00 [WUProp@Home] doing RPC to get work 21:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356699.09 cpu 2105852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105852.90 21:55:00 [---] Got 0 tasks 21:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:56:00 [---] [rr_sim] doing sim: CPU sched 21:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968326271.07G/439.78G) 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201848.02 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 220184.80 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220184.801892 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981663.22 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 198166.32 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418351.123596 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783496.90 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 178349.69 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596700.813128 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605147.21 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 160514.72 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757215.533708 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444632.49 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 144463.25 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901678.782230 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300169.24 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 130016.92 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031695.705899 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170152.31 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 117015.23 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148710.937202 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053137.08 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 105313.71 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1254024.645374 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947823.37 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 94782.34 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348806.982729 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853041.04 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 85304.10 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434111.086348 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767736.93 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 76773.69 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510884.779606 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690963.24 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 69096.32 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579981.103538 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621866.92 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 62186.69 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642167.795076 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559680.22 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 55968.02 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698135.817461 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503712.20 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 50371.22 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748507.037607 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453340.98 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 45334.10 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793841.135739 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 408006.88 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 40800.69 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834641.824058 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367206.19 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 36720.62 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871362.443544 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330485.58 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 33048.56 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904411.001082 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297437.02 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 29743.70 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934154.702866 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267693.32 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 26769.33 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960924.034472 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240923.98 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 24092.40 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1985016.432917 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216831.59 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 21683.16 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006699.591518 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195148.43 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 19514.84 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026214.434259 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175633.58 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 17563.36 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043777.792725 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158070.23 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 15807.02 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059584.815345 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142263.20 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 14226.32 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073811.135703 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128036.88 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 12803.69 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086614.824025 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115233.19 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 11523.32 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098138.143515 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103709.88 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 10370.99 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108509.131056 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93338.89 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 9333.89 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117843.019843 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84005.00 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 8400.50 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126243.519751 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75604.50 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 7560.45 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133803.969668 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68044.05 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 6804.40 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140608.374594 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61239.64 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 6123.96 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146732.339027 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55115.68 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 5511.57 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152243.907017 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49604.11 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 4960.41 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157204.318207 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44643.70 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 4464.37 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161668.688279 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40179.33 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 4017.93 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165686.621343 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36161.40 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3616.14 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169302.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32545.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172902.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28945.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176502.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25345.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180102.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21745.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183702.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18145.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187302.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14545.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190902.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10945.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194502.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7345.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198102.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3745.26 sec) 21:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201702.761102 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 145.26 sec) 21:56:00 [GPUGRID] [rr_sim] 2201848.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968326271.07G/439.78G) 21:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853848.93 21:56:00 [---] pick_jobs_to_run() start 21:56:00 [---] pick_jobs_to_run() end 21:56:00 [---] [rr_sim] end 21:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:56:00 [---] [cpu_sched_debug] preliminary job list: 21:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:56:00 [---] [cpu_sched_debug] final job list: 21:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:56:00 [---] [cpu_sched_debug] enforce_run_list: end 21:56:00 [---] choose_project(): 78960.000000 21:56:00 [WUProp@Home] doing RPC to get work 21:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356639.09 cpu 2105792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105792.90 21:56:00 [---] Got 0 tasks 21:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:57:00 [---] [rr_sim] doing sim: CPU sched 21:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968298680.71G/439.78G) 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201785.28 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 220178.53 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220178.528204 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981606.75 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 198160.68 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418339.203588 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783446.08 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 178344.61 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596683.811433 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605101.47 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 160510.15 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757193.958494 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444591.32 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 144459.13 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901653.090849 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300132.19 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 130013.22 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031666.309968 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170118.97 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 117011.90 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148678.207176 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053107.07 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 105310.71 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253988.914662 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947796.37 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 94779.64 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348768.551400 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 853016.73 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 85301.67 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434070.224464 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767715.06 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 76771.51 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510841.730222 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690943.55 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 69094.36 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579936.085404 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621849.20 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 62184.92 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642121.005068 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559664.28 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 55966.43 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698087.432765 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503697.85 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 50369.78 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748457.217693 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453328.06 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 45332.81 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793790.024128 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407995.26 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 40799.53 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834589.549919 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367195.73 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 36719.57 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871309.123132 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330476.16 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 33047.62 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904356.739023 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297428.54 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 29742.85 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934099.593325 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267685.69 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 26768.57 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960868.162196 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240917.12 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 24091.71 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984959.874181 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216825.41 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 21682.54 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006642.414967 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195142.87 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 19514.29 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026156.701674 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175628.58 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 17562.86 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043719.559711 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158065.72 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 15806.57 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059526.131944 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142259.15 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 14225.92 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073752.046954 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128033.24 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 12803.32 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086555.370463 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115229.91 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 11522.99 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098078.361621 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103706.92 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 10370.69 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108449.053663 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93336.23 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 9333.62 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117782.676501 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84002.61 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 8400.26 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126182.937055 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75602.34 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 7560.23 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133743.171554 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68042.11 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 6804.21 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140547.382602 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61237.90 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 6123.79 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146671.172546 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55114.11 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 5511.41 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152182.583496 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49602.70 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 4960.27 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157142.853351 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44642.43 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 4464.24 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161607.096220 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40178.19 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 4017.82 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165624.914802 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36160.37 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3616.04 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169240.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32544.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172840.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28944.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176440.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25344.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180040.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21744.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183640.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18144.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187240.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14544.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190840.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10944.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194440.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7344.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198040.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3744.33 sec) 21:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201640.951526 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 144.33 sec) 21:57:00 [GPUGRID] [rr_sim] 2201785.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968298680.71G/439.78G) 21:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853846.19 21:57:00 [---] pick_jobs_to_run() start 21:57:00 [---] pick_jobs_to_run() end 21:57:00 [---] [rr_sim] end 21:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:57:00 [---] [cpu_sched_debug] preliminary job list: 21:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:57:00 [---] [cpu_sched_debug] final job list: 21:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:57:00 [---] [cpu_sched_debug] enforce_run_list: end 21:57:00 [---] choose_project(): 79020.000000 21:57:00 [WUProp@Home] doing RPC to get work 21:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356579.09 cpu 2105732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105732.90 21:57:00 [---] Got 0 tasks 21:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:58:00 [---] [rr_sim] doing sim: CPU sched 21:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968271090.36G/439.78G) 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201722.55 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 220172.25 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220172.254516 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981550.29 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 198155.03 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418327.283580 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783395.26 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 178339.53 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596666.809738 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605055.74 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 160505.57 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757172.383280 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444550.16 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 144455.02 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901627.399468 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300095.15 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 130009.51 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031636.914038 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170085.63 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 117008.56 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148645.477150 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053077.07 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 105307.71 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253953.183951 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947769.36 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 94776.94 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348730.120072 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852992.43 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 85299.24 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1434029.362580 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767693.18 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 76769.32 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510798.680838 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690923.86 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 69092.39 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579891.067271 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621831.48 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 62183.15 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642074.215059 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559648.33 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 55964.83 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1698039.048069 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503683.50 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 50368.35 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748407.397779 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453315.15 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 45331.51 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793738.912517 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407983.63 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 40798.36 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834537.275781 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367185.27 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 36718.53 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871255.802719 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330466.74 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 33046.67 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904302.476963 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297420.07 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 29742.01 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1934044.483783 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267678.06 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 26767.81 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960812.289920 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240910.26 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 24091.03 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984903.315444 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216819.23 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 21681.92 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006585.238416 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195137.31 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 19513.73 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026098.969090 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175623.58 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 17562.36 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043661.326697 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158061.22 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 15806.12 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059467.448543 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142255.10 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 14225.51 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073692.958205 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128029.59 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 12802.96 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086495.916901 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115226.63 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 11522.66 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098018.579726 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103703.97 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 10370.40 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108388.976270 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93333.57 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 9333.36 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117722.333159 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84000.21 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 8400.02 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126122.354359 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75600.19 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 7560.02 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133682.373439 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68040.17 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 6804.02 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140486.390611 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61236.15 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 6123.62 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146610.006066 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55112.54 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 5511.25 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152121.259975 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49601.29 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 4960.13 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157081.388494 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44641.16 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 4464.12 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161545.504160 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40177.04 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 4017.70 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165563.208260 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36159.34 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3615.93 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169179.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32543.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172779.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28943.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176379.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25343.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179979.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21743.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183579.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18143.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187179.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14543.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190779.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10943.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194379.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7343.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197979.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3743.40 sec) 21:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201579.141950 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 143.40 sec) 21:58:00 [GPUGRID] [rr_sim] 2201722.55: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968271090.36G/439.78G) 21:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853843.46 21:58:00 [---] pick_jobs_to_run() start 21:58:00 [---] pick_jobs_to_run() end 21:58:00 [---] [rr_sim] end 21:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:58:00 [---] [cpu_sched_debug] preliminary job list: 21:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:58:00 [---] [cpu_sched_debug] final job list: 21:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:58:00 [---] [cpu_sched_debug] enforce_run_list: end 21:58:00 [---] choose_project(): 79080.000000 21:58:00 [WUProp@Home] doing RPC to get work 21:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356519.09 cpu 2105672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105672.90 21:58:00 [---] Got 0 tasks 21:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 21:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 21:59:00 [---] [rr_sim] doing sim: CPU sched 21:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968243500.00G/439.78G) 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201659.81 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 220165.98 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220165.980828 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981493.83 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 198149.38 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418315.363573 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783344.44 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 178334.44 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596649.808043 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1605010.00 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 160501.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757150.808067 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444509.00 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 144450.90 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901601.708088 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300058.10 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 130005.81 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031607.518107 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170052.29 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 117005.23 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148612.747124 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053047.06 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 105304.71 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253917.453239 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947742.36 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 94774.24 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348691.688743 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852968.12 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 85296.81 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433988.500697 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767671.31 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 76767.13 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510755.631455 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690904.18 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 69090.42 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579846.049137 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621813.76 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 62181.38 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1642027.425051 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559632.38 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 55963.24 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697990.663374 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503669.14 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 50366.91 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748357.577864 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453302.23 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 45330.22 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793687.800905 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407972.01 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 40797.20 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834485.001643 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367174.81 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 36717.48 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871202.482306 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330457.33 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 33045.73 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904248.214903 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297411.59 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 29741.16 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933989.374241 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267670.43 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 26767.04 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960756.417644 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240903.39 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 24090.34 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984846.756708 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216813.05 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 21681.31 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006528.061865 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195131.75 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 19513.17 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2026041.236506 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175618.57 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 17561.86 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043603.093683 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158056.71 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 15805.67 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059408.765143 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142251.04 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 14225.10 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073633.869456 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128025.94 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 12802.59 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086436.463338 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115223.34 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 11522.33 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097958.797832 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103701.01 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 10370.10 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108328.898877 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93330.91 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 9333.09 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117661.989817 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83997.82 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 8399.78 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126061.771663 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75598.04 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 7559.80 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133621.575324 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68038.23 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 6803.82 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140425.398620 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61234.41 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 6123.44 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146548.839585 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55110.97 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 5511.10 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152059.936455 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49599.87 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 4959.99 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157019.923637 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44639.88 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 4463.99 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161483.912101 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40175.90 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 4017.59 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165501.501719 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36158.31 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3615.83 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169117.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32542.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172717.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28942.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176317.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25342.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179917.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21742.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183517.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18142.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187117.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14542.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190717.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10942.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194317.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7342.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197917.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3742.48 sec) 21:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201517.332375 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142.48 sec) 21:59:00 [GPUGRID] [rr_sim] 2201659.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968243500.00G/439.78G) 21:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853840.72 21:59:00 [---] pick_jobs_to_run() start 21:59:00 [---] pick_jobs_to_run() end 21:59:00 [---] [rr_sim] end 21:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 21:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 21:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 21:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 21:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 21:59:00 [---] [cpu_sched_debug] preliminary job list: 21:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:59:00 [---] [cpu_sched_debug] final job list: 21:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 21:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 21:59:00 [---] [cpu_sched_debug] enforce_run_list: end 21:59:00 [---] choose_project(): 79140.000000 21:59:00 [WUProp@Home] doing RPC to get work 21:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356459.09 cpu 2105612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105612.90 21:59:00 [---] Got 0 tasks 22:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:00:00 [---] [rr_sim] doing sim: CPU sched 22:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968215909.64G/439.78G) 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201597.07 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 220159.71 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220159.707140 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981437.36 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 198143.74 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418303.443565 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783293.63 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 178329.36 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596632.806348 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604964.27 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 160496.43 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757129.232853 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444467.84 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 144446.78 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901576.016707 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1300021.05 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 130002.11 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031578.122176 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1170018.95 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 117001.89 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148580.017098 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1053017.05 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 105301.71 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253881.722528 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947715.35 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 94771.53 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348653.257414 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852943.81 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 85294.38 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433947.638813 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767649.43 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 76764.94 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510712.582071 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690884.49 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 69088.45 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579801.031003 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621796.04 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 62179.60 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641980.635043 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559616.44 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 55961.64 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697942.278678 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503654.79 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 50365.48 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748307.757950 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453289.31 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 45328.93 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793636.689294 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407960.38 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 40796.04 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834432.727504 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367164.34 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 36716.43 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871149.161893 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330447.91 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 33044.79 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904193.952844 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297403.12 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 29740.31 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933934.264699 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267662.81 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 26766.28 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960700.545368 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240896.53 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 24089.65 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984790.197971 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216806.87 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 21680.69 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006470.885314 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195126.19 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 19512.62 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025983.503922 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175613.57 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 17561.36 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043544.860669 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158052.21 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 15805.22 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059350.081742 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142246.99 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 14224.70 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073574.780707 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128022.29 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 12802.23 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086377.009776 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115220.06 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 11522.01 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097899.015938 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103698.06 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 10369.81 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108268.821484 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93328.25 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 9332.82 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117601.646475 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83995.42 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 8399.54 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126001.188967 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75595.88 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 7559.59 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133560.777210 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68036.29 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 6803.63 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140364.406628 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61232.66 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 6123.27 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146487.673105 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55109.40 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 5510.94 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151998.612934 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49598.46 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 4959.85 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156958.458780 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44638.61 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 4463.86 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161422.320042 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40174.75 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 4017.48 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165439.795177 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36157.28 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3615.73 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169055.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32541.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172655.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28941.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176255.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25341.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179855.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21741.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183455.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18141.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187055.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14541.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190655.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10941.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194255.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7341.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197855.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3741.55 sec) 22:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201455.522799 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141.55 sec) 22:00:00 [GPUGRID] [rr_sim] 2201597.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968215909.64G/439.78G) 22:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853837.98 22:00:00 [---] pick_jobs_to_run() start 22:00:00 [---] pick_jobs_to_run() end 22:00:00 [---] [rr_sim] end 22:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:00:00 [---] [cpu_sched_debug] preliminary job list: 22:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:00:00 [---] [cpu_sched_debug] final job list: 22:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:00:00 [---] [cpu_sched_debug] enforce_run_list: end 22:00:00 [---] choose_project(): 79200.000000 22:00:00 [WUProp@Home] doing RPC to get work 22:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356399.09 cpu 2105552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105552.90 22:00:00 [---] Got 0 tasks 22:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:01:00 [---] [rr_sim] doing sim: CPU sched 22:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968188319.29G/439.78G) 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201534.33 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 220153.43 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220153.433451 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981380.90 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 198138.09 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418291.523558 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783242.81 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 178324.28 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596615.804653 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604918.53 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 160491.85 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757107.657639 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444426.68 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 144442.67 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901550.325327 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299984.01 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 129998.40 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031548.726245 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169985.61 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 116998.56 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148547.287072 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052987.05 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 105298.70 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253845.991816 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947688.34 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 94768.83 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348614.826086 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852919.51 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 85291.95 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433906.776929 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767627.56 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 76762.76 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510669.532687 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690864.80 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 69086.48 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579756.012870 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621778.32 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 62177.83 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641933.845034 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559600.49 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 55960.05 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697893.893982 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503640.44 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 50364.04 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748257.938035 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453276.40 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 45327.64 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793585.577683 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407948.76 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 40794.88 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834380.453366 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367153.88 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 36715.39 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871095.841481 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330438.49 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 33043.85 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904139.690784 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297394.64 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 29739.46 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933879.155157 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267655.18 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 26765.52 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960644.673093 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240889.66 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 24088.97 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984733.639235 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216800.70 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 21680.07 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006413.708762 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195120.63 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 19512.06 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025925.771338 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175608.56 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 17560.86 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043486.627655 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158047.71 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 15804.77 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059291.398341 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142242.94 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 14224.29 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073515.691958 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128018.64 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 12801.86 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086317.556214 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115216.78 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 11521.68 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097839.234044 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103695.10 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 10369.51 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108208.744091 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93325.59 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 9332.56 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117541.303133 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83993.03 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 8399.30 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125940.606271 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75593.73 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 7559.37 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133499.979095 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68034.36 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 6803.44 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140303.414637 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61230.92 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 6123.09 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146426.506625 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55107.83 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 5510.78 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151937.289413 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49597.05 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 4959.70 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156896.993923 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44637.34 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 4463.73 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161360.727982 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40173.61 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 4017.36 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165378.088636 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36156.25 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3615.62 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168993.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32540.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172593.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28940.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176193.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25340.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179793.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21740.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183393.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18140.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186993.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14540.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190593.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10940.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194193.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7340.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197793.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3740.62 sec) 22:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201393.713223 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 140.62 sec) 22:01:00 [GPUGRID] [rr_sim] 2201534.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968188319.29G/439.78G) 22:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853835.25 22:01:00 [---] pick_jobs_to_run() start 22:01:00 [---] pick_jobs_to_run() end 22:01:00 [---] [rr_sim] end 22:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:01:00 [---] [cpu_sched_debug] preliminary job list: 22:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:01:00 [---] [cpu_sched_debug] final job list: 22:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:01:00 [---] [cpu_sched_debug] enforce_run_list: end 22:01:00 [---] choose_project(): 79260.000000 22:01:00 [WUProp@Home] doing RPC to get work 22:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356339.09 cpu 2105492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105492.90 22:01:00 [---] Got 0 tasks 22:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:02:00 [---] [rr_sim] doing sim: CPU sched 22:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968160728.93G/439.78G) 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201471.60 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 220147.16 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220147.159763 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981324.44 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 198132.44 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418279.603550 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783191.99 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 178319.20 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596598.802958 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604872.79 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 160487.28 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757086.082425 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444385.52 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 144438.55 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901524.633946 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299946.96 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 129994.70 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031519.330314 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169952.27 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 116995.23 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148514.557046 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052957.04 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 105295.70 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253810.261105 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947661.34 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 94766.13 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348576.394757 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852895.20 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 85289.52 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433865.915045 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767605.68 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 76760.57 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510626.483303 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690845.11 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 69084.51 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579710.994736 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621760.60 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 62176.06 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641887.055026 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559584.54 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 55958.45 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697845.509286 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503626.09 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 50362.61 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748208.118121 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453263.48 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 45326.35 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793534.466072 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407937.13 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 40793.71 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834328.179228 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367143.42 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 36714.34 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1871042.521068 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330429.08 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 33042.91 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904085.428724 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297386.17 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 29738.62 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933824.045615 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267647.55 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 26764.76 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960588.800817 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240882.80 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 24088.28 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984677.080498 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216794.52 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 21679.45 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006356.532211 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195115.07 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 19511.51 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025868.038753 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175603.56 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 17560.36 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043428.394641 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158043.20 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 15804.32 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059232.714940 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142238.88 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 14223.89 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073456.603209 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128014.99 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 12801.50 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086258.102651 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115213.49 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 11521.35 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097779.452149 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103692.15 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 10369.21 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108148.666698 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93322.93 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 9332.29 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117480.959791 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83990.64 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 8399.06 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125880.023575 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75591.57 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 7559.16 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133439.180981 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68032.42 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 6803.24 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140242.422646 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61229.17 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 6122.92 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146365.340144 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55106.26 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 5510.63 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151875.965893 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49595.63 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 4959.56 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156835.529067 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44636.07 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 4463.61 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161299.135923 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40172.46 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 4017.25 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165316.382094 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36155.22 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3615.52 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168931.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32539.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172531.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28939.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176131.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25339.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179731.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21739.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183331.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18139.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186931.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14539.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190531.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10939.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194131.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7339.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197731.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3739.69 sec) 22:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201331.903648 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 139.69 sec) 22:02:00 [GPUGRID] [rr_sim] 2201471.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968160728.93G/439.78G) 22:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853832.51 22:02:00 [---] pick_jobs_to_run() start 22:02:00 [---] pick_jobs_to_run() end 22:02:00 [---] [rr_sim] end 22:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:02:00 [---] [cpu_sched_debug] preliminary job list: 22:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:02:00 [---] [cpu_sched_debug] final job list: 22:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:02:00 [---] [cpu_sched_debug] enforce_run_list: end 22:02:00 [---] choose_project(): 79320.000000 22:02:00 [WUProp@Home] doing RPC to get work 22:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356279.09 cpu 2105432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105432.90 22:02:00 [---] Got 0 tasks 22:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:03:00 [---] [rr_sim] doing sim: CPU sched 22:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968133138.58G/439.78G) 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201408.86 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 220140.89 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220140.886075 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981267.97 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 198126.80 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418267.683542 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783141.18 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 178314.12 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596581.801263 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604827.06 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 160482.71 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757064.507212 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444344.35 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 144434.44 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901498.942565 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299909.92 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 129990.99 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031489.934384 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169918.93 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 116991.89 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148481.827020 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052927.03 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 105292.70 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253774.530393 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947634.33 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 94763.43 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348537.963429 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852870.90 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 85287.09 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433825.053161 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767583.81 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 76758.38 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510583.433920 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690825.43 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 69082.54 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579665.976602 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621742.88 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 62174.29 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641840.265017 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559568.60 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 55956.86 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697797.124590 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503611.74 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 50361.17 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748158.298206 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453250.56 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 45325.06 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793483.354460 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407925.51 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 40792.55 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834275.905089 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367132.96 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 36713.30 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870989.200655 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330419.66 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 33041.97 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1904031.166665 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297377.69 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 29737.77 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933768.936073 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267639.92 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 26763.99 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960532.928541 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240875.93 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 24087.59 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984620.521761 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216788.34 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 21678.83 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006299.355660 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195109.51 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 19510.95 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025810.306169 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175598.55 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 17559.86 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043370.161627 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158038.70 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 15803.87 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059174.031539 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142234.83 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 14223.48 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073397.514460 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128011.35 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 12801.13 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086198.649089 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115210.21 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 11521.02 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097719.670255 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103689.19 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 10368.92 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108088.589304 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93320.27 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 9332.03 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117420.616449 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83988.24 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 8398.82 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125819.440879 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75589.42 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 7558.94 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133378.382866 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68030.48 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 6803.05 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140181.430654 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61227.43 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 6122.74 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146304.173664 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55104.69 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 5510.47 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151814.642372 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49594.22 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 4959.42 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156774.064210 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44634.80 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 4463.48 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161237.543864 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40171.32 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 4017.13 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165254.675552 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36154.19 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3615.42 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168870.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32538.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172470.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28938.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176070.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25338.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179670.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21738.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183270.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18138.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186870.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14538.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190470.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10938.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194070.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7338.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197670.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3738.77 sec) 22:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201270.094072 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 138.77 sec) 22:03:00 [GPUGRID] [rr_sim] 2201408.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968133138.58G/439.78G) 22:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853829.77 22:03:00 [---] pick_jobs_to_run() start 22:03:00 [---] pick_jobs_to_run() end 22:03:00 [---] [rr_sim] end 22:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:03:00 [---] [cpu_sched_debug] preliminary job list: 22:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:03:00 [---] [cpu_sched_debug] final job list: 22:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:03:00 [---] [cpu_sched_debug] enforce_run_list: end 22:03:00 [---] choose_project(): 79380.000000 22:03:00 [WUProp@Home] doing RPC to get work 22:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356219.09 cpu 2105372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105372.90 22:03:00 [---] Got 0 tasks 22:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:04:00 [---] [rr_sim] doing sim: CPU sched 22:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968105548.22G/439.78G) 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201346.12 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 220134.61 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220134.612387 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981211.51 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 198121.15 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418255.763535 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783090.36 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 178309.04 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596564.799568 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604781.32 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 160478.13 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757042.931998 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444303.19 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 144430.32 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901473.251185 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299872.87 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 129987.29 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031460.538453 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169885.59 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 116988.56 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148449.096994 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052897.03 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 105289.70 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253738.799682 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947607.32 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 94760.73 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348499.532100 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852846.59 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 85284.66 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433784.191277 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767561.93 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 76756.19 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510540.384536 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690805.74 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 69080.57 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579620.958469 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621725.17 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 62172.52 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641793.475009 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559552.65 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 55955.26 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697748.739894 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503597.38 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 50359.74 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748108.478292 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453237.65 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 45323.76 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793432.242849 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407913.88 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 40791.39 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834223.630951 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367122.49 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 36712.25 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870935.880243 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330410.24 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 33041.02 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903976.904605 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297369.22 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 29736.92 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933713.826531 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267632.30 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 26763.23 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960477.056265 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240869.07 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 24086.91 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984563.963025 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216782.16 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 21678.22 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006242.179109 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195103.94 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 19510.39 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025752.573585 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175593.55 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 17559.36 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043311.928613 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158034.20 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 15803.42 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059115.348138 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142230.78 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 14223.08 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073338.425711 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128007.70 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 12800.77 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086139.195527 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115206.93 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 11520.69 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097659.888361 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103686.24 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 10368.62 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108028.511911 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93317.61 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 9331.76 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117360.273107 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83985.85 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 8398.59 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125758.858183 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75587.27 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 7558.73 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133317.584751 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68028.54 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 6802.85 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140120.438663 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61225.69 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 6122.57 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146243.007183 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55103.12 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 5510.31 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151753.318852 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49592.81 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 4959.28 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156712.599353 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44633.52 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 4463.35 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161175.951804 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40170.17 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 4017.02 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165192.969011 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36153.15 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3615.32 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168808.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32537.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172408.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28937.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176008.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25337.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179608.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21737.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183208.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18137.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186808.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14537.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190408.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10937.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194008.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7337.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197608.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3737.84 sec) 22:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201208.284496 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 137.84 sec) 22:04:00 [GPUGRID] [rr_sim] 2201346.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968105548.22G/439.78G) 22:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853827.04 22:04:00 [---] pick_jobs_to_run() start 22:04:00 [---] pick_jobs_to_run() end 22:04:00 [---] [rr_sim] end 22:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:04:00 [---] [cpu_sched_debug] preliminary job list: 22:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:04:00 [---] [cpu_sched_debug] final job list: 22:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:04:00 [---] [cpu_sched_debug] enforce_run_list: end 22:04:00 [---] choose_project(): 79440.000000 22:04:00 [WUProp@Home] doing RPC to get work 22:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356159.09 cpu 2105312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105312.90 22:04:00 [---] Got 0 tasks 22:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:05:00 [---] [rr_sim] doing sim: CPU sched 22:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968077957.86G/439.78G) 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201283.39 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 220128.34 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220128.338698 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981155.05 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 198115.50 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418243.843527 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1783039.54 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 178303.95 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596547.797873 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604735.59 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 160473.56 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 757021.356784 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444262.03 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 144426.20 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901447.559804 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299835.83 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 129983.58 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031431.142522 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169852.24 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 116985.22 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148416.366968 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052867.02 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 105286.70 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253703.068970 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947580.32 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 94758.03 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348461.100771 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852822.29 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 85282.23 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433743.329393 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767540.06 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 76754.01 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510497.335152 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690786.05 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 69078.61 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579575.940335 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621707.45 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 62170.74 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641746.685000 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559536.70 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 55953.67 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697700.355199 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503583.03 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 50358.30 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748058.658377 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453224.73 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 45322.47 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793381.131238 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407902.26 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 40790.23 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834171.356813 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367112.03 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 36711.20 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870882.559830 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330400.83 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 33040.08 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903922.642545 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297360.74 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 29736.07 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933658.716989 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267624.67 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 26762.47 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960421.183989 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240862.20 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 24086.22 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984507.404288 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216775.98 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 21677.60 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006185.002558 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195098.38 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 19509.84 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025694.841001 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175588.55 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 17558.85 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043253.695599 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158029.69 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 15802.97 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2059056.664738 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142226.72 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 14222.67 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073279.336962 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128004.05 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 12800.41 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086079.741965 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115203.65 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 11520.36 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097600.106467 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103683.28 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 10368.33 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107968.434518 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93314.95 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 9331.50 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117299.929765 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83983.46 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 8398.35 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125698.275487 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75585.11 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 7558.51 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133256.786637 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68026.60 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 6802.66 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140059.446672 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61223.94 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 6122.39 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146181.840703 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55101.55 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 5510.15 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151691.995331 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49591.39 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 4959.14 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156651.134496 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44632.25 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 4463.23 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161114.359745 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40169.03 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 4016.90 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165131.262469 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36152.12 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3615.21 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168746.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32536.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172346.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28936.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175946.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25336.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179546.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21736.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183146.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18136.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186746.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14536.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190346.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10936.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193946.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7336.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197546.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3736.91 sec) 22:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201146.474921 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 136.91 sec) 22:05:00 [GPUGRID] [rr_sim] 2201283.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968077957.86G/439.78G) 22:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853824.30 22:05:00 [---] pick_jobs_to_run() start 22:05:00 [---] pick_jobs_to_run() end 22:05:00 [---] [rr_sim] end 22:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:05:00 [---] [cpu_sched_debug] preliminary job list: 22:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:05:00 [---] [cpu_sched_debug] final job list: 22:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:05:00 [---] [cpu_sched_debug] enforce_run_list: end 22:05:00 [---] choose_project(): 79500.000000 22:05:00 [WUProp@Home] doing RPC to get work 22:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356099.09 cpu 2105252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105252.90 22:05:00 [---] Got 0 tasks 22:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:06:00 [---] [rr_sim] doing sim: CPU sched 22:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968050367.51G/439.78G) 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201220.65 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 220122.07 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220122.065010 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981098.59 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 198109.86 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418231.923519 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782988.73 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 178298.87 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596530.796178 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604689.85 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 160468.99 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756999.781570 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444220.87 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 144422.09 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901421.868423 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299798.78 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 129979.88 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031401.746591 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169818.90 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 116981.89 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148383.636942 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052837.01 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 105283.70 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253667.338258 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947553.31 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 94755.33 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348422.669443 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852797.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 85279.80 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433702.467509 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767518.18 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 76751.82 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510454.285768 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690766.36 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 69076.64 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579530.922202 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621689.73 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 62168.97 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641699.894992 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559520.76 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 55952.08 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697651.970503 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503568.68 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 50356.87 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1748008.838463 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453211.81 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 45321.18 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793330.019627 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407890.63 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 40789.06 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834119.082674 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367101.57 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 36710.16 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870829.239417 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330391.41 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 33039.14 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903868.380486 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297352.27 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 29735.23 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933603.607447 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267617.04 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 26761.70 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960365.311713 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240855.34 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 24085.53 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984450.845552 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216769.80 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 21676.98 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006127.826007 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195092.82 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 19509.28 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025637.108416 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175583.54 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 17558.35 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043195.462585 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158025.19 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 15802.52 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058997.981337 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142222.67 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 14222.27 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073220.248213 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128000.40 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 12800.04 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2086020.288402 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115200.36 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 11520.04 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097540.324572 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103680.33 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 10368.03 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107908.357125 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93312.29 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 9331.23 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117239.586423 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83981.06 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 8398.11 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125637.692791 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75582.96 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 7558.30 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133195.988522 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68024.66 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 6802.47 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139998.454680 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61222.20 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 6122.22 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146120.674222 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55099.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 5510.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151630.671810 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49589.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 4959.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156589.669640 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44630.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 4463.10 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161052.767686 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40167.88 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 4016.79 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165069.555928 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36151.09 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3615.11 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168684.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32535.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172284.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28935.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175884.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25335.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179484.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21735.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183084.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18135.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186684.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14535.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190284.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10935.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193884.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7335.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197484.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3735.98 sec) 22:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201084.665345 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 135.98 sec) 22:06:00 [GPUGRID] [rr_sim] 2201220.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968050367.51G/439.78G) 22:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853821.56 22:06:00 [---] pick_jobs_to_run() start 22:06:00 [---] pick_jobs_to_run() end 22:06:00 [---] [rr_sim] end 22:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:06:00 [---] [cpu_sched_debug] preliminary job list: 22:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:06:00 [---] [cpu_sched_debug] final job list: 22:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:06:00 [---] [cpu_sched_debug] enforce_run_list: end 22:06:00 [---] choose_project(): 79560.000000 22:06:00 [WUProp@Home] doing RPC to get work 22:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 356039.09 cpu 2105192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105192.90 22:06:00 [---] Got 0 tasks 22:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:07:00 [---] [rr_sim] doing sim: CPU sched 22:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (968022777.15G/439.78G) 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201157.91 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 220115.79 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220115.791322 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1981042.12 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 198104.21 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418220.003512 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782937.91 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 178293.79 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596513.794483 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604644.12 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 160464.41 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756978.206356 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444179.71 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 144417.97 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901396.177043 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299761.74 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 129976.17 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031372.350661 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169785.56 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 116978.56 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148350.906917 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052807.01 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 105280.70 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253631.607547 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947526.31 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 94752.63 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348384.238114 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852773.68 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 85277.37 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433661.605625 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767496.31 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 76749.63 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510411.236384 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690746.68 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 69074.67 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579485.904068 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621672.01 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 62167.20 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641653.104983 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559504.81 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 55950.48 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697603.585807 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503554.33 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 50355.43 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747959.018548 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453198.89 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 45319.89 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793278.908015 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407879.01 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 40787.90 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834066.808536 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367091.10 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 36709.11 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870775.919004 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330381.99 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 33038.20 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903814.118426 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297343.79 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 29734.38 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933548.497905 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267609.42 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 26760.94 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960309.439437 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240848.47 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 24084.85 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984394.286815 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216763.63 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 21676.36 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006070.649456 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195087.26 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 19508.73 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025579.375832 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175578.54 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 17557.85 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043137.229571 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158020.68 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 15802.07 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058939.297936 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142218.62 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 14221.86 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073161.159464 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127996.75 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 12799.68 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085960.834840 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115197.08 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 11519.71 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097480.542678 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103677.37 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 10367.74 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107848.279732 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93309.63 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 9330.96 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117179.243081 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83978.67 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 8397.87 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125577.110095 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75580.80 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 7558.08 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133135.190407 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68022.72 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 6802.27 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139937.462689 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61220.45 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 6122.05 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146059.507742 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55098.41 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 5509.84 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151569.348290 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49588.56 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 4958.86 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156528.204783 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44629.71 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 4462.97 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160991.175627 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40166.74 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 4016.67 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165007.849386 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36150.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3615.01 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168622.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32535.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172222.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28935.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175822.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25335.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179422.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21735.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183022.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18135.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186622.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14535.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190222.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10935.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193822.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7335.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197422.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3735.06 sec) 22:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2201022.855769 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 135.06 sec) 22:07:00 [GPUGRID] [rr_sim] 2201157.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (968022777.15G/439.78G) 22:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853818.83 22:07:00 [---] pick_jobs_to_run() start 22:07:00 [---] pick_jobs_to_run() end 22:07:00 [---] [rr_sim] end 22:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:07:00 [---] [cpu_sched_debug] preliminary job list: 22:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:07:00 [---] [cpu_sched_debug] final job list: 22:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:07:00 [---] [cpu_sched_debug] enforce_run_list: end 22:07:00 [---] choose_project(): 79620.000000 22:07:00 [WUProp@Home] doing RPC to get work 22:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355979.09 cpu 2105132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105132.90 22:07:00 [---] Got 0 tasks 22:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:08:00 [---] [rr_sim] doing sim: CPU sched 22:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967995186.80G/439.78G) 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201095.18 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 220109.52 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220109.517634 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980985.66 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 198098.57 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418208.083504 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782887.09 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 178288.71 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596496.792788 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604598.38 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 160459.84 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756956.631143 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444138.55 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 144413.85 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901370.485662 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299724.69 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 129972.47 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031342.954730 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169752.22 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 116975.22 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148318.176891 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052777.00 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 105277.70 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253595.876835 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947499.30 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 94749.93 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348345.806786 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852749.37 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 85274.94 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433620.743741 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767474.43 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 76747.44 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510368.187001 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690726.99 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 69072.70 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579440.885934 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621654.29 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 62165.43 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641606.314975 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559488.86 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 55948.89 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697555.201111 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503539.98 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 50354.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747909.198634 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453185.98 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 45318.60 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793227.796404 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407867.38 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 40786.74 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1834014.534398 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367080.64 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 36708.06 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870722.598592 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330372.58 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 33037.26 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903759.856366 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297335.32 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 29733.53 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933493.388363 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267601.79 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 26760.18 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960253.567161 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240841.61 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 24084.16 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984337.728079 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216757.45 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 21675.74 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2006013.472905 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195081.70 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 19508.17 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025521.643248 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175573.53 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 17557.35 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043078.996557 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158016.18 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 15801.62 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058880.614535 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142214.56 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 14221.46 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073102.070715 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127993.11 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 12799.31 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085901.381278 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115193.80 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 11519.38 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097420.760784 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103674.42 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 10367.44 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107788.202339 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93306.97 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 9330.70 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117118.899739 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83976.28 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 8397.63 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125516.527399 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75578.65 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 7557.86 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133074.392293 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68020.78 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 6802.08 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139876.470697 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61218.71 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 6121.87 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145998.341261 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55096.84 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 5509.68 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151508.024769 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49587.15 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 4958.72 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156466.739926 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44628.44 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 4462.84 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160929.583567 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40165.59 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 4016.56 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164946.142844 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36149.03 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3614.90 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168561.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32534.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172161.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28934.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175761.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25334.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179361.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21734.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182961.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18134.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186561.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14534.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190161.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10934.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193761.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7334.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197361.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3734.13 sec) 22:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200961.046194 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 134.13 sec) 22:08:00 [GPUGRID] [rr_sim] 2201095.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967995186.80G/439.78G) 22:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853816.09 22:08:00 [---] pick_jobs_to_run() start 22:08:00 [---] pick_jobs_to_run() end 22:08:00 [---] [rr_sim] end 22:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:08:00 [---] [cpu_sched_debug] preliminary job list: 22:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:08:00 [---] [cpu_sched_debug] final job list: 22:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:08:00 [---] [cpu_sched_debug] enforce_run_list: end 22:08:00 [---] choose_project(): 79680.000000 22:08:00 [WUProp@Home] doing RPC to get work 22:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355919.09 cpu 2105072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105072.90 22:08:00 [---] Got 0 tasks 22:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:09:00 [---] [rr_sim] doing sim: CPU sched 22:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967967596.44G/439.78G) 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201032.44 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 220103.24 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220103.243946 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980929.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 198092.92 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418196.163497 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782836.28 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 178283.63 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596479.791093 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604552.65 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 160455.26 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756935.055929 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444097.38 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 144409.74 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901344.794282 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299687.65 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 129968.76 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031313.558799 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169718.88 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 116971.89 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148285.446865 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052746.99 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 105274.70 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253560.146124 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947472.29 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 94747.23 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348307.375457 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852725.06 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 85272.51 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433579.881857 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767452.56 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 76745.26 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510325.137617 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690707.30 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 69070.73 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579395.867801 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621636.57 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 62163.66 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641559.524966 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559472.91 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 55947.29 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697506.816415 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503525.62 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 50352.56 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747859.378719 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453173.06 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 45317.31 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793176.684793 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407855.75 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 40785.58 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833962.260259 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367070.18 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 36707.02 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870669.278179 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330363.16 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 33036.32 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903705.594307 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297326.85 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 29732.68 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933438.278822 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267594.16 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 26759.42 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960197.694885 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240834.74 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 24083.47 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984281.169342 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216751.27 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 21675.13 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005956.296354 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195076.14 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 19507.61 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025463.910664 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175568.53 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 17556.85 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043020.763543 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158011.68 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 15801.17 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058821.931134 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142210.51 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 14221.05 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073042.981966 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127989.46 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 12798.95 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085841.927715 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115190.51 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 11519.05 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097360.978889 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103671.46 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 10367.15 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107728.124946 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93304.31 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 9330.43 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117058.556397 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83973.88 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 8397.39 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125455.944703 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75576.49 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 7557.65 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133013.594178 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68018.85 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 6801.88 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139815.478706 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61216.96 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 6121.70 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145937.174781 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55095.26 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 5509.53 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151446.701249 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49585.74 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 4958.57 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156405.275069 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44627.16 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 4462.72 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160867.991508 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40164.45 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 4016.44 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164884.436303 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36148.00 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3614.80 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168499.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32533.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172099.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28933.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175699.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25333.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179299.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21733.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182899.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18133.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186499.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14533.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190099.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10933.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193699.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7333.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197299.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3733.20 sec) 22:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200899.236618 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 133.20 sec) 22:09:00 [GPUGRID] [rr_sim] 2201032.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967967596.44G/439.78G) 22:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853813.35 22:09:00 [---] pick_jobs_to_run() start 22:09:00 [---] pick_jobs_to_run() end 22:09:00 [---] [rr_sim] end 22:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:09:00 [---] [cpu_sched_debug] preliminary job list: 22:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:09:00 [---] [cpu_sched_debug] final job list: 22:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:09:00 [---] [cpu_sched_debug] enforce_run_list: end 22:09:00 [---] choose_project(): 79740.000000 22:09:00 [WUProp@Home] doing RPC to get work 22:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355859.09 cpu 2105012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105012.90 22:09:00 [---] Got 0 tasks 22:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:10:00 [---] [rr_sim] doing sim: CPU sched 22:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967967596.44G/439.78G) 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2201032.44 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 220103.24 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220103.243946 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980929.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 198092.92 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418196.163497 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782836.28 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 178283.63 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596479.791093 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604552.65 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 160455.26 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756935.055929 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444097.38 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 144409.74 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901344.794282 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299687.65 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 129968.76 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031313.558799 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169718.88 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 116971.89 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148285.446865 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052746.99 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 105274.70 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253560.146124 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947472.29 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 94747.23 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348307.375457 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852725.06 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 85272.51 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433579.881857 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767452.56 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 76745.26 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510325.137617 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690707.30 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 69070.73 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579395.867801 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621636.57 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 62163.66 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641559.524966 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559472.91 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 55947.29 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697506.816415 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503525.62 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 50352.56 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747859.378719 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453173.06 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 45317.31 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793176.684793 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407855.75 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 40785.58 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833962.260259 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367070.18 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 36707.02 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870669.278179 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330363.16 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 33036.32 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903705.594307 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297326.85 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 29732.68 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933438.278822 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267594.16 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 26759.42 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960197.694885 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240834.74 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 24083.47 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984281.169342 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216751.27 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 21675.13 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005956.296354 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195076.14 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 19507.61 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025463.910664 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175568.53 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 17556.85 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2043020.763543 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158011.68 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 15801.17 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058821.931134 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142210.51 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 14221.05 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2073042.981966 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127989.46 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 12798.95 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085841.927715 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115190.51 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 11519.05 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097360.978889 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103671.46 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 10367.15 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107728.124946 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93304.31 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 9330.43 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117058.556397 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83973.88 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 8397.39 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125455.944703 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75576.49 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 7557.65 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133013.594178 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68018.85 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 6801.88 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139815.478706 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61216.96 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 6121.70 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145937.174781 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55095.26 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 5509.53 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151446.701249 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49585.74 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 4958.57 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156405.275069 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44627.16 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 4462.72 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160867.991508 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40164.45 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 4016.44 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164884.436303 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36148.00 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3614.80 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168499.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32533.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172099.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28933.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175699.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25333.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179299.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21733.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182899.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18133.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186499.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14533.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190099.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10933.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193699.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7333.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197299.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3733.20 sec) 22:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200899.236618 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 133.20 sec) 22:10:00 [GPUGRID] [rr_sim] 2201032.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967967596.44G/439.78G) 22:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853873.35 22:10:00 [---] pick_jobs_to_run() start 22:10:00 [---] pick_jobs_to_run() end 22:10:00 [---] [rr_sim] end 22:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:10:00 [---] [cpu_sched_debug] preliminary job list: 22:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:10:00 [---] [cpu_sched_debug] final job list: 22:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:10:00 [---] [cpu_sched_debug] enforce_run_list: end 22:10:00 [---] choose_project(): 79800.000000 22:10:00 [WUProp@Home] doing RPC to get work 22:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355799.09 cpu 2105012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2105012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2105012.90 22:10:00 [---] Got 0 tasks 22:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:11:00 [---] [rr_sim] doing sim: CPU sched 22:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967940006.08G/439.78G) 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200969.70 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 220096.97 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220096.970257 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980872.73 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 198087.27 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418184.243489 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782785.46 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 178278.55 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596462.789397 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604506.91 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 160450.69 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756913.480715 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444056.22 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 144405.62 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901319.102901 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299650.60 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 129965.06 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031284.162868 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169685.54 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 116968.55 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148252.716839 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052716.99 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 105271.70 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253524.415412 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947445.29 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 94744.53 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348268.944128 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852700.76 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 85270.08 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433539.019973 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767430.68 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 76743.07 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510282.088233 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690687.61 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 69068.76 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579350.849667 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621618.85 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 62161.89 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641512.734958 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559456.97 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 55945.70 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697458.431719 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503511.27 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 50351.13 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747809.558805 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453160.14 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 45316.01 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793125.573182 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407844.13 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 40784.41 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833909.986121 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367059.72 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 36705.97 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870615.957766 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330353.74 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 33035.37 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903651.332247 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297318.37 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 29731.84 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933383.169280 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267586.53 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 26758.65 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960141.822609 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240827.88 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 24082.79 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984224.610606 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216745.09 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 21674.51 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005899.119802 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195070.58 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 19507.06 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025406.178080 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175563.52 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 17556.35 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042962.530529 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158007.17 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 15800.72 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058763.247733 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142206.45 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 14220.65 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072983.893217 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127985.81 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 12798.58 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085782.474153 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115187.23 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 11518.72 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097301.196995 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103668.51 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 10366.85 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107668.047553 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93301.66 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 9330.17 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116998.213055 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83971.49 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 8397.15 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125395.362007 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75574.34 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 7557.43 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132952.796064 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68016.91 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 6801.69 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139754.486715 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61215.22 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 6121.52 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145876.008301 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55093.69 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 5509.37 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151385.377728 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49584.32 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 4958.43 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156343.810212 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44625.89 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 4462.59 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160806.399449 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40163.30 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 4016.33 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164822.729761 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36146.97 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3614.70 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168437.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32532.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172037.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28932.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175637.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25332.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179237.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21732.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182837.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18132.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186437.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14532.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190037.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10932.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193637.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7332.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197237.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3732.28 sec) 22:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200837.427042 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132.28 sec) 22:11:00 [GPUGRID] [rr_sim] 2200969.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967940006.08G/439.78G) 22:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853870.62 22:11:00 [---] pick_jobs_to_run() start 22:11:00 [---] pick_jobs_to_run() end 22:11:00 [---] [rr_sim] end 22:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:11:00 [---] [cpu_sched_debug] preliminary job list: 22:11:00 [---] [cpu_sched_debug] final job list: 22:11:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 22:11:00 [---] [work_fetch] Request work fetch: CPUs idle 22:11:00 [---] [cpu_sched_debug] enforce_run_list: end 22:11:00 [---] choose_project(): 79860.000000 22:11:00 [WUProp@Home] doing RPC to get work 22:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355739.09 cpu 2104952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104952.90 22:11:00 [---] Got 0 tasks 22:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:12:00 [---] [rr_sim] doing sim: CPU sched 22:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967940006.08G/439.78G) 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200969.70 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 220096.97 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220096.970257 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980872.73 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 198087.27 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418184.243489 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782785.46 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 178278.55 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596462.789397 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604506.91 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 160450.69 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756913.480715 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444056.22 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 144405.62 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901319.102901 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299650.60 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 129965.06 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031284.162868 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169685.54 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 116968.55 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148252.716839 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052716.99 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 105271.70 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253524.415412 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947445.29 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 94744.53 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348268.944128 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852700.76 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 85270.08 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433539.019973 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767430.68 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 76743.07 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510282.088233 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690687.61 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 69068.76 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579350.849667 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621618.85 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 62161.89 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641512.734958 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559456.97 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 55945.70 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697458.431719 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503511.27 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 50351.13 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747809.558805 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453160.14 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 45316.01 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793125.573182 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407844.13 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 40784.41 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833909.986121 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367059.72 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 36705.97 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870615.957766 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330353.74 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 33035.37 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903651.332247 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297318.37 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 29731.84 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933383.169280 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267586.53 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 26758.65 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960141.822609 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240827.88 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 24082.79 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984224.610606 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216745.09 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 21674.51 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005899.119802 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195070.58 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 19507.06 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025406.178080 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175563.52 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 17556.35 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042962.530529 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158007.17 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 15800.72 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058763.247733 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142206.45 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 14220.65 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072983.893217 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127985.81 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 12798.58 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085782.474153 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115187.23 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 11518.72 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097301.196995 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103668.51 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 10366.85 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107668.047553 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93301.66 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 9330.17 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116998.213055 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83971.49 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 8397.15 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125395.362007 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75574.34 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 7557.43 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132952.796064 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68016.91 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 6801.69 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139754.486715 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61215.22 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 6121.52 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145876.008301 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55093.69 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 5509.37 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151385.377728 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49584.32 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 4958.43 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156343.810212 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44625.89 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 4462.59 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160806.399449 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40163.30 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 4016.33 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164822.729761 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36146.97 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3614.70 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168437.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32532.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172037.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28932.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175637.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25332.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179237.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21732.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182837.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18132.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186437.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14532.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190037.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10932.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193637.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7332.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197237.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3732.28 sec) 22:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200837.427042 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 132.28 sec) 22:12:00 [GPUGRID] [rr_sim] 2200969.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967940006.08G/439.78G) 22:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853930.62 22:12:00 [---] pick_jobs_to_run() start 22:12:00 [---] pick_jobs_to_run() end 22:12:00 [---] [rr_sim] end 22:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:12:00 [---] [cpu_sched_debug] preliminary job list: 22:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:12:00 [---] [cpu_sched_debug] final job list: 22:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 22:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:12:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 967940006.08G 22:12:00 [---] [cpu_sched_debug] enforce_run_list: end 22:12:00 [---] choose_project(): 79920.000000 22:12:00 [WUProp@Home] doing RPC to get work 22:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355679.09 cpu 2104952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104952.90 22:12:00 [---] Got 0 tasks 22:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:13:00 [---] [rr_sim] doing sim: CPU sched 22:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967912415.73G/439.78G) 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200906.97 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 220090.70 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220090.696569 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980816.27 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 198081.63 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418172.323481 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782734.64 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 178273.46 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596445.787702 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604461.18 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 160446.12 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756891.905501 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1444015.06 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 144401.51 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901293.411520 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299613.55 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 129961.36 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031254.766938 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169652.20 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 116965.22 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148219.986813 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052686.98 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 105268.70 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253488.684701 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947418.28 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 94741.83 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348230.512800 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852676.45 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 85267.65 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433498.158089 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767408.81 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 76740.88 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510239.038849 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690667.93 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 69066.79 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579305.831534 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621601.13 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 62160.11 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641465.944949 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559441.02 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 55944.10 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697410.047024 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503496.92 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 50349.69 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747759.738890 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453147.23 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 45314.72 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793074.461571 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407832.50 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 40783.25 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833857.711983 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367049.25 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 36704.93 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870562.637354 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330344.33 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 33034.43 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903597.070187 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297309.90 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 29730.99 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933328.059738 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267578.91 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 26757.89 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960085.950333 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240821.02 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 24082.10 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984168.051869 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216738.91 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 21673.89 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005841.943251 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195065.02 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 19506.50 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025348.445495 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175558.52 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 17555.85 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042904.297515 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 158002.67 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 15800.27 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058704.564333 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142202.40 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 14220.24 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072924.804469 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127982.16 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 12798.22 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085723.020591 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115183.95 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 11518.39 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097241.415101 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103665.55 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 10366.56 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107607.970160 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93299.00 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 9329.90 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116937.869713 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83969.10 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 8396.91 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125334.779311 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75572.19 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 7557.22 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132891.997949 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68014.97 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 6801.50 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139693.494723 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61213.47 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 6121.35 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145814.841820 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55092.12 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 5509.21 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151324.054207 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49582.91 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 4958.29 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156282.345356 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44624.62 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 4462.46 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160744.807389 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40162.16 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 4016.22 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164761.023220 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36145.94 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3614.59 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168375.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32531.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171975.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28931.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175575.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25331.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179175.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21731.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182775.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18131.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186375.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14531.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189975.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10931.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193575.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7331.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197175.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3731.35 sec) 22:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200775.617467 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 131.35 sec) 22:13:00 [GPUGRID] [rr_sim] 2200906.97: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967912415.73G/439.78G) 22:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853927.88 22:13:00 [---] pick_jobs_to_run() start 22:13:00 [---] pick_jobs_to_run() end 22:13:00 [---] [rr_sim] end 22:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:13:00 [---] [cpu_sched_debug] preliminary job list: 22:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 22:13:00 [---] [cpu_sched_debug] final job list: 22:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:13:00 [---] [cpu_sched_debug] enforce_run_list: end 22:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:14:00 [---] [rr_sim] doing sim: CPU sched 22:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967884825.37G/439.78G) 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200844.23 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 220084.42 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220084.422881 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980759.81 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 198075.98 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418160.403474 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782683.83 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 178268.38 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596428.786007 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604415.44 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 160441.54 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756870.330288 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443973.90 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 144397.39 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901267.720140 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299576.51 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 129957.65 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031225.371007 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169618.86 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 116961.89 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148187.256787 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052656.97 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 105265.70 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253452.953989 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947391.27 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 94739.13 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348192.081471 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852652.15 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 85265.21 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433457.296205 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767386.93 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 76738.69 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510195.989466 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690648.24 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 69064.82 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579260.813400 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621583.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 62158.34 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641419.154941 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559425.07 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 55942.51 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697361.662328 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503482.57 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 50348.26 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747709.918976 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453134.31 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 45313.43 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1793023.349959 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407820.88 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 40782.09 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833805.437844 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367038.79 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 36703.88 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870509.316941 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330334.91 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 33033.49 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903542.808128 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297301.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 29730.14 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933272.950196 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267571.28 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 26757.13 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1960030.078057 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240814.15 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 24081.42 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984111.493132 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216732.74 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 21673.27 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005784.766700 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195059.46 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 19505.95 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025290.712911 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175553.52 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 17555.35 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042846.064501 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157998.16 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 15799.82 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058645.880932 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142198.35 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 14219.83 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072865.715720 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127978.51 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 12797.85 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085663.567029 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115180.66 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 11518.07 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097181.633207 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103662.60 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 10366.26 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107547.892767 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93296.34 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 9329.63 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116877.526371 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83966.70 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 8396.67 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125274.196615 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75570.03 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 7557.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132831.199834 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68013.03 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 6801.30 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139632.502732 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61211.73 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 6121.17 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145753.675340 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55090.55 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 5509.06 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151262.730687 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49581.50 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 4958.15 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156220.880499 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44623.35 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 4462.33 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160683.215330 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40161.01 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 4016.10 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164699.316678 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36144.91 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3614.49 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168313.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32530.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171913.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28930.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175513.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25330.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179113.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21730.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182713.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18130.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186313.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14530.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189913.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10930.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193513.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7330.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197113.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3730.42 sec) 22:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200713.807891 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 130.42 sec) 22:14:00 [GPUGRID] [rr_sim] 2200844.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967884825.37G/439.78G) 22:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853925.14 22:14:00 [---] pick_jobs_to_run() start 22:14:00 [---] pick_jobs_to_run() end 22:14:00 [---] [rr_sim] end 22:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:14:00 [---] [cpu_sched_debug] preliminary job list: 22:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:14:00 [---] [cpu_sched_debug] final job list: 22:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:14:00 [---] [cpu_sched_debug] enforce_run_list: end 22:14:00 [---] choose_project(): 80040.000000 22:14:00 [WUProp@Home] doing RPC to get work 22:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355559.09 cpu 2104832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104832.90 22:14:00 [---] Got 0 tasks 22:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:15:00 [---] [rr_sim] doing sim: CPU sched 22:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967857235.01G/439.78G) 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200781.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 220078.15 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220078.149193 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980703.34 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 198070.33 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418148.483466 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782633.01 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 178263.30 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596411.784312 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604369.71 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 160436.97 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756848.755074 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443932.74 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 144393.27 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901242.028759 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299539.46 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 129953.95 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031195.975076 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169585.52 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 116958.55 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148154.526761 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052626.97 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 105262.70 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253417.223278 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947364.27 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 94736.43 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348153.650143 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852627.84 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 85262.78 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433416.434321 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767365.06 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 76736.51 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510152.940082 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690628.55 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 69062.86 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579215.795266 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621565.70 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 62156.57 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641372.364932 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559409.13 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 55940.91 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697313.277632 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503468.21 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 50346.82 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747660.099061 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453121.39 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 45312.14 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792972.238348 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407809.25 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 40780.93 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833753.163706 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367028.33 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 36702.83 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870455.996528 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330325.50 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 33032.55 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903488.546068 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297292.95 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 29729.29 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933217.840654 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267563.65 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 26756.37 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959974.205781 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240807.29 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 24080.73 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1984054.934396 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216726.56 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 21672.66 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005727.590149 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195053.90 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 19505.39 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025232.980327 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175548.51 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 17554.85 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042787.831487 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157993.66 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 15799.37 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058587.197531 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142194.29 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 14219.43 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072806.626971 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127974.86 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 12797.49 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085604.113466 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115177.38 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 11517.74 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097121.851312 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103659.64 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 10365.96 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107487.815374 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93293.68 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 9329.37 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116817.183029 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83964.31 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 8396.43 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125213.613919 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75567.88 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 7556.79 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132770.401720 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68011.09 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 6801.11 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139571.510741 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61209.98 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 6121.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145692.508859 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55088.98 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 5508.90 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151201.407166 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49580.08 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 4958.01 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156159.415642 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44622.08 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 4462.21 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160621.623271 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40159.87 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 4015.99 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164637.610136 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36143.88 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3614.39 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168251.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32529.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171851.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28929.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175451.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25329.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179051.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21729.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182651.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18129.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186251.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14529.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189851.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10929.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193451.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7329.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197051.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3729.49 sec) 22:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200651.998315 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 129.49 sec) 22:15:00 [GPUGRID] [rr_sim] 2200781.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967857235.01G/439.78G) 22:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853922.40 22:15:00 [---] pick_jobs_to_run() start 22:15:00 [---] pick_jobs_to_run() end 22:15:00 [---] [rr_sim] end 22:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:15:00 [---] [cpu_sched_debug] preliminary job list: 22:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:15:00 [---] [cpu_sched_debug] final job list: 22:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:15:00 [---] [cpu_sched_debug] enforce_run_list: end 22:15:00 [---] choose_project(): 80100.000000 22:15:00 [WUProp@Home] doing RPC to get work 22:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355499.09 cpu 2104772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104772.90 22:15:00 [---] Got 0 tasks 22:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:16:00 [---] [rr_sim] doing sim: CPU sched 22:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967829644.66G/439.78G) 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200718.76 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 220071.88 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220071.875505 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980646.88 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 198064.69 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418136.563459 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782582.19 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 178258.22 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596394.782617 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604323.97 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 160432.40 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756827.179860 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443891.58 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 144389.16 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901216.337379 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299502.42 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 129950.24 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031166.579145 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169552.18 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 116955.22 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148121.796735 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052596.96 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 105259.70 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253381.492566 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947337.26 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 94733.73 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348115.218814 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852603.54 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 85260.35 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433375.572437 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767343.18 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 76734.32 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510109.890698 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690608.86 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 69060.89 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579170.777133 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621547.98 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 62154.80 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641325.574924 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559393.18 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 55939.32 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697264.892936 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503453.86 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 50345.39 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747610.279147 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453108.48 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 45310.85 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792921.126737 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407797.63 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 40779.76 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833700.889568 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367017.87 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 36701.79 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870402.676115 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330316.08 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 33031.61 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903434.284008 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297284.47 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 29728.45 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933162.731112 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267556.02 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 26755.60 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959918.333505 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240800.42 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 24080.04 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983998.375659 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216720.38 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 21672.04 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005670.413598 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195048.34 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 19504.83 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025175.247743 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175543.51 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 17554.35 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042729.598473 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157989.16 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 15798.92 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058528.514130 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142190.24 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 14219.02 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072747.538222 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127971.22 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 12797.12 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085544.659904 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115174.10 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 11517.41 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097062.069418 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103656.69 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 10365.67 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107427.737981 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93291.02 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 9329.10 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116756.839687 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83961.92 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 8396.19 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125153.031223 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75565.72 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 7556.57 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132709.603605 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68009.15 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 6800.92 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139510.518749 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61208.24 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 6120.82 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145631.342379 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55087.41 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 5508.74 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151140.083645 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49578.67 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 4957.87 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156097.950785 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44620.80 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 4462.08 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160560.031211 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40158.72 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 4015.87 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164575.903595 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36142.85 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3614.29 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168190.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32528.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171790.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28928.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175390.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25328.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178990.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21728.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182590.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18128.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186190.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14528.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189790.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10928.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193390.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7328.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196990.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3728.57 sec) 22:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200590.188740 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 128.57 sec) 22:16:00 [GPUGRID] [rr_sim] 2200718.76: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967829644.66G/439.78G) 22:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853919.67 22:16:00 [---] pick_jobs_to_run() start 22:16:00 [---] pick_jobs_to_run() end 22:16:00 [---] [rr_sim] end 22:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:16:00 [---] [cpu_sched_debug] preliminary job list: 22:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:16:00 [---] [cpu_sched_debug] final job list: 22:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:16:00 [---] [cpu_sched_debug] enforce_run_list: end 22:16:00 [---] choose_project(): 80160.000000 22:16:00 [WUProp@Home] doing RPC to get work 22:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355439.09 cpu 2104712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104712.90 22:16:00 [---] Got 0 tasks 22:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:17:00 [---] [rr_sim] doing sim: CPU sched 22:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967802054.30G/439.78G) 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200656.02 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 220065.60 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220065.601816 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980590.42 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 198059.04 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418124.643451 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782531.37 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 178253.14 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596377.780922 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604278.24 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 160427.82 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756805.604646 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443850.41 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 144385.04 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901190.645998 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299465.37 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 129946.54 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031137.183214 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169518.83 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 116951.88 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148089.066709 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052566.95 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 105256.70 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253345.761855 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947310.26 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 94731.03 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348076.787485 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852579.23 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 85257.92 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433334.710553 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767321.31 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 76732.13 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510066.841314 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690589.18 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 69058.92 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579125.758999 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621530.26 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 62153.03 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641278.784915 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559377.23 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 55937.72 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697216.508240 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503439.51 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 50343.95 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747560.459232 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453095.56 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 45309.56 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792870.015126 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407786.00 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 40778.60 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833648.615429 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 367007.40 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 36700.74 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870349.355703 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330306.66 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 33030.67 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903380.021949 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297276.00 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 29727.60 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933107.621570 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267548.40 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 26754.84 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959862.461229 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240793.56 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 24079.36 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983941.816923 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216714.20 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 21671.42 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005613.237047 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195042.78 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 19504.28 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025117.515158 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175538.50 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 17553.85 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042671.365459 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157984.65 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 15798.47 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058469.830729 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142186.19 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 14218.62 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072688.449473 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127967.57 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 12796.76 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085485.206342 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115170.81 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 11517.08 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2097002.287524 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103653.73 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 10365.37 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107367.660588 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93288.36 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 9328.84 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116696.496345 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83959.52 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 8395.95 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125092.448527 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75563.57 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 7556.36 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132648.805491 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68007.21 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 6800.72 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139449.526758 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61206.49 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 6120.65 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145570.175898 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55085.84 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 5508.58 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151078.760125 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49577.26 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 4957.73 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156036.485929 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44619.53 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 4461.95 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160498.439152 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40157.58 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 4015.76 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164514.197053 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36141.82 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3614.18 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168128.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32527.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171728.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28927.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175328.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25327.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178928.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21727.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182528.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18127.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186128.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14527.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189728.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10927.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193328.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7327.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196928.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3727.64 sec) 22:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200528.379164 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127.64 sec) 22:17:00 [GPUGRID] [rr_sim] 2200656.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967802054.30G/439.78G) 22:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853916.93 22:17:00 [---] pick_jobs_to_run() start 22:17:00 [---] pick_jobs_to_run() end 22:17:00 [---] [rr_sim] end 22:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:17:00 [---] [cpu_sched_debug] preliminary job list: 22:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:17:00 [---] [cpu_sched_debug] final job list: 22:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:17:00 [---] [cpu_sched_debug] enforce_run_list: end 22:17:00 [---] choose_project(): 80220.000000 22:17:00 [WUProp@Home] doing RPC to get work 22:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355379.09 cpu 2104652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104652.90 22:17:00 [---] Got 0 tasks 22:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:18:00 [---] [rr_sim] doing sim: CPU sched 22:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967774463.95G/439.78G) 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200593.28 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 220059.33 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220059.328128 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980533.95 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 198053.40 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418112.723443 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782480.56 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 178248.06 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596360.779227 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604232.50 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 160423.25 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756784.029432 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443809.25 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 144380.93 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901164.954617 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299428.33 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 129942.83 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031107.787284 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169485.49 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 116948.55 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148056.336683 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052536.94 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 105253.69 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253310.031143 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947283.25 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 94728.33 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1348038.356157 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852554.93 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 85255.49 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433293.848669 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767299.43 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 76729.94 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1510023.791930 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690569.49 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 69056.95 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579080.740865 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621512.54 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 62151.25 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641231.994907 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559361.29 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 55936.13 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697168.123544 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503425.16 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 50342.52 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747510.639318 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453082.64 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 45308.26 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792818.903514 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407774.38 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 40777.44 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833596.341291 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366996.94 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 36699.69 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870296.035290 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330297.25 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 33029.72 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903325.759889 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297267.52 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 29726.75 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1933052.512028 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267540.77 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 26754.08 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959806.588953 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240786.69 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 24078.67 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983885.258186 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216708.02 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 21670.80 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005556.060496 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195037.22 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 19503.72 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025059.782574 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175533.50 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 17553.35 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042613.132445 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157980.15 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 15798.01 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058411.147328 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142182.13 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 14218.21 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072629.360724 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127963.92 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 12796.39 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085425.752779 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115167.53 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 11516.75 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096942.505630 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103650.78 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 10365.08 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107307.583195 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93285.70 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 9328.57 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116636.153003 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83957.13 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 8395.71 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125031.865831 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75561.42 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 7556.14 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132588.007376 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68005.27 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 6800.53 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139388.534766 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61204.75 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 6120.47 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145509.009418 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55084.27 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 5508.43 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151017.436604 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49575.84 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 4957.58 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155975.021072 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44618.26 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 4461.83 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160436.847093 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40156.43 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 4015.64 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164452.490512 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36140.79 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3614.08 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168066.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32526.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171666.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28926.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175266.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25326.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178866.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21726.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182466.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18126.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186066.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14526.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189666.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10926.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193266.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7326.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196866.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3726.71 sec) 22:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200466.569588 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 126.71 sec) 22:18:00 [GPUGRID] [rr_sim] 2200593.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967774463.95G/439.78G) 22:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853914.19 22:18:00 [---] pick_jobs_to_run() start 22:18:00 [---] pick_jobs_to_run() end 22:18:00 [---] [rr_sim] end 22:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:18:00 [---] [cpu_sched_debug] preliminary job list: 22:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:18:00 [---] [cpu_sched_debug] final job list: 22:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:18:00 [---] [cpu_sched_debug] enforce_run_list: end 22:18:00 [---] choose_project(): 80280.000000 22:18:00 [WUProp@Home] doing RPC to get work 22:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355319.09 cpu 2104592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104592.90 22:18:00 [---] Got 0 tasks 22:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:19:00 [---] [rr_sim] doing sim: CPU sched 22:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967746873.59G/439.78G) 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200530.54 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 220053.05 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220053.054440 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980477.49 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 198047.75 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418100.803436 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782429.74 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 178242.97 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596343.777532 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604186.77 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 160418.68 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756762.454219 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443768.09 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 144376.81 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901139.263237 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299391.28 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 129939.13 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031078.391353 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169452.15 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 116945.22 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1148023.606657 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052506.94 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 105250.69 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253274.300432 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947256.24 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 94725.62 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347999.924828 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852530.62 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 85253.06 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433252.986785 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767277.56 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 76727.76 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509980.742547 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690549.80 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 69054.98 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1579035.722732 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621494.82 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 62149.48 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641185.204899 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559345.34 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 55934.53 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697119.738849 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503410.81 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 50341.08 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747460.819404 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453069.72 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 45306.97 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792767.791903 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407762.75 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 40776.28 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833544.067153 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366986.48 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 36698.65 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870242.714877 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330287.83 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 33028.78 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903271.497829 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297259.05 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 29725.90 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932997.402486 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267533.14 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 26753.31 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959750.716678 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240779.83 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 24077.98 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983828.699450 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216701.84 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 21670.18 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005498.883945 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195031.66 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 19503.17 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2025002.049990 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175528.49 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 17552.85 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042554.899431 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157975.64 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 15797.56 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058352.463928 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142178.08 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 14217.81 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072570.271975 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127960.27 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 12796.03 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085366.299217 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115164.25 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 11516.42 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096882.723735 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103647.82 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 10364.78 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107247.505802 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93283.04 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 9328.30 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116575.809661 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83954.73 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 8395.47 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124971.283135 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75559.26 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 7555.93 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132527.209261 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68003.34 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 6800.33 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139327.542775 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61203.00 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 6120.30 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145447.842937 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55082.70 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 5508.27 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150956.113084 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49574.43 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 4957.44 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155913.556215 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44616.99 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 4461.70 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160375.255033 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40155.29 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 4015.53 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164390.783970 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36139.76 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3613.98 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168004.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32525.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171604.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28925.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175204.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25325.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178804.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21725.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182404.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18125.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186004.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14525.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189604.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10925.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193204.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7325.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196804.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3725.78 sec) 22:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200404.760013 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 125.78 sec) 22:19:00 [GPUGRID] [rr_sim] 2200530.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967746873.59G/439.78G) 22:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853911.46 22:19:00 [---] pick_jobs_to_run() start 22:19:00 [---] pick_jobs_to_run() end 22:19:00 [---] [rr_sim] end 22:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:19:00 [---] [cpu_sched_debug] preliminary job list: 22:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:19:00 [---] [cpu_sched_debug] final job list: 22:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:19:00 [---] [cpu_sched_debug] enforce_run_list: end 22:19:00 [---] choose_project(): 80340.000000 22:19:00 [WUProp@Home] doing RPC to get work 22:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355259.09 cpu 2104532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104532.90 22:19:00 [---] Got 0 tasks 22:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:20:00 [---] [rr_sim] doing sim: CPU sched 22:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967719283.23G/439.78G) 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200467.81 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 220046.78 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220046.780752 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980421.03 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 198042.10 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418088.883428 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782378.92 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 178237.89 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596326.775837 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604141.03 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 160414.10 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756740.879005 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443726.93 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 144372.69 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901113.571856 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299354.24 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 129935.42 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031048.995422 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169418.81 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 116941.88 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147990.876632 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052476.93 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 105247.69 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253238.569720 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947229.24 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 94722.92 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347961.493500 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852506.31 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 85250.63 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433212.124901 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767255.68 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 76725.57 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509937.693163 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690530.11 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 69053.01 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578990.704598 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621477.10 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 62147.71 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641138.414890 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559329.39 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 55932.94 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697071.354153 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503396.45 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 50339.65 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747410.999489 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453056.81 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 45305.68 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792716.680292 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407751.13 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 40775.11 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833491.793014 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366976.01 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 36697.60 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870189.394465 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330278.41 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 33027.84 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903217.235770 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297250.57 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 29725.06 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932942.292944 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267525.51 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 26752.55 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959694.844402 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240772.96 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 24077.30 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983772.140713 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216695.67 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 21669.57 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005441.707393 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195026.10 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 19502.61 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024944.317406 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175523.49 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 17552.35 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042496.666417 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157971.14 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 15797.11 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058293.780527 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142174.03 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 14217.40 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072511.183226 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127956.62 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 12795.66 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085306.845655 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115160.96 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 11516.10 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096822.941841 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103644.87 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 10364.49 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107187.428409 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93280.38 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 9328.04 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116515.466319 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83952.34 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 8395.23 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124910.700439 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75557.11 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 7555.71 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132466.411147 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68001.40 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 6800.14 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139266.550784 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61201.26 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 6120.13 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145386.676457 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55081.13 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 5508.11 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150894.789563 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49573.02 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 4957.30 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155852.091358 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44615.72 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 4461.57 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160313.662974 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40154.14 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 4015.41 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164329.077428 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36138.73 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3613.87 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167942.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32524.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171542.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28924.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175142.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25324.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178742.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21724.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182342.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18124.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185942.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14524.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189542.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10924.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193142.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7324.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196742.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3724.86 sec) 22:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200342.950437 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 124.86 sec) 22:20:00 [GPUGRID] [rr_sim] 2200467.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967719283.23G/439.78G) 22:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853908.72 22:20:00 [---] pick_jobs_to_run() start 22:20:00 [---] pick_jobs_to_run() end 22:20:00 [---] [rr_sim] end 22:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:20:00 [---] [cpu_sched_debug] preliminary job list: 22:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:20:00 [---] [cpu_sched_debug] final job list: 22:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:20:00 [---] [cpu_sched_debug] enforce_run_list: end 22:20:00 [---] choose_project(): 80400.000000 22:20:00 [WUProp@Home] doing RPC to get work 22:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355199.09 cpu 2104472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104472.90 22:20:00 [---] Got 0 tasks 22:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:21:00 [---] [rr_sim] doing sim: CPU sched 22:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967691692.88G/439.78G) 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200405.07 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 220040.51 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220040.507063 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980364.56 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 198036.46 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418076.963421 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782328.11 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 178232.81 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596309.774142 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604095.30 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 160409.53 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756719.303791 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443685.77 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 144368.58 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901087.880475 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299317.19 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 129931.72 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1031019.599491 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169385.47 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 116938.55 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147958.146606 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052446.92 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 105244.69 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253202.839009 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947202.23 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 94720.22 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347923.062171 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852482.01 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 85248.20 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433171.263017 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767233.81 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 76723.38 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509894.643779 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690510.43 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 69051.04 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578945.686465 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621459.38 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 62145.94 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641091.624882 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559313.45 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 55931.34 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1697022.969457 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503382.10 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 50338.21 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747361.179575 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453043.89 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 45304.39 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792665.568681 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407739.50 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 40773.95 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833439.518876 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366965.55 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 36696.56 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870136.074052 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330269.00 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 33026.90 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903162.973710 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297242.10 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 29724.21 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932887.183402 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267517.89 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 26751.79 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959638.972126 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240766.10 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 24076.61 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983715.581977 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216689.49 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 21668.95 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005384.530842 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195020.54 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 19502.05 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024886.584822 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175518.49 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 17551.85 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042438.433403 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157966.64 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 15796.66 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058235.097126 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142169.97 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 14217.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072452.094477 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127952.98 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 12795.30 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085247.392093 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115157.68 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 11515.77 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096763.159947 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103641.91 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 10364.19 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107127.351015 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93277.72 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 9327.77 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116455.122977 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83949.95 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 8394.99 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124850.117743 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75554.95 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 7555.50 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132405.613032 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67999.46 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 6799.95 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139205.558792 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61199.51 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 6119.95 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145325.509977 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55079.56 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 5507.96 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150833.466042 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49571.60 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 4957.16 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155790.626502 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44614.44 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 4461.44 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160252.070915 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40153.00 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 4015.30 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164267.370887 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36137.70 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3613.77 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167881.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32523.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171481.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28923.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175081.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25323.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178681.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21723.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182281.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18123.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185881.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14523.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189481.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10923.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193081.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7323.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196681.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3723.93 sec) 22:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200281.140862 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 123.93 sec) 22:21:00 [GPUGRID] [rr_sim] 2200405.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967691692.88G/439.78G) 22:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853905.98 22:21:00 [---] pick_jobs_to_run() start 22:21:00 [---] pick_jobs_to_run() end 22:21:00 [---] [rr_sim] end 22:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:21:00 [---] [cpu_sched_debug] preliminary job list: 22:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:21:00 [---] [cpu_sched_debug] final job list: 22:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:21:00 [---] [cpu_sched_debug] enforce_run_list: end 22:21:00 [---] choose_project(): 80460.000000 22:21:00 [WUProp@Home] doing RPC to get work 22:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355139.09 cpu 2104412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104412.90 22:21:00 [---] Got 0 tasks 22:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:22:00 [---] [rr_sim] doing sim: CPU sched 22:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967664102.52G/439.78G) 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200342.33 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 220034.23 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220034.233375 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980308.10 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 198030.81 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418065.043413 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782277.29 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 178227.73 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596292.772447 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604049.56 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 160404.96 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756697.728577 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443644.61 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 144364.46 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901062.189095 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299280.14 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 129928.01 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030990.203561 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169352.13 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 116935.21 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147925.416580 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052416.92 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 105241.69 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253167.108297 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947175.23 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 94717.52 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347884.630842 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852457.70 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 85245.77 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433130.401133 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767211.93 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 76721.19 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509851.594395 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690490.74 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 69049.07 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578900.668331 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621441.67 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 62144.17 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1641044.834873 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559297.50 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 55929.75 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696974.584761 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503367.75 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 50336.77 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747311.359660 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453030.97 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 45303.10 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792614.457069 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407727.88 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 40772.79 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833387.244738 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366955.09 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 36695.51 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870082.753639 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330259.58 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 33025.96 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903108.711650 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297233.62 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 29723.36 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932832.073861 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267510.26 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 26751.03 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959583.099850 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240759.23 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 24075.92 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983659.023240 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216683.31 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 21668.33 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005327.354291 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195014.98 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 19501.50 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024828.852237 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175513.48 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 17551.35 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042380.200389 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157962.13 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 15796.21 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058176.413725 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142165.92 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 14216.59 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072393.005728 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127949.33 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 12794.93 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085187.938530 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115154.40 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 11515.44 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096703.378052 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103638.96 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 10363.90 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107067.273622 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93275.06 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 9327.51 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116394.779635 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83947.55 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 8394.76 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124789.535047 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75552.80 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 7555.28 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132344.814918 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67997.52 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 6799.75 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139144.566801 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61197.77 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 6119.78 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145264.343496 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55077.99 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 5507.80 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150772.142522 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49570.19 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 4957.02 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155729.161645 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44613.17 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 4461.32 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160190.478856 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40151.85 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 4015.19 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164205.664345 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36136.67 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3613.67 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167819.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32523.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171419.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28923.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175019.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25323.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178619.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21723.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182219.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18123.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185819.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14523.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189419.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10923.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193019.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7323.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196619.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3723.00 sec) 22:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200219.331286 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 123.00 sec) 22:22:00 [GPUGRID] [rr_sim] 2200342.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967664102.52G/439.78G) 22:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853903.25 22:22:00 [---] pick_jobs_to_run() start 22:22:00 [---] pick_jobs_to_run() end 22:22:00 [---] [rr_sim] end 22:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:22:00 [---] [cpu_sched_debug] preliminary job list: 22:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:22:00 [---] [cpu_sched_debug] final job list: 22:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:22:00 [---] [cpu_sched_debug] enforce_run_list: end 22:22:00 [---] choose_project(): 80520.000000 22:22:00 [WUProp@Home] doing RPC to get work 22:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355079.09 cpu 2104352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104352.90 22:22:00 [---] Got 0 tasks 22:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:23:00 [---] [rr_sim] doing sim: CPU sched 22:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967636512.16G/439.78G) 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200279.60 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 220027.96 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220027.959687 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980251.64 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 198025.16 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418053.123405 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782226.47 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 178222.65 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596275.770752 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1604003.83 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 160400.38 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756676.153364 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443603.44 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 144360.34 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901036.497714 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299243.10 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 129924.31 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030960.807630 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169318.79 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 116931.88 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147892.686554 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052386.91 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 105238.69 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253131.377585 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947148.22 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 94714.82 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347846.199514 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852433.40 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 85243.34 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433089.539250 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767190.06 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 76719.01 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509808.545012 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690471.05 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 69047.11 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578855.650197 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621423.95 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 62142.39 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640998.044865 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559281.55 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 55928.16 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696926.200065 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503353.40 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 50335.34 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747261.539746 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453018.06 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 45301.81 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792563.345458 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407716.25 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 40771.63 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833334.970599 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366944.63 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 36694.46 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1870029.433226 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330250.16 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 33025.02 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903054.449591 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297225.15 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 29722.51 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932776.964319 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267502.63 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 26750.26 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959527.227574 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240752.37 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 24075.24 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983602.464503 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216677.13 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 21667.71 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005270.177740 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195009.42 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 19500.94 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024771.119653 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175508.48 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 17550.85 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042321.967375 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157957.63 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 15795.76 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058117.730324 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142161.87 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 14216.19 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072333.916979 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127945.68 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 12794.57 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085128.484968 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115151.11 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 11515.11 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096643.596158 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103636.00 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 10363.60 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107007.196229 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93272.40 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 9327.24 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116334.436293 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83945.16 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 8394.52 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124728.952351 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75550.64 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 7555.06 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132284.016803 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67995.58 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 6799.56 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139083.574810 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61196.02 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 6119.60 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145203.177016 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55076.42 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 5507.64 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150710.819001 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49568.78 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 4956.88 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155667.696788 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44611.90 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 4461.19 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160128.886796 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40150.71 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 4015.07 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164143.957804 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36135.64 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3613.56 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167757.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32522.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171357.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28922.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174957.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25322.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178557.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21722.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182157.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18122.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185757.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14522.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189357.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10922.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192957.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7322.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196557.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3722.08 sec) 22:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200157.521710 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 122.08 sec) 22:23:00 [GPUGRID] [rr_sim] 2200279.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967636512.16G/439.78G) 22:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853900.51 22:23:00 [---] pick_jobs_to_run() start 22:23:00 [---] pick_jobs_to_run() end 22:23:00 [---] [rr_sim] end 22:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:23:00 [---] [cpu_sched_debug] preliminary job list: 22:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:23:00 [---] [cpu_sched_debug] final job list: 22:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:23:00 [---] [cpu_sched_debug] enforce_run_list: end 22:23:00 [---] choose_project(): 80580.000000 22:23:00 [WUProp@Home] doing RPC to get work 22:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 355019.09 cpu 2104292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104292.90 22:23:00 [---] Got 0 tasks 22:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:24:00 [---] [rr_sim] doing sim: CPU sched 22:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967608921.81G/439.78G) 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200216.86 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 220021.69 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220021.685999 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980195.17 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 198019.52 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418041.203398 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782175.66 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 178217.57 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596258.769057 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603958.09 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 160395.81 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756654.578150 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443562.28 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 144356.23 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 901010.806334 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299206.05 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 129920.61 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030931.411699 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169285.45 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 116928.54 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147859.956528 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052356.90 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 105235.69 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253095.646874 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947121.21 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 94712.12 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347807.768185 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852409.09 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 85240.91 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433048.677366 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767168.18 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 76716.82 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509765.495628 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690451.36 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 69045.14 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578810.632064 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621406.23 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 62140.62 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640951.254856 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559265.61 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 55926.56 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696877.815369 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503339.04 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 50333.90 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747211.719831 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 453005.14 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 45300.51 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792512.233847 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407704.63 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 40770.46 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833282.696461 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366934.16 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 36693.42 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869976.112814 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330240.75 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 33024.07 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1903000.187531 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297216.67 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 29721.67 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932721.854777 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267495.01 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 26749.50 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959471.355298 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240745.50 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 24074.55 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983545.905767 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216670.95 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 21667.10 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005213.001189 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 195003.86 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 19500.39 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024713.387069 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175503.47 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 17550.35 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042263.734361 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157953.13 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 15795.31 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058059.046923 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142157.81 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 14215.78 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072274.828230 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127942.03 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 12794.20 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085069.031406 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115147.83 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 11514.78 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096583.814264 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103633.05 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 10363.30 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106947.118836 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93269.74 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 9326.97 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116274.092951 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83942.77 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 8394.28 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124668.369655 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75548.49 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 7554.85 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132223.218688 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67993.64 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 6799.36 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139022.582818 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61194.28 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 6119.43 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145142.010535 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55074.85 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 5507.48 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150649.495481 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49567.36 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 4956.74 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155606.231931 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44610.63 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 4461.06 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160067.294737 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40149.57 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 4014.96 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164082.251262 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36134.61 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3613.46 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167695.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32521.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171295.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28921.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174895.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25321.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178495.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21721.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182095.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18121.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185695.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14521.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189295.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10921.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192895.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7321.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196495.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3721.15 sec) 22:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200095.712135 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 121.15 sec) 22:24:00 [GPUGRID] [rr_sim] 2200216.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967608921.81G/439.78G) 22:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853897.77 22:24:00 [---] pick_jobs_to_run() start 22:24:00 [---] pick_jobs_to_run() end 22:24:00 [---] [rr_sim] end 22:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:24:00 [---] [cpu_sched_debug] preliminary job list: 22:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:24:00 [---] [cpu_sched_debug] final job list: 22:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:24:00 [---] [cpu_sched_debug] enforce_run_list: end 22:24:00 [---] choose_project(): 80640.000000 22:24:00 [WUProp@Home] doing RPC to get work 22:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354959.09 cpu 2104232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104232.90 22:24:00 [---] Got 0 tasks 22:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:25:00 [---] [rr_sim] doing sim: CPU sched 22:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967581331.45G/439.78G) 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200154.12 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 220015.41 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220015.412311 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980138.71 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 198013.87 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418029.283390 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782124.84 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 178212.48 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596241.767362 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603912.36 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 160391.24 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756633.002936 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443521.12 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 144352.11 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900985.114953 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299169.01 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 129916.90 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030902.015768 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169252.11 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 116925.21 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147827.226502 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052326.90 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 105232.69 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253059.916162 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947094.21 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 94709.42 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347769.336857 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852384.79 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 85238.48 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1433007.815482 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767146.31 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 76714.63 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509722.446244 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690431.68 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 69043.17 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578765.613930 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621388.51 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 62138.85 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640904.464848 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559249.66 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 55924.97 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696829.430674 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503324.69 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 50332.47 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747161.899917 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452992.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 45299.22 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792461.122236 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407693.00 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 40769.30 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833230.422323 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366923.70 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 36692.37 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869922.792401 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330231.33 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 33023.13 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902945.925471 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297208.20 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 29720.82 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932666.745235 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267487.38 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 26748.74 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959415.483022 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240738.64 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 24073.86 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983489.347030 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216664.78 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 21666.48 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005155.824638 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194998.30 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 19499.83 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024655.654485 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175498.47 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 17549.85 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042205.501347 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157948.62 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 15794.86 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2058000.363523 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142153.76 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 14215.38 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072215.739481 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127938.38 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 12793.84 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2085009.577843 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115144.55 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 11514.45 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096524.032370 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103630.09 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 10363.01 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106887.041443 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93267.08 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 9326.71 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116213.749609 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83940.37 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 8394.04 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124607.786959 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75546.34 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 7554.63 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132162.420574 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67991.70 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 6799.17 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138961.590827 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61192.53 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 6119.25 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145080.844055 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55073.28 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 5507.33 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150588.171960 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49565.95 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 4956.60 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155544.767074 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44609.36 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 4460.94 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160005.702678 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40148.42 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 4014.84 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164020.544720 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36133.58 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3613.36 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167633.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32520.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171233.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28920.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174833.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25320.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178433.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21720.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182033.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18120.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185633.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14520.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189233.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10920.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192833.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7320.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196433.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3720.22 sec) 22:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2200033.902559 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 120.22 sec) 22:25:00 [GPUGRID] [rr_sim] 2200154.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967581331.45G/439.78G) 22:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853895.04 22:25:00 [---] pick_jobs_to_run() start 22:25:00 [---] pick_jobs_to_run() end 22:25:00 [---] [rr_sim] end 22:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:25:00 [---] [cpu_sched_debug] preliminary job list: 22:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:25:00 [---] [cpu_sched_debug] final job list: 22:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:25:00 [---] [cpu_sched_debug] enforce_run_list: end 22:25:00 [---] choose_project(): 80700.000000 22:25:00 [WUProp@Home] doing RPC to get work 22:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354899.09 cpu 2104172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104172.90 22:25:00 [---] Got 0 tasks 22:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:26:00 [---] [rr_sim] doing sim: CPU sched 22:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967553741.10G/439.78G) 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200091.39 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 220009.14 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220009.138622 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980082.25 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 198008.22 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418017.363382 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782074.02 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 178207.40 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596224.765667 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603866.62 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 160386.66 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756611.427722 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443479.96 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 144348.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900959.423572 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299131.96 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 129913.20 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030872.619837 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169218.77 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 116921.88 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147794.496476 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052296.89 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 105229.69 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1253024.185451 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947067.20 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 94706.72 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347730.905528 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852360.48 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 85236.05 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432966.953598 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767124.43 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 76712.44 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509679.396860 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690411.99 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 69041.20 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578720.595797 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621370.79 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 62137.08 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640857.674839 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559233.71 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 55923.37 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696781.045978 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503310.34 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 50331.03 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747112.080002 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452979.31 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 45297.93 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792410.010624 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407681.38 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 40768.14 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833178.148184 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366913.24 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 36691.32 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869869.471988 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330221.91 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 33022.19 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902891.663412 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297199.72 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 29719.97 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932611.635693 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267479.75 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 26747.98 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959359.610746 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240731.78 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 24073.18 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983432.788294 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216658.60 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 21665.86 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005098.648087 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194992.74 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 19499.27 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024597.921900 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175493.46 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 17549.35 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042147.268333 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157944.12 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 15794.41 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057941.680122 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142149.71 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 14214.97 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072156.650732 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127934.74 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 12793.47 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084950.124281 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115141.26 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 11514.13 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096464.250475 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103627.14 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 10362.71 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106826.964050 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93264.42 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 9326.44 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116153.406267 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83937.98 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 8393.80 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124547.204263 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75544.18 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 7554.42 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132101.622459 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67989.76 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 6798.98 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138900.598836 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61190.79 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 6119.08 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145019.677574 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55071.71 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 5507.17 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150526.848439 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49564.54 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 4956.45 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155483.302218 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44608.08 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 4460.81 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159944.110618 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40147.28 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 4014.73 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163958.838179 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36132.55 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3613.25 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167572.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32519.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171172.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28919.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174772.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25319.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178372.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21719.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181972.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18119.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185572.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14519.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189172.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10919.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192772.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7319.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196372.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3719.29 sec) 22:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199972.092983 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 119.29 sec) 22:26:00 [GPUGRID] [rr_sim] 2200091.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967553741.10G/439.78G) 22:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853892.30 22:26:00 [---] pick_jobs_to_run() start 22:26:00 [---] pick_jobs_to_run() end 22:26:00 [---] [rr_sim] end 22:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:26:00 [---] [cpu_sched_debug] preliminary job list: 22:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:26:00 [---] [cpu_sched_debug] final job list: 22:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:26:00 [---] [cpu_sched_debug] enforce_run_list: end 22:26:00 [---] choose_project(): 80760.000000 22:26:00 [WUProp@Home] doing RPC to get work 22:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354839.09 cpu 2104112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104112.90 22:26:00 [---] Got 0 tasks 22:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:27:00 [---] [rr_sim] doing sim: CPU sched 22:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967526150.74G/439.78G) 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2200028.65 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 220002.86 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 220002.864934 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1980025.78 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 198002.58 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 418005.443375 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1782023.21 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 178202.32 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596207.763971 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603820.89 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 160382.09 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756589.852508 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443438.80 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 144343.88 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900933.732192 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299094.92 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 129909.49 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030843.223907 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169185.43 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 116918.54 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147761.766450 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052266.88 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 105226.69 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252988.454739 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947040.19 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 94704.02 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347692.474200 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852336.18 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 85233.62 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432926.091714 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767102.56 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 76710.26 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509636.347476 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690392.30 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 69039.23 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578675.577663 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621353.07 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 62135.31 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640810.884831 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559217.76 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 55921.78 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696732.661282 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503295.99 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 50329.60 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747062.260088 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452966.39 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 45296.64 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792358.899013 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407669.75 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 40766.98 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833125.874046 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366902.78 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 36690.28 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869816.151575 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330212.50 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 33021.25 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902837.401352 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297191.25 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 29719.12 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932556.526151 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267472.12 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 26747.21 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959303.738470 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240724.91 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 24072.49 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983376.229557 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216652.42 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 21665.24 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2005041.471536 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194987.18 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 19498.72 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024540.189316 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175488.46 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 17548.85 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042089.035319 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157939.61 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 15793.96 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057882.996721 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142145.65 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 14214.57 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072097.561983 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127931.09 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 12793.11 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084890.670719 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115137.98 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 11513.80 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096404.468581 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103624.18 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 10362.42 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106766.886657 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93261.76 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 9326.18 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116093.062926 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83935.59 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 8393.56 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124486.621567 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75542.03 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 7554.20 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132040.824345 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67987.82 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 6798.78 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138839.606844 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61189.04 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 6118.90 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144958.511094 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55070.14 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 5507.01 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150465.524919 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49563.12 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 4956.31 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155421.837361 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44606.81 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 4460.68 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159882.518559 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40146.13 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 4014.61 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163897.131637 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36131.52 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3613.15 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167510.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32518.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171110.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28918.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174710.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25318.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178310.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21718.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181910.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18118.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185510.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14518.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189110.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10918.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192710.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7318.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196310.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3718.37 sec) 22:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199910.283408 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 118.37 sec) 22:27:00 [GPUGRID] [rr_sim] 2200028.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967526150.74G/439.78G) 22:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853889.56 22:27:00 [---] pick_jobs_to_run() start 22:27:00 [---] pick_jobs_to_run() end 22:27:00 [---] [rr_sim] end 22:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:27:00 [---] [cpu_sched_debug] preliminary job list: 22:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:27:00 [---] [cpu_sched_debug] final job list: 22:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:27:00 [---] [cpu_sched_debug] enforce_run_list: end 22:27:00 [---] choose_project(): 80820.000000 22:27:00 [WUProp@Home] doing RPC to get work 22:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354779.09 cpu 2104052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2104052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2104052.90 22:27:00 [---] Got 0 tasks 22:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:28:00 [---] [rr_sim] doing sim: CPU sched 22:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967498560.38G/439.78G) 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199965.91 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 219996.59 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219996.591246 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979969.32 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 197996.93 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417993.523367 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781972.39 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 178197.24 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596190.762276 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603775.15 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 160377.52 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756568.277295 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443397.64 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 144339.76 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900908.040811 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299057.87 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 129905.79 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030813.827976 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169152.08 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 116915.21 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147729.036424 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052236.88 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 105223.69 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252952.724028 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 947013.19 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 94701.32 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347654.042871 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852311.87 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 85231.19 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432885.229830 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767080.68 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 76708.07 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509593.298093 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690372.61 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 69037.26 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578630.559529 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621335.35 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 62133.54 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640764.094822 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559201.82 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 55920.18 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696684.276586 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503281.64 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 50328.16 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1747012.440173 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452953.47 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 45295.35 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792307.787402 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407658.13 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 40765.81 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833073.599908 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366892.31 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 36689.23 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869762.831163 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330203.08 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 33020.31 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902783.139292 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297182.77 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 29718.28 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932501.416609 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267464.50 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 26746.45 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959247.866194 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240718.05 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 24071.80 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983319.670821 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216646.24 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 21664.62 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004984.294984 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194981.62 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 19498.16 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024482.456732 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175483.46 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 17548.35 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2042030.802305 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157935.11 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 15793.51 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057824.313320 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142141.60 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 14214.16 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2072038.473234 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127927.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 12792.74 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084831.217157 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115134.70 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 11513.47 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096344.686687 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103621.23 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 10362.12 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106706.809264 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93259.10 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 9325.91 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116032.719584 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83933.19 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 8393.32 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124426.038871 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75539.87 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 7553.99 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131980.026230 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67985.89 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 6798.59 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138778.614853 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61187.30 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 6118.73 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144897.344613 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55068.57 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 5506.86 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150404.201398 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49561.71 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 4956.17 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155360.372504 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44605.54 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 4460.55 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159820.926500 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40144.99 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 4014.50 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163835.425096 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36130.49 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3613.05 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167448.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32517.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171048.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28917.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174648.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25317.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178248.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21717.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181848.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18117.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185448.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14517.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189048.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10917.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192648.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7317.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196248.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3717.44 sec) 22:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199848.473832 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 117.44 sec) 22:28:00 [GPUGRID] [rr_sim] 2199965.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967498560.38G/439.78G) 22:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853886.83 22:28:00 [---] pick_jobs_to_run() start 22:28:00 [---] pick_jobs_to_run() end 22:28:00 [---] [rr_sim] end 22:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:28:00 [---] [cpu_sched_debug] preliminary job list: 22:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:28:00 [---] [cpu_sched_debug] final job list: 22:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:28:00 [---] [cpu_sched_debug] enforce_run_list: end 22:28:00 [---] choose_project(): 80880.000000 22:28:00 [WUProp@Home] doing RPC to get work 22:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354719.09 cpu 2103992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103992.90 22:28:00 [---] Got 0 tasks 22:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:29:00 [---] [rr_sim] doing sim: CPU sched 22:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967470970.03G/439.78G) 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199903.18 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 219990.32 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219990.317558 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979912.86 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 197991.29 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417981.603360 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781921.57 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 178192.16 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596173.760581 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603729.41 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 160372.94 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756546.702081 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443356.47 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 144335.65 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900882.349431 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1299020.83 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 129902.08 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030784.432045 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169118.74 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 116911.87 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147696.306398 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052206.87 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 105220.69 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252916.993316 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946986.18 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 94698.62 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347615.611542 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852287.56 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 85228.76 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432844.367946 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767058.81 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 76705.88 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509550.248709 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690352.93 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 69035.29 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578585.541396 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621317.63 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 62131.76 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640717.304814 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559185.87 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 55918.59 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696635.891890 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503267.28 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 50326.73 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746962.620259 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452940.56 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 45294.06 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792256.675791 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407646.50 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 40764.65 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1833021.325769 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366881.85 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 36688.18 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869709.510750 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330193.66 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 33019.37 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902728.877233 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297174.30 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 29717.43 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932446.307067 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267456.87 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 26745.69 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959191.993918 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240711.18 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 24071.12 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983263.112084 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216640.06 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 21664.01 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004927.118433 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194976.06 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 19497.61 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024424.724148 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175478.45 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 17547.85 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041972.569291 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157930.61 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 15793.06 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057765.629919 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142137.55 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 14213.75 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071979.384485 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127923.79 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 12792.38 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084771.763594 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115131.41 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 11513.14 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096284.904793 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103618.27 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 10361.83 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106646.731871 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93256.44 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 9325.64 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115972.376242 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83930.80 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 8393.08 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124365.456175 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75537.72 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 7553.77 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131919.228115 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67983.95 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 6798.39 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138717.622861 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61185.55 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 6118.56 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144836.178133 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55067.00 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 5506.70 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150342.877877 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49560.30 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 4956.03 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155298.907647 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44604.27 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 4460.43 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159759.334440 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40143.84 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 4014.38 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163773.718554 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36129.46 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3612.95 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167386.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32516.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170986.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28916.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174586.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25316.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178186.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21716.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181786.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18116.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185386.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14516.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188986.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10916.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192586.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7316.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196186.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3716.51 sec) 22:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199786.664256 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 116.51 sec) 22:29:00 [GPUGRID] [rr_sim] 2199903.18: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967470970.03G/439.78G) 22:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853884.09 22:29:00 [---] pick_jobs_to_run() start 22:29:00 [---] pick_jobs_to_run() end 22:29:00 [---] [rr_sim] end 22:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:29:00 [---] [cpu_sched_debug] preliminary job list: 22:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:29:00 [---] [cpu_sched_debug] final job list: 22:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:29:00 [---] [cpu_sched_debug] enforce_run_list: end 22:29:00 [---] choose_project(): 80940.000000 22:29:00 [WUProp@Home] doing RPC to get work 22:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354659.09 cpu 2103932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103932.90 22:29:00 [---] Got 0 tasks 22:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:30:00 [---] [rr_sim] doing sim: CPU sched 22:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967443379.67G/439.78G) 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199840.44 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 219984.04 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219984.043869 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979856.39 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 197985.64 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417969.683352 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781870.76 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 178187.08 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596156.758886 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603683.68 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 160368.37 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756525.126867 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443315.31 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 144331.53 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900856.658050 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298983.78 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 129898.38 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030755.036114 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169085.40 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 116908.54 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147663.576372 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052176.86 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 105217.69 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252881.262605 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946959.18 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 94695.92 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347577.180214 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852263.26 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 85226.33 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432803.506062 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767036.93 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 76703.69 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509507.199325 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690333.24 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 69033.32 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578540.523262 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621299.92 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 62129.99 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640670.514805 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559169.92 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 55916.99 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696587.507194 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503252.93 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 50325.29 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746912.800344 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452927.64 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 45292.76 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792205.564179 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407634.87 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 40763.49 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832969.051631 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366871.39 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 36687.14 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869656.190337 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330184.25 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 33018.42 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902674.615173 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297165.82 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 29716.58 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932391.197525 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267449.24 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 26744.92 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959136.121642 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240704.32 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 24070.43 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983206.553347 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216633.89 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 21663.39 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004869.941882 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194970.50 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 19497.05 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024366.991563 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175473.45 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 17547.34 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041914.336277 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157926.10 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 15792.61 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057706.946518 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142133.49 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 14213.35 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071920.295736 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127920.14 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 12792.01 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084712.310032 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115128.13 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 11512.81 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096225.122898 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103615.32 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 10361.53 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106586.654478 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93253.78 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 9325.38 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115912.032900 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83928.41 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 8392.84 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124304.873479 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75535.57 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 7553.56 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131858.430001 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67982.01 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 6798.20 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138656.630870 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61183.81 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 6118.38 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144775.011653 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55065.43 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 5506.54 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150281.554357 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49558.88 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 4955.89 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155237.442791 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44603.00 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 4460.30 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159697.742381 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40142.70 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 4014.27 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163712.012012 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36128.43 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3612.84 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167324.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32515.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170924.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28915.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174524.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25315.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178124.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21715.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181724.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18115.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185324.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14515.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188924.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10915.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192524.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7315.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196124.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3715.58 sec) 22:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199724.854681 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115.58 sec) 22:30:00 [GPUGRID] [rr_sim] 2199840.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967443379.67G/439.78G) 22:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853881.35 22:30:00 [---] pick_jobs_to_run() start 22:30:00 [---] pick_jobs_to_run() end 22:30:00 [---] [rr_sim] end 22:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:30:00 [---] [cpu_sched_debug] preliminary job list: 22:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:30:00 [---] [cpu_sched_debug] final job list: 22:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:30:00 [---] [cpu_sched_debug] enforce_run_list: end 22:30:00 [---] choose_project(): 81000.000000 22:30:00 [WUProp@Home] doing RPC to get work 22:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354599.09 cpu 2103872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103872.90 22:30:00 [---] Got 0 tasks 22:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:31:00 [---] [rr_sim] doing sim: CPU sched 22:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967415789.31G/439.78G) 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199777.70 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 219977.77 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219977.770181 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979799.93 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 197979.99 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417957.763344 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781819.94 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 178181.99 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596139.757191 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603637.94 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 160363.79 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756503.551653 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443274.15 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 144327.42 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900830.966669 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298946.74 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 129894.67 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030725.640184 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169052.06 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 116905.21 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147630.846347 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052146.86 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 105214.69 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252845.531893 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946932.17 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 94693.22 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347538.748885 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852238.95 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 85223.90 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432762.644178 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 767015.06 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 76701.51 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509464.149941 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690313.55 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 69031.36 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578495.505128 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621282.20 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 62128.22 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640623.724797 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559153.98 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 55915.40 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696539.122498 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503238.58 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 50323.86 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746862.980430 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452914.72 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 45291.47 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792154.452568 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407623.25 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 40762.32 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832916.777493 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366860.92 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 36686.09 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869602.869925 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330174.83 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 33017.48 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902620.353113 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297157.35 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 29715.73 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932336.087983 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267441.61 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 26744.16 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959080.249366 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240697.45 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 24069.75 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983149.994611 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216627.71 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 21662.77 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004812.765331 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194964.94 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 19496.49 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024309.258979 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175468.44 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 17546.84 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041856.103263 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157921.60 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 15792.16 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057648.263118 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142129.44 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 14212.94 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071861.206987 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127916.49 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 12791.65 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084652.856470 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115124.85 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 11512.48 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096165.341004 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103612.36 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 10361.24 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106526.577085 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93251.12 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 9325.11 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115851.689558 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83926.01 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 8392.60 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124244.290783 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75533.41 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 7553.34 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131797.631886 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67980.07 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 6798.01 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138595.638879 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61182.06 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 6118.21 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144713.845172 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55063.86 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 5506.39 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150220.230836 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49557.47 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 4955.75 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155175.977934 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44601.72 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 4460.17 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159636.150322 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40141.55 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 4014.16 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163650.305471 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36127.40 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3612.74 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167263.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32514.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170863.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28914.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174463.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25314.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178063.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21714.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181663.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18114.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185263.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14514.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188863.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10914.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192463.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7314.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196063.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3714.66 sec) 22:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199663.045105 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114.66 sec) 22:31:00 [GPUGRID] [rr_sim] 2199777.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967415789.31G/439.78G) 22:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853878.61 22:31:00 [---] pick_jobs_to_run() start 22:31:00 [---] pick_jobs_to_run() end 22:31:00 [---] [rr_sim] end 22:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:31:00 [---] [cpu_sched_debug] preliminary job list: 22:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:31:00 [---] [cpu_sched_debug] final job list: 22:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:31:00 [---] [cpu_sched_debug] enforce_run_list: end 22:31:00 [---] choose_project(): 81060.000000 22:31:00 [WUProp@Home] doing RPC to get work 22:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354539.09 cpu 2103812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103812.90 22:31:00 [---] Got 0 tasks 22:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:32:00 [---] [rr_sim] doing sim: CPU sched 22:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967388198.96G/439.78G) 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199714.96 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 219971.50 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219971.496493 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979743.47 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 197974.35 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417945.843337 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781769.12 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 178176.91 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596122.755496 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603592.21 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 160359.22 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756481.976440 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443232.99 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 144323.30 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900805.275289 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298909.69 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 129890.97 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030696.244253 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1169018.72 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 116901.87 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147598.116321 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052116.85 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 105211.68 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252809.801182 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946905.16 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 94690.52 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347500.317557 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852214.65 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 85221.46 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432721.782294 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766993.18 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 76699.32 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509421.100558 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690293.86 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 69029.39 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578450.486995 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621264.48 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 62126.45 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640576.934788 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559138.03 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 55913.80 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696490.737803 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503224.23 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 50322.42 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746813.160515 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452901.80 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 45290.18 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792103.340957 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407611.62 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 40761.16 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832864.503354 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366850.46 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 36685.05 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869549.549512 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330165.42 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 33016.54 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902566.091054 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297148.87 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 29714.89 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932280.978441 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267433.99 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 26743.40 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1959024.377090 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240690.59 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 24069.06 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983093.435874 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216621.53 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 21662.15 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004755.588780 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194959.38 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 19495.94 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024251.526395 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175463.44 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 17546.34 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041797.870249 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157917.09 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 15791.71 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057589.579717 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142125.39 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 14212.54 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071802.118238 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127912.85 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 12791.28 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084593.402907 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115121.56 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 11512.16 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096105.559110 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103609.41 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 10360.94 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106466.499692 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93248.47 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 9324.85 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115791.346216 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83923.62 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 8392.36 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124183.708087 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75531.26 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 7553.13 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131736.833771 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67978.13 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 6797.81 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138534.646887 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61180.32 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 6118.03 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144652.678692 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55062.29 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 5506.23 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150158.907316 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49556.06 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 4955.61 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155114.513077 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44600.45 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 4460.05 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159574.558262 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40140.41 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 4014.04 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163588.598929 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36126.37 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3612.64 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167201.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32513.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170801.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28913.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174401.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25313.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178001.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21713.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181601.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18113.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185201.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14513.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188801.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10913.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192401.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7313.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196001.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3713.73 sec) 22:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199601.235529 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 113.73 sec) 22:32:00 [GPUGRID] [rr_sim] 2199714.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967388198.96G/439.78G) 22:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853875.88 22:32:00 [---] pick_jobs_to_run() start 22:32:00 [---] pick_jobs_to_run() end 22:32:00 [---] [rr_sim] end 22:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:32:00 [---] [cpu_sched_debug] preliminary job list: 22:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:32:00 [---] [cpu_sched_debug] final job list: 22:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:32:00 [---] [cpu_sched_debug] enforce_run_list: end 22:32:00 [---] choose_project(): 81120.000000 22:32:00 [WUProp@Home] doing RPC to get work 22:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354479.09 cpu 2103752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103752.90 22:32:00 [---] Got 0 tasks 22:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:33:00 [---] [rr_sim] doing sim: CPU sched 22:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967360608.60G/439.78G) 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199652.23 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 219965.22 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219965.222805 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979687.01 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 197968.70 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417933.923329 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781718.30 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 178171.83 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596105.753801 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603546.47 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 160354.65 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756460.401226 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443191.83 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 144319.18 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900779.583908 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298872.64 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 129887.26 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030666.848322 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168985.38 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 116898.54 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147565.386295 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052086.84 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 105208.68 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252774.070470 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946878.16 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 94687.82 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347461.886228 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852190.34 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 85219.03 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432680.920410 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766971.31 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 76697.13 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509378.051174 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690274.18 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 69027.42 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578405.468861 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621246.76 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 62124.68 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640530.144780 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559122.08 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 55912.21 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696442.353107 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503209.87 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 50320.99 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746763.340601 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452888.89 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 45288.89 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792052.229346 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407600.00 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 40760.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832812.229216 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366840.00 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 36684.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869496.229099 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330156.00 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 33015.60 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902511.828994 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297140.40 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 29714.04 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932225.868900 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267426.36 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 26742.64 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958968.504814 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240683.72 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 24068.37 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1983036.877138 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216615.35 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 21661.54 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004698.412229 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194953.82 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 19495.38 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024193.793811 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175458.43 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 17545.84 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041739.637235 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157912.59 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 15791.26 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057530.896316 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142121.33 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 14212.13 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071743.029489 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127909.20 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 12790.92 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084533.949345 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115118.28 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 11511.83 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2096045.777215 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103606.45 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 10360.65 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106406.422299 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93245.81 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 9324.58 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115731.002874 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83921.23 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 8392.12 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124123.125391 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75529.10 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 7552.91 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131676.035657 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67976.19 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 6797.62 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138473.654896 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61178.57 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 6117.86 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144591.512211 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55060.72 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 5506.07 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150097.583795 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49554.64 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 4955.46 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155053.048220 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44599.18 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 4459.92 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159512.966203 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40139.26 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 4013.93 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163526.892388 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36125.34 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3612.53 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167139.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32512.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170739.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28912.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174339.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25312.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177939.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21712.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181539.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18112.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185139.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14512.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188739.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10912.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192339.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7312.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195939.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3712.80 sec) 22:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199539.425954 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 112.80 sec) 22:33:00 [GPUGRID] [rr_sim] 2199652.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967360608.60G/439.78G) 22:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853873.14 22:33:00 [---] pick_jobs_to_run() start 22:33:00 [---] pick_jobs_to_run() end 22:33:00 [---] [rr_sim] end 22:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:33:00 [---] [cpu_sched_debug] preliminary job list: 22:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:33:00 [---] [cpu_sched_debug] final job list: 22:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:33:00 [---] [cpu_sched_debug] enforce_run_list: end 22:33:00 [---] choose_project(): 81180.000000 22:33:00 [WUProp@Home] doing RPC to get work 22:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354419.09 cpu 2103692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103692.90 22:33:00 [---] Got 0 tasks 22:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:35:00 [---] [rr_sim] doing sim: CPU sched 22:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967333018.25G/439.78G) 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199589.49 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 219958.95 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219958.949117 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979630.54 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 197963.05 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417922.003322 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781667.49 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 178166.75 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596088.752106 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603500.74 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 160350.07 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756438.826012 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443150.67 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 144315.07 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900753.892527 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298835.60 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 129883.56 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030637.452391 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168952.04 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 116895.20 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147532.656269 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052056.83 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 105205.68 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252738.339759 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946851.15 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 94685.12 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347423.454899 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852166.04 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 85216.60 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432640.058526 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766949.43 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 76694.94 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509335.001790 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690254.49 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 69025.45 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578360.450728 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621229.04 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 62122.90 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640483.354771 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559106.14 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 55910.61 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696393.968411 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503195.52 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 50319.55 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746713.520686 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452875.97 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 45287.60 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1792001.117734 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407588.37 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 40758.84 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832759.955078 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366829.54 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 36682.95 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869442.908686 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330146.58 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 33014.66 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902457.566934 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297131.92 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 29713.19 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932170.759358 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267418.73 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 26741.87 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958912.632538 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240676.86 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 24067.69 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982980.318401 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216609.17 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 21660.92 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004641.235678 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194948.26 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 19494.83 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024136.061227 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175453.43 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 17545.34 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041681.404221 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157908.09 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 15790.81 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057472.212915 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142117.28 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 14211.73 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071683.940740 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127905.55 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 12790.56 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084474.495783 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115115.00 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 11511.50 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095985.995321 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103603.50 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 10360.35 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106346.344906 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93243.15 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 9324.31 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115670.659532 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83918.83 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 8391.88 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124062.542695 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75526.95 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 7552.69 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131615.237542 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67974.25 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 6797.43 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138412.662905 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61176.83 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 6117.68 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144530.345731 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55059.15 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 5505.91 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150036.260274 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49553.23 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 4955.32 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154991.583364 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44597.91 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 4459.79 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159451.374144 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40138.12 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 4013.81 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163465.185846 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36124.31 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3612.43 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167077.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32511.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170677.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28911.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174277.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25311.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177877.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21711.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181477.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18111.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185077.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14511.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188677.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10911.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192277.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7311.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195877.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3711.87 sec) 22:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199477.616378 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 111.87 sec) 22:35:00 [GPUGRID] [rr_sim] 2199589.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967333018.25G/439.78G) 22:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853930.40 22:35:00 [---] pick_jobs_to_run() start 22:35:00 [---] pick_jobs_to_run() end 22:35:00 [---] [rr_sim] end 22:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:35:00 [---] [cpu_sched_debug] preliminary job list: 22:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:35:00 [---] [cpu_sched_debug] final job list: 22:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:35:00 [---] [cpu_sched_debug] enforce_run_list: end 22:35:00 [---] choose_project(): 81300.000000 22:35:00 [WUProp@Home] doing RPC to get work 22:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354299.09 cpu 2103632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103632.90 22:35:00 [---] Got 0 tasks 22:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:36:00 [---] [rr_sim] doing sim: CPU sched 22:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967305427.89G/439.78G) 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199526.75 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 219952.68 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219952.675428 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979574.08 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 197957.41 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417910.083314 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781616.67 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 178161.67 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596071.750411 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603455.00 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 160345.50 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756417.250798 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443109.50 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 144310.95 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900728.201147 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298798.55 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 129879.86 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030608.056461 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168918.70 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 116891.87 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147499.926243 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1052026.83 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 105202.68 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252702.609047 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946824.15 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 94682.41 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347385.023571 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852141.73 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 85214.17 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432599.196642 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766927.56 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 76692.76 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509291.952406 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690234.80 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 69023.48 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578315.432594 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621211.32 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 62121.13 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640436.564763 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559090.19 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 55909.02 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696345.583715 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503181.17 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 50318.12 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746663.700772 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452863.05 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 45286.31 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791950.006123 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407576.75 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 40757.67 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832707.680939 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366819.07 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 36681.91 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869389.588274 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330137.17 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 33013.72 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902403.304875 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297123.45 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 29712.34 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932115.649816 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267411.10 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 26741.11 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958856.760263 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240669.99 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 24067.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982923.759665 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216602.99 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 21660.30 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004584.059127 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194942.70 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 19494.27 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024078.328642 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175448.43 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 17544.84 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041623.171207 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157903.58 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 15790.36 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057413.529514 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142113.22 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 14211.32 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071624.851991 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127901.90 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 12790.19 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084415.042221 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115111.71 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 11511.17 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095926.213427 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103600.54 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 10360.05 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106286.267513 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93240.49 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 9324.05 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115610.316190 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83916.44 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 8391.64 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124001.959999 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75524.79 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 7552.48 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131554.439428 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67972.31 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 6797.23 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138351.670913 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61175.08 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 6117.51 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144469.179250 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55057.58 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 5505.76 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149974.936754 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49551.82 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 4955.18 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154930.118507 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44596.64 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 4459.66 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159389.782084 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40136.97 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 4013.70 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163403.479304 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36123.27 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3612.33 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167015.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32510.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170615.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28910.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174215.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25310.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177815.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21710.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181415.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18110.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185015.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14510.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188615.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10910.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192215.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7310.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195815.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3710.95 sec) 22:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199415.806802 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 110.95 sec) 22:36:00 [GPUGRID] [rr_sim] 2199526.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967305427.89G/439.78G) 22:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853927.67 22:36:00 [---] pick_jobs_to_run() start 22:36:00 [---] pick_jobs_to_run() end 22:36:00 [---] [rr_sim] end 22:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:36:00 [---] [cpu_sched_debug] preliminary job list: 22:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:36:00 [---] [cpu_sched_debug] final job list: 22:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:36:00 [---] [cpu_sched_debug] enforce_run_list: end 22:36:00 [---] choose_project(): 81360.000000 22:36:00 [WUProp@Home] doing RPC to get work 22:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354239.09 cpu 2103572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103572.90 22:36:00 [---] Got 0 tasks 22:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:37:00 [---] [rr_sim] doing sim: CPU sched 22:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967277837.53G/439.78G) 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199464.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 219946.40 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219946.401740 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979517.62 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 197951.76 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417898.163306 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781565.85 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 178156.59 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596054.748716 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603409.27 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 160340.93 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756395.675584 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443068.34 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 144306.83 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900702.509766 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298761.51 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 129876.15 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030578.660530 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168885.36 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 116888.54 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147467.196217 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051996.82 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 105199.68 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252666.878335 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946797.14 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 94679.71 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347346.592242 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852117.43 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 85211.74 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432558.334758 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766905.68 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 76690.57 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509248.903022 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690215.11 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 69021.51 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578270.414460 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621193.60 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 62119.36 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640389.774755 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559074.24 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 55907.42 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696297.199019 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503166.82 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 50316.68 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746613.880858 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452850.14 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 45285.01 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791898.894512 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407565.12 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 40756.51 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832655.406801 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366808.61 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 36680.86 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869336.267861 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330127.75 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 33012.77 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902349.042815 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297114.97 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 29711.50 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932060.540274 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267403.48 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 26740.35 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958800.887987 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240663.13 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 24066.31 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982867.200928 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216596.82 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 21659.68 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004526.882575 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194937.13 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 19493.71 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2024020.596058 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175443.42 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 17544.34 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041564.938193 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157899.08 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 15789.91 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057354.846113 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142109.17 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 14210.92 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071565.763242 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127898.25 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 12789.83 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084355.588658 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115108.43 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 11510.84 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095866.431533 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103597.59 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 10359.76 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106226.190120 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93237.83 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 9323.78 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115549.972848 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83914.04 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 8391.40 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123941.377303 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75522.64 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 7552.26 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131493.641313 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67970.38 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 6797.04 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138290.678922 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61173.34 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 6117.33 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144408.012770 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55056.00 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 5505.60 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149913.613233 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49550.40 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 4955.04 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154868.653650 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44595.36 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 4459.54 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159328.190025 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40135.83 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 4013.58 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163341.772763 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36122.24 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3612.22 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166953.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32510.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170553.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28910.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174153.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25310.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177753.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21710.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181353.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18110.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184953.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14510.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188553.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10910.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192153.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7310.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195753.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3710.02 sec) 22:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199353.997227 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 110.02 sec) 22:37:00 [GPUGRID] [rr_sim] 2199464.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967277837.53G/439.78G) 22:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853924.93 22:37:00 [---] pick_jobs_to_run() start 22:37:00 [---] pick_jobs_to_run() end 22:37:00 [---] [rr_sim] end 22:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:37:00 [---] [cpu_sched_debug] preliminary job list: 22:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:37:00 [---] [cpu_sched_debug] final job list: 22:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:37:00 [---] [cpu_sched_debug] enforce_run_list: end 22:37:00 [---] choose_project(): 81420.000000 22:37:00 [WUProp@Home] doing RPC to get work 22:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354179.09 cpu 2103512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103512.90 22:37:00 [---] Got 0 tasks 22:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:38:00 [---] [rr_sim] doing sim: CPU sched 22:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967250247.18G/439.78G) 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199401.28 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 219940.13 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219940.128052 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979461.15 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 197946.12 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417886.243299 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781515.04 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 178151.50 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596037.747021 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603363.53 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 160336.35 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756374.100371 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1443027.18 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 144302.72 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900676.818386 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298724.46 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 129872.45 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030549.264599 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168852.02 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 116885.20 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147434.466191 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051966.81 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 105196.68 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252631.147624 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946770.13 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 94677.01 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347308.160914 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852093.12 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 85209.31 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432517.472874 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766883.81 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 76688.38 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509205.853639 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690195.43 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 69019.54 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578225.396327 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621175.88 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 62117.59 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640342.984746 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559058.30 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 55905.83 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696248.814323 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503152.47 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 50315.25 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746564.060943 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452837.22 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 45283.72 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791847.782901 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407553.50 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 40755.35 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832603.132663 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366798.15 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 36679.81 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869282.947448 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330118.33 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 33011.83 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902294.780755 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297106.50 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 29710.65 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1932005.430732 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267395.85 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 26739.58 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958745.015711 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240656.26 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 24065.63 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982810.642192 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216590.64 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 21659.06 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004469.706024 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194931.57 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 19493.16 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023962.863474 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175438.42 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 17543.84 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041506.705178 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157894.58 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 15789.46 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057296.162713 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142105.12 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 14210.51 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071506.674493 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127894.61 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 12789.46 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084296.135096 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115105.15 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 11510.51 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095806.649638 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103594.63 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 10359.46 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106166.112726 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93235.17 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 9323.52 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115489.629506 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83911.65 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 8391.17 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123880.794607 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75520.49 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 7552.05 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131432.843198 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67968.44 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 6796.84 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138229.686931 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61171.59 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 6117.16 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144346.846289 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55054.43 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 5505.44 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149852.289712 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49548.99 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 4954.90 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154807.188793 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44594.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 4459.41 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159266.597966 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40134.68 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 4013.47 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163280.066221 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36121.21 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3612.12 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166892.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32509.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170492.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28909.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174092.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25309.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177692.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21709.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181292.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18109.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184892.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14509.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188492.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10909.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192092.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7309.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195692.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3709.09 sec) 22:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199292.187651 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 109.09 sec) 22:38:00 [GPUGRID] [rr_sim] 2199401.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967250247.18G/439.78G) 22:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853922.19 22:38:00 [---] pick_jobs_to_run() start 22:38:00 [---] pick_jobs_to_run() end 22:38:00 [---] [rr_sim] end 22:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:38:00 [---] [cpu_sched_debug] preliminary job list: 22:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:38:00 [---] [cpu_sched_debug] final job list: 22:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:38:00 [---] [cpu_sched_debug] enforce_run_list: end 22:38:00 [---] choose_project(): 81480.000000 22:38:00 [WUProp@Home] doing RPC to get work 22:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354119.09 cpu 2103452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103452.90 22:38:00 [---] Got 0 tasks 22:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:39:00 [---] [rr_sim] doing sim: CPU sched 22:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967222656.82G/439.78G) 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199338.54 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 219933.85 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219933.854364 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979404.69 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 197940.47 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417874.323291 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781464.22 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 178146.42 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596020.745326 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603317.80 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 160331.78 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756352.525157 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442986.02 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 144298.60 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900651.127005 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298687.42 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 129868.74 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030519.868668 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168818.67 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 116881.87 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147401.736165 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051936.81 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 105193.68 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252595.416912 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946743.13 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 94674.31 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347269.729585 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852068.81 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 85206.88 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432476.610990 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766861.93 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 76686.19 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509162.804255 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690175.74 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 69017.57 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578180.378193 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621158.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 62115.82 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640296.194738 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559042.35 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 55904.23 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696200.429628 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503138.11 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 50313.81 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746514.241029 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452824.30 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 45282.43 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791796.671289 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407541.87 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 40754.19 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832550.858524 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366787.69 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 36678.77 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869229.627036 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330108.92 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 33010.89 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902240.518696 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297098.02 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 29709.80 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931950.321190 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267388.22 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 26738.82 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958689.143435 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240649.40 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 24064.94 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982754.083455 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216584.46 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 21658.45 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004412.529473 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194926.01 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 19492.60 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023905.130890 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175433.41 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 17543.34 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041448.472164 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157890.07 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 15789.01 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057237.479312 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142101.06 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 14210.11 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071447.585744 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127890.96 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 12789.10 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084236.681534 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115101.86 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 11510.19 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095746.867744 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103591.68 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 10359.17 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106106.035333 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93232.51 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 9323.25 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115429.286164 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83909.26 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 8390.93 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123820.211911 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75518.33 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 7551.83 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131372.045084 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67966.50 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 6796.65 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138168.694939 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61169.85 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 6116.98 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144285.679809 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55052.86 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 5505.29 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149790.966192 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49547.58 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 4954.76 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154745.723936 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44592.82 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 4459.28 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159205.005907 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40133.54 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 4013.35 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163218.359680 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36120.18 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3612.02 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166830.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32508.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170430.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28908.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174030.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25308.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177630.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21708.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181230.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18108.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184830.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14508.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188430.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10908.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192030.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7308.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195630.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3708.17 sec) 22:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199230.378075 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 108.17 sec) 22:39:00 [GPUGRID] [rr_sim] 2199338.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967222656.82G/439.78G) 22:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853919.46 22:39:00 [---] pick_jobs_to_run() start 22:39:00 [---] pick_jobs_to_run() end 22:39:00 [---] [rr_sim] end 22:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:39:00 [---] [cpu_sched_debug] preliminary job list: 22:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:39:00 [---] [cpu_sched_debug] final job list: 22:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:39:00 [---] [cpu_sched_debug] enforce_run_list: end 22:39:00 [---] choose_project(): 81540.000000 22:39:00 [WUProp@Home] doing RPC to get work 22:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 354059.09 cpu 2103392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103392.90 22:39:00 [---] Got 0 tasks 22:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:40:00 [---] [rr_sim] doing sim: CPU sched 22:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967195066.46G/439.78G) 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199275.81 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 219927.58 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219927.580676 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979348.23 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 197934.82 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417862.403284 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781413.40 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 178141.34 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 596003.743631 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603272.06 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 160327.21 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756330.949943 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442944.86 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 144294.49 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900625.435624 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298650.37 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 129865.04 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030490.472737 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168785.33 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 116878.53 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147369.006139 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051906.80 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 105190.68 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252559.686201 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946716.12 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 94671.61 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347231.298256 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852044.51 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 85204.45 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432435.749106 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766840.06 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 76684.01 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509119.754871 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690156.05 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 69015.61 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578135.360060 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621140.45 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 62114.04 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640249.404729 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559026.40 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 55902.64 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696152.044932 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503123.76 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 50312.38 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746464.421114 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452811.39 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 45281.14 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791745.559678 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407530.25 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 40753.02 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832498.584386 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366777.22 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 36677.72 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869176.306623 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330099.50 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 33009.95 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902186.256636 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297089.55 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 29708.96 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931895.211648 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267380.60 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 26738.06 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958633.271159 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240642.54 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 24064.25 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982697.524718 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216578.28 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 21657.83 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004355.352922 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194920.45 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 19492.05 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023847.398305 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175428.41 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 17542.84 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041390.239150 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157885.57 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 15788.56 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057178.795911 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142097.01 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 14209.70 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071388.496995 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127887.31 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 12788.73 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084177.227971 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115098.58 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 11509.86 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095687.085850 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103588.72 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 10358.87 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2106045.957940 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93229.85 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 9322.98 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115368.942822 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83906.86 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 8390.69 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123759.629215 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75516.18 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 7551.62 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131311.246969 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67964.56 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 6796.46 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138107.702948 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61168.10 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 6116.81 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144224.513329 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55051.29 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 5505.13 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149729.642671 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49546.16 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 4954.62 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154684.259080 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44591.55 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 4459.15 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159143.413847 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40132.39 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 4013.24 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163156.653138 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36119.15 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3611.92 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166768.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32507.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170368.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28907.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173968.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25307.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177568.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21707.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181168.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18107.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184768.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14507.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188368.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10907.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191968.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7307.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195568.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3707.24 sec) 22:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199168.568500 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 107.24 sec) 22:40:00 [GPUGRID] [rr_sim] 2199275.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967195066.46G/439.78G) 22:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853916.72 22:40:00 [---] pick_jobs_to_run() start 22:40:00 [---] pick_jobs_to_run() end 22:40:00 [---] [rr_sim] end 22:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:40:00 [---] [cpu_sched_debug] preliminary job list: 22:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:40:00 [---] [cpu_sched_debug] final job list: 22:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:40:00 [---] [cpu_sched_debug] enforce_run_list: end 22:40:00 [---] choose_project(): 81600.000000 22:40:00 [WUProp@Home] doing RPC to get work 22:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353999.09 cpu 2103332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103332.90 22:40:00 [---] Got 0 tasks 22:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:41:00 [---] [rr_sim] doing sim: CPU sched 22:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967167476.11G/439.78G) 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199213.07 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 219921.31 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219921.306987 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979291.76 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 197929.18 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417850.483276 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781362.59 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 178136.26 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595986.741936 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603226.33 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 160322.63 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756309.374729 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442903.70 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 144290.37 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900599.744244 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298613.33 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 129861.33 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030461.076807 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168751.99 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 116875.20 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147336.276113 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051876.79 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 105187.68 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252523.955489 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946689.11 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 94668.91 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347192.866928 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 852020.20 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 85202.02 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432394.887222 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766818.18 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 76681.82 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509076.705487 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690136.36 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 69013.64 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578090.341926 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621122.73 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 62112.27 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640202.614721 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 559010.46 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 55901.05 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696103.660236 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503109.41 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 50310.94 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746414.601200 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452798.47 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 45279.85 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791694.448067 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407518.62 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 40751.86 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832446.310248 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366766.76 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 36676.68 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869122.986210 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330090.08 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 33009.01 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902131.994576 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297081.08 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 29708.11 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931840.102106 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267372.97 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 26737.30 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958577.398883 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240635.67 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 24063.57 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982640.965982 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216572.10 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 21657.21 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004298.176371 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194914.89 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 19491.49 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023789.665721 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175423.40 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 17542.34 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041332.006136 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157881.06 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 15788.11 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057120.112510 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142092.96 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 14209.30 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071329.408246 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127883.66 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 12788.37 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084117.774409 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115095.30 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 11509.53 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095627.303956 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103585.77 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 10358.58 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105985.880547 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93227.19 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 9322.72 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115308.599480 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83904.47 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 8390.45 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123699.046519 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75514.02 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 7551.40 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131250.448855 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67962.62 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 6796.26 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138046.710956 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61166.36 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 6116.64 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144163.346848 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55049.72 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 5504.97 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149668.319151 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49544.75 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 4954.48 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154622.794223 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44590.28 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 4459.03 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159081.821788 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40131.25 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 4013.12 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163094.946596 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36118.12 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3611.81 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166706.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32506.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170306.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28906.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173906.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25306.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177506.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21706.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181106.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18106.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184706.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14506.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188306.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10906.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191906.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7306.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195506.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3706.31 sec) 22:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199106.758924 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 106.31 sec) 22:41:00 [GPUGRID] [rr_sim] 2199213.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967167476.11G/439.78G) 22:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853913.98 22:41:00 [---] pick_jobs_to_run() start 22:41:00 [---] pick_jobs_to_run() end 22:41:00 [---] [rr_sim] end 22:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:41:00 [---] [cpu_sched_debug] preliminary job list: 22:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:41:00 [---] [cpu_sched_debug] final job list: 22:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:41:00 [---] [cpu_sched_debug] enforce_run_list: end 22:41:00 [---] choose_project(): 81660.000000 22:41:00 [WUProp@Home] doing RPC to get work 22:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353939.09 cpu 2103272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103272.90 22:41:00 [---] Got 0 tasks 22:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:42:00 [---] [rr_sim] doing sim: CPU sched 22:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967139885.75G/439.78G) 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199150.33 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 219915.03 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219915.033299 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979235.30 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 197923.53 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417838.563268 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781311.77 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 178131.18 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595969.740241 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603180.59 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 160318.06 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756287.799516 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442862.53 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 144286.25 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900574.052863 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298576.28 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 129857.63 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030431.680876 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168718.65 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 116871.87 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147303.546087 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051846.79 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 105184.68 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252488.224778 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946662.11 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 94666.21 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347154.435599 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851995.90 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 85199.59 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432354.025338 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766796.31 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 76679.63 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1509033.656104 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690116.68 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 69011.67 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578045.323792 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621105.01 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 62110.50 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640155.824712 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558994.51 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 55899.45 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696055.275540 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503095.06 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 50309.51 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746364.781285 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452785.55 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 45278.56 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791643.336456 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407507.00 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 40750.70 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832394.036109 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366756.30 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 36675.63 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869069.665797 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330080.67 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 33008.07 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902077.732517 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297072.60 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 29707.26 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931784.992564 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267365.34 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 26736.53 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958521.526607 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240628.81 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 24062.88 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982584.407245 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216565.93 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 21656.59 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004240.999820 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194909.33 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 19490.93 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023731.933137 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175418.40 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 17541.84 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041273.773122 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157876.56 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 15787.66 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057061.429109 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142088.90 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 14208.89 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071270.319497 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127880.01 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 12788.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2084058.320847 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115092.01 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 11509.20 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095567.522061 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103582.81 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 10358.28 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105925.803154 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93224.53 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 9322.45 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115248.256138 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83902.08 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 8390.21 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123638.463823 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75511.87 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 7551.19 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131189.650740 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67960.68 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 6796.07 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137985.718965 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61164.61 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 6116.46 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144102.180368 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55048.15 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 5504.82 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149606.995630 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49543.34 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 4954.33 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154561.329366 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44589.00 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 4458.90 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159020.229729 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40130.10 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 4013.01 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163033.240055 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36117.09 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3611.71 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166644.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32505.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170244.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28905.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173844.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25305.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177444.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21705.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181044.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18105.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184644.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14505.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188244.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10905.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191844.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7305.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195444.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3705.38 sec) 22:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2199044.949348 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 105.38 sec) 22:42:00 [GPUGRID] [rr_sim] 2199150.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967139885.75G/439.78G) 22:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853911.25 22:42:00 [---] pick_jobs_to_run() start 22:42:00 [---] pick_jobs_to_run() end 22:42:00 [---] [rr_sim] end 22:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:42:00 [---] [cpu_sched_debug] preliminary job list: 22:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:42:00 [---] [cpu_sched_debug] final job list: 22:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:42:00 [---] [cpu_sched_debug] enforce_run_list: end 22:42:00 [---] choose_project(): 81720.000000 22:42:00 [WUProp@Home] doing RPC to get work 22:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353879.09 cpu 2103212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103212.90 22:42:00 [---] Got 0 tasks 22:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:43:00 [---] [rr_sim] doing sim: CPU sched 22:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967112295.40G/439.78G) 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199087.60 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 219908.76 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219908.759611 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979178.84 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 197917.88 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417826.643261 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781260.95 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 178126.10 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595952.738546 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603134.86 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 160313.49 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756266.224302 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442821.37 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 144282.14 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900548.361483 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298539.23 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 129853.92 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030402.284945 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168685.31 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 116868.53 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147270.816062 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051816.78 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 105181.68 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252452.494066 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946635.10 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 94663.51 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347116.004271 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851971.59 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 85197.16 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432313.163454 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766774.43 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 76677.44 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508990.606720 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690096.99 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 69009.70 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1578000.305659 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621087.29 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 62108.73 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640109.034704 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558978.56 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 55897.86 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1696006.890844 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503080.71 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 50308.07 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746314.961371 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452772.63 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 45277.26 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791592.224845 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407495.37 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 40749.54 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832341.761971 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366745.83 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 36674.58 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1869016.345385 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330071.25 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 33007.13 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1902023.470457 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297064.13 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 29706.41 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931729.883022 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267357.71 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 26735.77 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958465.654331 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240621.94 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 24062.19 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982527.848509 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216559.75 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 21655.97 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004183.823269 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194903.77 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 19490.38 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023674.200553 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175413.40 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 17541.34 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041215.540108 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157872.06 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 15787.21 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2057002.745708 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142084.85 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 14208.49 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071211.230748 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127876.37 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 12787.64 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083998.867285 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115088.73 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 11508.87 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095507.740167 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103579.86 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 10357.99 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105865.725761 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93221.87 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 9322.19 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115187.912796 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83899.68 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 8389.97 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123577.881127 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75509.71 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 7550.97 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131128.852625 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67958.74 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 6795.87 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137924.726974 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61162.87 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 6116.29 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144041.013887 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55046.58 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 5504.66 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149545.672109 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49541.92 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 4954.19 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154499.864509 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44587.73 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 4458.77 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158958.637669 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40128.96 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 4012.90 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162971.533513 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36116.06 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3611.61 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166583.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32504.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170183.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28904.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173783.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25304.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177383.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21704.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180983.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18104.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184583.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14504.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188183.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10904.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191783.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7304.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195383.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3704.46 sec) 22:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198983.139773 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 104.46 sec) 22:43:00 [GPUGRID] [rr_sim] 2199087.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967112295.40G/439.78G) 22:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853908.51 22:43:00 [---] pick_jobs_to_run() start 22:43:00 [---] pick_jobs_to_run() end 22:43:00 [---] [rr_sim] end 22:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:43:00 [---] [cpu_sched_debug] preliminary job list: 22:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:43:00 [---] [cpu_sched_debug] final job list: 22:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:43:00 [---] [cpu_sched_debug] enforce_run_list: end 22:43:00 [---] choose_project(): 81780.000000 22:43:00 [WUProp@Home] doing RPC to get work 22:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353819.09 cpu 2103152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103152.90 22:43:00 [---] Got 0 tasks 22:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:44:00 [---] [rr_sim] doing sim: CPU sched 22:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967084705.04G/439.78G) 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2199024.86 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 219902.49 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219902.485923 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979122.37 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 197912.24 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417814.723253 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781210.14 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 178121.01 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595935.736850 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603089.12 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 160308.91 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756244.649088 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442780.21 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 144278.02 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900522.670102 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298502.19 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 129850.22 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030372.889014 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168651.97 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 116865.20 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147238.086036 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051786.77 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 105178.68 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252416.763355 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946608.10 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 94660.81 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347077.572942 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851947.29 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 85194.73 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432272.301570 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766752.56 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 76675.26 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508947.557336 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690077.30 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 69007.73 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577955.287525 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621069.57 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 62106.96 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640062.244695 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558962.61 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 55896.26 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695958.506148 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503066.35 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 50306.64 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746265.141456 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452759.72 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 45275.97 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791541.113233 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407483.75 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 40748.37 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832289.487833 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366735.37 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 36673.54 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868963.024972 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330061.83 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 33006.18 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901969.208397 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297055.65 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 29705.57 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931674.773480 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267350.09 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 26735.01 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958409.782055 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240615.08 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 24061.51 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982471.289772 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216553.57 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 21655.36 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004126.646718 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194898.21 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 19489.82 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023616.467969 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175408.39 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 17540.84 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041157.307094 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157867.55 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 15786.76 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056944.062308 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142080.80 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 14208.08 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071152.142000 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127872.72 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 12787.27 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083939.413722 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115085.45 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 11508.54 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095447.958273 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103576.90 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 10357.69 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105805.648368 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93219.21 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 9321.92 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115127.569454 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83897.29 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 8389.73 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123517.298431 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75507.56 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 7550.76 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131068.054511 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67956.80 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 6795.68 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137863.734982 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61161.12 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 6116.11 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143979.847407 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55045.01 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 5504.50 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149484.348589 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49540.51 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 4954.05 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154438.399653 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44586.46 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 4458.65 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158897.045610 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40127.81 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 4012.78 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162909.826972 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36115.03 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3611.50 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166521.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32503.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170121.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28903.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173721.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25303.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177321.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21703.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180921.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18103.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184521.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14503.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188121.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10903.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191721.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7303.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195321.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3703.53 sec) 22:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198921.330197 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103.53 sec) 22:44:00 [GPUGRID] [rr_sim] 2199024.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967084705.04G/439.78G) 22:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853905.77 22:44:00 [---] pick_jobs_to_run() start 22:44:00 [---] pick_jobs_to_run() end 22:44:00 [---] [rr_sim] end 22:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:44:00 [---] [cpu_sched_debug] preliminary job list: 22:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:44:00 [---] [cpu_sched_debug] final job list: 22:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:44:00 [---] [cpu_sched_debug] enforce_run_list: end 22:44:00 [---] choose_project(): 81840.000000 22:44:00 [WUProp@Home] doing RPC to get work 22:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353759.09 cpu 2103092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103092.90 22:44:00 [---] Got 0 tasks 22:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:45:00 [---] [rr_sim] doing sim: CPU sched 22:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967057114.68G/439.78G) 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198962.12 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 219896.21 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219896.212234 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979065.91 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 197906.59 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417802.803245 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781159.32 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 178115.93 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595918.735155 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1603043.39 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 160304.34 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756223.073874 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442739.05 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 144273.90 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900496.978721 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298465.14 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 129846.51 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030343.493084 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168618.63 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 116861.86 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147205.356010 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051756.77 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 105175.68 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252381.032643 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946581.09 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 94658.11 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347039.141613 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851922.98 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 85192.30 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432231.439686 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766730.68 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 76673.07 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508904.507952 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690057.61 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 69005.76 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577910.269391 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621051.85 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 62105.19 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1640015.454687 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558946.67 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 55894.67 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695910.121453 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503052.00 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 50305.20 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746215.321542 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452746.80 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 45274.68 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791490.001622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407472.12 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 40747.21 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832237.213694 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366724.91 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 36672.49 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868909.704559 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330052.42 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 33005.24 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901914.946338 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297047.18 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 29704.72 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931619.663939 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267342.46 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 26734.25 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958353.909779 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240608.21 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 24060.82 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982414.731036 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216547.39 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 21654.74 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004069.470167 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194892.65 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 19489.27 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023558.735384 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175403.39 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 17540.34 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041099.074080 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157863.05 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 15786.30 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056885.378907 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142076.74 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 14207.67 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071093.053251 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127869.07 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 12786.91 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083879.960160 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115082.16 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 11508.22 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095388.176378 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103573.95 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 10357.39 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105745.570975 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93216.55 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 9321.66 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115067.226112 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83894.90 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 8389.49 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123456.715735 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75505.41 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 7550.54 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131007.256396 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67954.87 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 6795.49 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137802.742991 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61159.38 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 6115.94 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143918.680926 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55043.44 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 5504.34 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149423.025068 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49539.10 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 4953.91 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154376.934796 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44585.19 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 4458.52 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158835.453551 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40126.67 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 4012.67 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162848.120430 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36114.00 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3611.40 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166459.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32502.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170059.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28902.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173659.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25302.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177259.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21702.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180859.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18102.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184459.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14502.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188059.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10902.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191659.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7302.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195259.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3702.60 sec) 22:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198859.520622 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 102.60 sec) 22:45:00 [GPUGRID] [rr_sim] 2198962.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967057114.68G/439.78G) 22:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853903.04 22:45:00 [---] pick_jobs_to_run() start 22:45:00 [---] pick_jobs_to_run() end 22:45:00 [---] [rr_sim] end 22:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:45:00 [---] [cpu_sched_debug] preliminary job list: 22:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:45:00 [---] [cpu_sched_debug] final job list: 22:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:45:00 [---] [cpu_sched_debug] enforce_run_list: end 22:45:00 [---] choose_project(): 81900.000000 22:45:00 [WUProp@Home] doing RPC to get work 22:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353699.09 cpu 2103032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2103032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2103032.90 22:45:00 [---] Got 0 tasks 22:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:46:00 [---] [rr_sim] doing sim: CPU sched 22:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967029524.33G/439.78G) 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198899.39 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 219889.94 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219889.938546 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1979009.45 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 197900.94 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417790.883238 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781108.50 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 178110.85 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595901.733460 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602997.65 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 160299.77 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756201.498661 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442697.89 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 144269.79 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900471.287341 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298428.10 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 129842.81 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030314.097153 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168585.29 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 116858.53 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147172.625984 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051726.76 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 105172.68 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252345.301932 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946554.08 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 94655.41 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1347000.710285 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851898.68 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 85189.87 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432190.577802 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766708.81 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 76670.88 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508861.458568 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690037.93 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 69003.79 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577865.251258 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621034.13 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 62103.41 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639968.664678 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558930.72 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 55893.07 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695861.736757 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503037.65 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 50303.76 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746165.501627 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452733.88 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 45273.39 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791438.890011 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407460.50 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 40746.05 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832184.939556 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366714.45 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 36671.44 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868856.384147 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330043.00 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 33004.30 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901860.684278 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297038.70 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 29703.87 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931564.554397 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267334.83 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 26733.48 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958298.037503 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240601.35 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 24060.13 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982358.172299 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216541.21 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 21654.12 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2004012.293615 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194887.09 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 19488.71 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023501.002800 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175398.38 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 17539.84 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2041040.841066 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157858.54 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 15785.85 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056826.695506 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142072.69 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 14207.27 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2071033.964502 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127865.42 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 12786.54 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083820.506598 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115078.88 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 11507.89 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095328.394484 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103570.99 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 10357.10 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105685.493582 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93213.89 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 9321.39 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115006.882770 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83892.50 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 8389.25 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123396.133039 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75503.25 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 7550.33 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130946.458282 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67952.93 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 6795.29 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137741.751000 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61157.63 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 6115.76 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143857.514446 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55041.87 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 5504.19 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149361.701548 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49537.68 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 4953.77 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154315.469939 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44583.92 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 4458.39 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158773.861491 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40125.52 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 4012.55 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162786.413888 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36112.97 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3611.30 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166397.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32501.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169997.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28901.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173597.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25301.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177197.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21701.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180797.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18101.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184397.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14501.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187997.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10901.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191597.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7301.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195197.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3701.67 sec) 22:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198797.711046 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 101.67 sec) 22:46:00 [GPUGRID] [rr_sim] 2198899.39: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967029524.33G/439.78G) 22:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853900.30 22:46:00 [---] pick_jobs_to_run() start 22:46:00 [---] pick_jobs_to_run() end 22:46:00 [---] [rr_sim] end 22:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:46:00 [---] [cpu_sched_debug] preliminary job list: 22:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:46:00 [---] [cpu_sched_debug] final job list: 22:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:46:00 [---] [cpu_sched_debug] enforce_run_list: end 22:46:00 [---] choose_project(): 81960.000000 22:46:00 [WUProp@Home] doing RPC to get work 22:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353639.09 cpu 2102972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102972.90 22:46:00 [---] Got 0 tasks 22:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:47:00 [---] [rr_sim] doing sim: CPU sched 22:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (967001933.97G/439.78G) 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198836.65 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 219883.66 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219883.664858 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978952.98 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 197895.30 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417778.963230 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781057.69 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 178105.77 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595884.731765 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602951.92 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 160295.19 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756179.923447 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442656.73 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 144265.67 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900445.595960 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298391.05 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 129839.11 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030284.701222 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168551.95 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 116855.19 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147139.895958 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051696.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 105169.68 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252309.571220 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946527.08 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 94652.71 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346962.278956 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851874.37 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 85187.44 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432149.715919 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766686.93 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 76668.69 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508818.409185 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 690018.24 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 69001.82 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577820.233124 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 621016.42 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 62101.64 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639921.874670 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558914.77 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 55891.48 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695813.352061 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503023.30 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 50302.33 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746115.681713 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452720.97 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 45272.10 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791387.778400 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407448.87 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 40744.89 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832132.665418 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366703.98 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 36670.40 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868803.063734 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330033.58 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 33003.36 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901806.422219 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297030.23 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 29703.02 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931509.444855 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267327.20 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 26732.72 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958242.165227 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240594.48 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 24059.45 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982301.613563 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216535.04 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 21653.50 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003955.117064 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194881.53 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 19488.15 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023443.270216 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175393.38 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 17539.34 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040982.608052 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157854.04 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 15785.40 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056768.012105 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142068.64 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 14206.86 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070974.875753 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127861.77 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 12786.18 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083761.053035 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115075.60 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 11507.56 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095268.612590 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103568.04 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 10356.80 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105625.416189 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93211.23 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 9321.12 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114946.539428 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83890.11 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 8389.01 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123335.550343 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75501.10 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 7550.11 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130885.660167 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67950.99 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 6795.10 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137680.759008 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61155.89 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 6115.59 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143796.347965 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55040.30 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 5504.03 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149300.378027 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49536.27 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 4953.63 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154254.005082 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44582.64 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 4458.26 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158712.269432 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40124.38 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 4012.44 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162724.707347 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36111.94 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3611.19 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166335.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32500.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169935.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28900.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173535.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25300.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177135.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21700.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180735.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18100.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184335.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14500.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187935.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10900.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191535.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7300.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195135.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3700.75 sec) 22:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198735.901470 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 100.75 sec) 22:47:00 [GPUGRID] [rr_sim] 2198836.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (967001933.97G/439.78G) 22:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853897.56 22:47:00 [---] pick_jobs_to_run() start 22:47:00 [---] pick_jobs_to_run() end 22:47:00 [---] [rr_sim] end 22:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:47:00 [---] [cpu_sched_debug] preliminary job list: 22:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:47:00 [---] [cpu_sched_debug] final job list: 22:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:47:00 [---] [cpu_sched_debug] enforce_run_list: end 22:47:00 [---] choose_project(): 82020.000000 22:47:00 [WUProp@Home] doing RPC to get work 22:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353579.09 cpu 2102912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102912.90 22:47:00 [---] Got 0 tasks 22:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:48:00 [---] [rr_sim] doing sim: CPU sched 22:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966974343.62G/439.78G) 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198773.91 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 219877.39 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219877.391170 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978896.52 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 197889.65 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417767.043223 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1781006.87 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 178100.69 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595867.730070 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602906.18 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 160290.62 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756158.348233 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442615.56 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 144261.56 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900419.904579 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298354.01 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 129835.40 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030255.305291 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168518.61 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 116851.86 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147107.165932 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051666.75 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 105166.67 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252273.840509 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946500.07 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 94650.01 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346923.847628 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851850.06 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 85185.01 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432108.854035 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766665.06 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 76666.51 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508775.359801 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689998.55 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 68999.86 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577775.214991 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620998.70 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 62099.87 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639875.084661 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558898.83 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 55889.88 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695764.967365 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 503008.94 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 50300.89 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746065.861798 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452708.05 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 45270.80 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791336.666788 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407437.24 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 40743.72 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832080.391279 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366693.52 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 36669.35 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868749.743321 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330024.17 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 33002.42 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901752.160159 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297021.75 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 29702.18 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931454.335313 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267319.58 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 26731.96 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958186.292951 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240587.62 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 24058.76 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982245.054826 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216528.86 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 21652.89 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003897.940513 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194875.97 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 19487.60 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023385.537632 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175388.37 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 17538.84 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040924.375038 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157849.54 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 15784.95 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056709.328704 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142064.58 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 14206.46 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070915.787004 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127858.12 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 12785.81 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083701.599473 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115072.31 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 11507.23 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095208.830696 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103565.08 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 10356.51 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105565.338796 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93208.57 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 9320.86 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114886.196086 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83887.72 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 8388.77 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123274.967647 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75498.94 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 7549.89 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130824.862052 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67949.05 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 6794.90 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137619.767017 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61154.14 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 6115.41 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143735.181485 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55038.73 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 5503.87 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149239.054506 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49534.86 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 4953.49 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154192.540225 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44581.37 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 4458.14 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158650.677373 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40123.23 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 4012.32 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162663.000805 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36110.91 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3611.09 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166274.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32499.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169874.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28899.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173474.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25299.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177074.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21699.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180674.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18099.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184274.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14499.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187874.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10899.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191474.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7299.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195074.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3699.82 sec) 22:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198674.091895 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 99.82 sec) 22:48:00 [GPUGRID] [rr_sim] 2198773.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966974343.62G/439.78G) 22:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853894.82 22:48:00 [---] pick_jobs_to_run() start 22:48:00 [---] pick_jobs_to_run() end 22:48:00 [---] [rr_sim] end 22:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:48:00 [---] [cpu_sched_debug] preliminary job list: 22:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:48:00 [---] [cpu_sched_debug] final job list: 22:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:48:00 [---] [cpu_sched_debug] enforce_run_list: end 22:48:00 [---] choose_project(): 82080.000000 22:48:00 [WUProp@Home] doing RPC to get work 22:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353519.09 cpu 2102852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102852.90 22:48:00 [---] Got 0 tasks 22:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:49:00 [---] [rr_sim] doing sim: CPU sched 22:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966946753.26G/439.78G) 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198711.17 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 219871.12 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219871.117482 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978840.06 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 197884.01 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417755.123215 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780956.05 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 178095.61 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595850.728375 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602860.45 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 160286.04 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756136.773019 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442574.40 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 144257.44 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900394.213199 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298316.96 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 129831.70 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030225.909361 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168485.27 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 116848.53 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147074.435906 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051636.74 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 105163.67 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252238.109797 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946473.07 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 94647.31 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346885.416299 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851825.76 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 85182.58 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432067.992151 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766643.18 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 76664.32 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508732.310417 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689978.86 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 68997.89 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577730.196857 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620980.98 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 62098.10 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639828.294653 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558882.88 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 55888.29 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695716.582669 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502994.59 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 50299.46 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1746016.041884 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452695.13 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 45269.51 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791285.555177 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407425.62 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 40742.56 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1832028.117141 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366683.06 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 36668.31 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868696.422908 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330014.75 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 33001.48 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901697.898099 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297013.28 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 29701.33 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931399.225771 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267311.95 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 26731.19 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958130.420675 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240580.75 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 24058.08 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982188.496089 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216522.68 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 21652.27 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003840.763962 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194870.41 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 19487.04 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023327.805047 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175383.37 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 17538.34 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040866.142024 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157845.03 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 15784.50 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056650.645303 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142060.53 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 14206.05 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070856.698255 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127854.48 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 12785.45 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083642.145911 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115069.03 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 11506.90 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095149.048801 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103562.13 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 10356.21 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105505.261403 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93205.91 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 9320.59 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114825.852744 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83885.32 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 8388.53 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123214.384951 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75496.79 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 7549.68 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130764.063938 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67947.11 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 6794.71 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137558.775026 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61152.40 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 6115.24 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143674.015005 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55037.16 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 5503.72 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149177.730986 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49533.44 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 4953.34 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154131.075369 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44580.10 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 4458.01 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158589.085313 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40122.09 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 4012.21 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162601.294264 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36109.88 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3610.99 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166212.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32498.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169812.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28898.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173412.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25298.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177012.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21698.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180612.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18098.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184212.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14498.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187812.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10898.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191412.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7298.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195012.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3698.89 sec) 22:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198612.282319 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 98.89 sec) 22:49:00 [GPUGRID] [rr_sim] 2198711.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966946753.26G/439.78G) 22:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853892.09 22:49:00 [---] pick_jobs_to_run() start 22:49:00 [---] pick_jobs_to_run() end 22:49:00 [---] [rr_sim] end 22:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:49:00 [---] [cpu_sched_debug] preliminary job list: 22:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:49:00 [---] [cpu_sched_debug] final job list: 22:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:49:00 [---] [cpu_sched_debug] enforce_run_list: end 22:49:00 [---] choose_project(): 82140.000000 22:49:00 [WUProp@Home] doing RPC to get work 22:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353459.09 cpu 2102792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102792.90 22:49:00 [---] Got 0 tasks 22:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:50:00 [---] [rr_sim] doing sim: CPU sched 22:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966919162.90G/439.78G) 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198648.44 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 219864.84 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219864.843793 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978783.59 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 197878.36 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417743.203207 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780905.23 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 178090.52 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595833.726680 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602814.71 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 160281.47 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756115.197805 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442533.24 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 144253.32 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900368.521818 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298279.92 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 129827.99 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030196.513430 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168451.92 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 116845.19 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147041.705880 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051606.73 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 105160.67 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252202.379086 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946446.06 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 94644.61 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346846.984970 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851801.45 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 85180.15 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432027.130267 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766621.31 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 76662.13 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508689.261033 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689959.18 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 68995.92 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577685.178723 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620963.26 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 62096.33 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639781.504644 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558866.93 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 55886.69 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695668.197973 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502980.24 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 50298.02 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745966.221969 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452682.22 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 45268.22 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791234.443566 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407413.99 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 40741.40 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831975.843003 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366672.59 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 36667.26 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868643.102496 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330005.34 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 33000.53 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901643.636040 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297004.80 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 29700.48 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931344.116229 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267304.32 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 26730.43 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958074.548399 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240573.89 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 24057.39 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982131.937353 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216516.50 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 21651.65 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003783.587411 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194864.85 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 19486.49 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023270.072463 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175378.37 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 17537.84 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040807.909010 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157840.53 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 15784.05 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056591.961903 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142056.48 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 14205.65 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070797.609506 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127850.83 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 12785.08 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083582.692349 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115065.75 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 11506.57 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095089.266907 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103559.17 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 10355.92 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105445.184010 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93203.25 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 9320.33 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114765.509402 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83882.93 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 8388.29 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123153.802255 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75494.64 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 7549.46 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130703.265823 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67945.17 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 6794.52 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137497.783034 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61150.65 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 6115.07 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143612.848524 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55035.59 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 5503.56 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149116.407465 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49532.03 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 4953.20 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154069.610512 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44578.83 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 4457.88 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158527.493254 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40120.94 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 4012.09 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162539.587722 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36108.85 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3610.89 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166150.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32497.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169750.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28897.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173350.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25297.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176950.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21697.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180550.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18097.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184150.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14497.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187750.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10897.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191350.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7297.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194950.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3697.97 sec) 22:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198550.472743 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 97.97 sec) 22:50:00 [GPUGRID] [rr_sim] 2198648.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966919162.90G/439.78G) 22:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853889.35 22:50:00 [---] pick_jobs_to_run() start 22:50:00 [---] pick_jobs_to_run() end 22:50:00 [---] [rr_sim] end 22:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:50:00 [---] [cpu_sched_debug] preliminary job list: 22:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:50:00 [---] [cpu_sched_debug] final job list: 22:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:50:00 [---] [cpu_sched_debug] enforce_run_list: end 22:50:00 [---] choose_project(): 82200.000000 22:50:00 [WUProp@Home] doing RPC to get work 22:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353399.09 cpu 2102732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102732.90 22:50:00 [---] Got 0 tasks 22:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:54:00 [---] [rr_sim] doing sim: CPU sched 22:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966919162.90G/439.78G) 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198648.44 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 219864.84 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219864.843793 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978783.59 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 197878.36 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417743.203207 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780905.23 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 178090.52 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595833.726680 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602814.71 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 160281.47 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756115.197805 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442533.24 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 144253.32 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900368.521818 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298279.92 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 129827.99 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030196.513430 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168451.92 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 116845.19 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147041.705880 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051606.73 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 105160.67 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252202.379086 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946446.06 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 94644.61 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346846.984970 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851801.45 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 85180.15 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1432027.130267 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766621.31 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 76662.13 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508689.261033 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689959.18 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 68995.92 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577685.178723 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620963.26 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 62096.33 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639781.504644 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558866.93 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 55886.69 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695668.197973 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502980.24 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 50298.02 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745966.221969 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452682.22 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 45268.22 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791234.443566 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407413.99 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 40741.40 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831975.843003 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366672.59 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 36667.26 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868643.102496 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 330005.34 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 33000.53 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901643.636040 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 297004.80 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 29700.48 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931344.116229 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267304.32 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 26730.43 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958074.548399 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240573.89 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 24057.39 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982131.937353 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216516.50 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 21651.65 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003783.587411 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194864.85 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 19486.49 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023270.072463 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175378.37 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 17537.84 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040807.909010 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157840.53 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 15784.05 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056591.961903 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142056.48 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 14205.65 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070797.609506 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127850.83 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 12785.08 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083582.692349 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115065.75 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 11506.57 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095089.266907 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103559.17 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 10355.92 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105445.184010 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93203.25 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 9320.33 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114765.509402 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83882.93 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 8388.29 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123153.802255 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75494.64 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 7549.46 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130703.265823 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67945.17 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 6794.52 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137497.783034 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61150.65 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 6115.07 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143612.848524 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55035.59 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 5503.56 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149116.407465 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49532.03 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 4953.20 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154069.610512 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44578.83 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 4457.88 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158527.493254 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40120.94 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 4012.09 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162539.587722 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36108.85 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3610.89 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166150.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32497.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169750.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28897.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173350.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25297.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176950.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21697.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180550.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18097.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184150.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14497.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187750.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10897.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191350.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7297.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194950.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3697.97 sec) 22:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198550.472743 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 97.97 sec) 22:54:00 [GPUGRID] [rr_sim] 2198648.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966919162.90G/439.78G) 22:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854129.35 22:54:00 [---] pick_jobs_to_run() start 22:54:00 [---] pick_jobs_to_run() end 22:54:00 [---] [rr_sim] end 22:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:54:00 [---] [cpu_sched_debug] preliminary job list: 22:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:54:00 [---] [cpu_sched_debug] final job list: 22:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:54:00 [---] [cpu_sched_debug] enforce_run_list: end 22:54:00 [---] choose_project(): 82440.000000 22:54:00 [WUProp@Home] doing RPC to get work 22:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353159.09 cpu 2102732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102732.90 22:54:00 [---] Got 0 tasks 22:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:55:00 [---] [rr_sim] doing sim: CPU sched 22:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966891572.55G/439.78G) 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198585.70 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 219858.57 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219858.570105 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978727.13 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 197872.71 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417731.283200 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780854.42 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 178085.44 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595816.724985 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602768.98 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 160276.90 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756093.622592 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442492.08 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 144249.21 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900342.830438 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298242.87 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 129824.29 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030167.117499 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168418.58 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 116841.86 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147008.975854 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051576.73 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 105157.67 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252166.648374 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946419.05 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 94641.91 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346808.553642 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851777.15 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 85177.71 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431986.268383 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766599.43 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 76659.94 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508646.211650 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689939.49 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 68993.95 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577640.160590 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620945.54 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 62094.55 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639734.714636 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558850.99 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 55885.10 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695619.813278 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502965.89 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 50296.59 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745916.402055 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452669.30 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 45266.93 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791183.331955 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407402.37 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 40740.24 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831923.568864 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366662.13 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 36666.21 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868589.782083 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329995.92 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 32999.59 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901589.373980 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296996.33 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 29699.63 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931289.006687 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267296.69 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 26729.67 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958018.676123 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240567.02 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 24056.70 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982075.378616 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216510.32 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 21651.03 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003726.410860 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194859.29 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 19485.93 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023212.339879 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175373.36 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 17537.34 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040749.675996 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157836.03 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 15783.60 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056533.278502 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142052.42 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 14205.24 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070738.520757 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127847.18 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 12784.72 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083523.238786 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115062.46 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 11506.25 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095029.485013 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103556.22 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 10355.62 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105385.106617 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93200.59 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 9320.06 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114705.166060 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83880.53 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 8388.05 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123093.219559 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75492.48 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 7549.25 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130642.467708 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67943.23 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 6794.32 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137436.791043 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61148.91 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 6114.89 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143551.682044 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55034.02 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 5503.40 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149055.083944 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49530.62 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 4953.06 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154008.145655 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44577.56 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 4457.76 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158465.901195 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40119.80 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 4011.98 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162477.881180 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36107.82 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3610.78 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166088.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32497.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169688.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28897.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173288.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25297.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176888.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21697.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180488.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18097.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184088.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14497.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187688.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10897.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191288.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7297.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194888.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3697.04 sec) 22:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198488.663168 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 97.04 sec) 22:55:00 [GPUGRID] [rr_sim] 2198585.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966891572.55G/439.78G) 22:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854126.61 22:55:00 [---] pick_jobs_to_run() start 22:55:00 [---] pick_jobs_to_run() end 22:55:00 [---] [rr_sim] end 22:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:55:00 [---] [cpu_sched_debug] preliminary job list: 22:55:00 [---] [cpu_sched_debug] final job list: 22:55:00 [---] [cpu_sched_debug] using 0.00 out of 8 CPUs 22:55:00 [---] [work_fetch] Request work fetch: CPUs idle 22:55:00 [---] [cpu_sched_debug] enforce_run_list: end 22:55:00 [---] choose_project(): 82500.000000 22:55:00 [WUProp@Home] doing RPC to get work 22:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353099.09 cpu 2102672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102672.90 22:55:00 [---] Got 0 tasks 22:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:56:00 [---] [rr_sim] doing sim: CPU sched 22:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966891572.55G/439.78G) 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198585.70 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 219858.57 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219858.570105 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978727.13 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 197872.71 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417731.283200 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780854.42 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 178085.44 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595816.724985 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602768.98 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 160276.90 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756093.622592 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442492.08 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 144249.21 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900342.830438 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298242.87 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 129824.29 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030167.117499 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168418.58 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 116841.86 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1147008.975854 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051576.73 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 105157.67 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252166.648374 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946419.05 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 94641.91 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346808.553642 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851777.15 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 85177.71 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431986.268383 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766599.43 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 76659.94 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508646.211650 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689939.49 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 68993.95 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577640.160590 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620945.54 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 62094.55 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639734.714636 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558850.99 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 55885.10 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695619.813278 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502965.89 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 50296.59 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745916.402055 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452669.30 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 45266.93 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791183.331955 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407402.37 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 40740.24 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831923.568864 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366662.13 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 36666.21 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868589.782083 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329995.92 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 32999.59 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901589.373980 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296996.33 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 29699.63 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931289.006687 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267296.69 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 26729.67 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1958018.676123 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240567.02 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 24056.70 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982075.378616 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216510.32 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 21651.03 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003726.410860 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194859.29 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 19485.93 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023212.339879 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175373.36 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 17537.34 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040749.675996 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157836.03 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 15783.60 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056533.278502 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142052.42 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 14205.24 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070738.520757 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127847.18 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 12784.72 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083523.238786 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115062.46 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 11506.25 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2095029.485013 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103556.22 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 10355.62 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105385.106617 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93200.59 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 9320.06 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114705.166060 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83880.53 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 8388.05 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123093.219559 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75492.48 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 7549.25 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130642.467708 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67943.23 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 6794.32 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137436.791043 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61148.91 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 6114.89 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143551.682044 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55034.02 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 5503.40 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149055.083944 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49530.62 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 4953.06 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154008.145655 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44577.56 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 4457.76 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158465.901195 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40119.80 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 4011.98 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162477.881180 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36107.82 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3610.78 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166088.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32497.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169688.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28897.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173288.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25297.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176888.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21697.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180488.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18097.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184088.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14497.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187688.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10897.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191288.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7297.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194888.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3697.04 sec) 22:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198488.663168 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 97.04 sec) 22:56:00 [GPUGRID] [rr_sim] 2198585.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966891572.55G/439.78G) 22:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854186.61 22:56:00 [---] pick_jobs_to_run() start 22:56:00 [---] pick_jobs_to_run() end 22:56:00 [---] [rr_sim] end 22:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:56:00 [---] [cpu_sched_debug] preliminary job list: 22:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:56:00 [---] [cpu_sched_debug] final job list: 22:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 22:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:56:00 [GPUGRID] [task] Resuming task e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: FLOPS left 966891572.55G 22:56:00 [---] [cpu_sched_debug] enforce_run_list: end 22:56:00 [---] choose_project(): 82560.000000 22:56:00 [WUProp@Home] doing RPC to get work 22:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 353039.09 cpu 2102672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102672.90 22:56:00 [---] Got 0 tasks 22:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:57:00 [---] [rr_sim] doing sim: CPU sched 22:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966863982.19G/439.78G) 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198522.96 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 219852.30 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219852.296417 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978670.67 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 197867.07 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417719.363192 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780803.60 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 178080.36 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595799.723290 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602723.24 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 160272.32 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756072.047378 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442450.92 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 144245.09 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900317.139057 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298205.83 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 129820.58 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030137.721568 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168385.24 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 116838.52 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146976.245828 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051546.72 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 105154.67 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252130.917662 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946392.05 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 94639.20 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346770.122313 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851752.84 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 85175.28 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431945.406499 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766577.56 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 76657.76 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508603.162266 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689919.80 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 68991.98 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577595.142456 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620927.82 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 62092.78 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639687.924628 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558835.04 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 55883.50 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695571.428582 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502951.54 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 50295.15 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745866.582140 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452656.38 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 45265.64 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791132.220343 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407390.74 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 40739.07 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831871.294726 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366651.67 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 36665.17 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868536.461670 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329986.50 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 32998.65 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901535.111920 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296987.85 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 29698.79 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931233.897145 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267289.07 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 26728.91 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957962.803848 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240560.16 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 24056.02 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1982018.819880 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216504.14 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 21650.41 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003669.234309 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194853.73 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 19485.37 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023154.607295 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175368.36 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 17536.84 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040691.442982 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157831.52 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 15783.15 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056474.595101 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142048.37 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 14204.84 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070679.432008 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127843.53 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 12784.35 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083463.785224 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115059.18 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 11505.92 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094969.703118 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103553.26 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 10355.33 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105325.029224 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93197.93 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 9319.79 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114644.822718 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83878.14 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 8387.81 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123032.636863 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75490.33 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 7549.03 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130581.669594 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67941.29 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 6794.13 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137375.799051 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61147.17 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 6114.72 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143490.515563 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55032.45 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 5503.24 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148993.760424 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49529.20 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 4952.92 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153946.680798 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44576.28 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 4457.63 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158404.309135 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40118.66 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 4011.87 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162416.174639 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36106.79 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3610.68 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166026.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32496.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169626.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28896.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173226.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25296.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176826.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21696.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180426.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18096.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184026.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14496.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187626.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10896.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191226.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7296.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194826.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3696.11 sec) 22:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198426.853592 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 96.11 sec) 22:57:00 [GPUGRID] [rr_sim] 2198522.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966863982.19G/439.78G) 22:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854183.88 22:57:00 [---] pick_jobs_to_run() start 22:57:00 [---] pick_jobs_to_run() end 22:57:00 [---] [rr_sim] end 22:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:57:00 [---] [cpu_sched_debug] preliminary job list: 22:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 22:57:00 [---] [cpu_sched_debug] final job list: 22:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:57:00 [---] [cpu_sched_debug] enforce_run_list: end 22:57:00 [---] choose_project(): 82620.000000 22:57:00 [WUProp@Home] doing RPC to get work 22:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352979.09 cpu 2102612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102612.90 22:57:00 [---] Got 0 tasks 22:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:58:00 [---] [rr_sim] doing sim: CPU sched 22:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966836391.83G/439.78G) 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198460.23 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 219846.02 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219846.022729 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978614.20 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 197861.42 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417707.443185 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780752.78 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 178075.28 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595782.721595 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602677.51 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 160267.75 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756050.472164 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442409.76 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 144240.98 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900291.447676 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298168.78 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 129816.88 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030108.325637 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168351.90 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 116835.19 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146943.515802 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051516.71 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 105151.67 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252095.186951 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946365.04 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 94636.50 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346731.690985 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851728.54 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 85172.85 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431904.544615 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766555.68 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 76655.57 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508560.112882 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689900.11 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 68990.01 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577550.124323 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620910.10 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 62091.01 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639641.134619 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558819.09 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 55881.91 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695523.043886 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502937.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 50293.72 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745816.762226 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452643.47 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 45264.35 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791081.108732 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407379.12 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 40737.91 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831819.020588 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366641.21 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 36664.12 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868483.141258 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329977.09 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 32997.71 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901480.849861 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296979.38 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 29697.94 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931178.787603 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267281.44 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 26728.14 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957906.931572 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240553.30 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 24055.33 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981962.261143 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216497.97 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 21649.80 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003612.057758 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194848.17 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 19484.82 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023096.874711 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175363.35 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 17536.34 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040633.209968 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157827.02 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 15782.70 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056415.911700 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142044.32 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 14204.43 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070620.343259 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127839.88 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 12783.99 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083404.331662 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115055.90 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 11505.59 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094909.921224 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103550.31 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 10355.03 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105264.951831 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93195.28 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 9319.53 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114584.479376 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83875.75 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 8387.57 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122972.054167 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75488.17 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 7548.82 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130520.871479 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67939.36 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 6793.94 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137314.807060 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61145.42 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 6114.54 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143429.349083 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55030.88 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 5503.09 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148932.436903 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49527.79 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 4952.78 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153885.215942 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44575.01 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 4457.50 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158342.717076 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40117.51 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 4011.75 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162354.468097 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36105.76 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3610.58 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165965.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32495.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169565.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28895.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173165.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25295.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176765.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21695.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180365.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18095.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183965.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14495.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187565.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10895.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191165.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7295.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194765.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3695.18 sec) 22:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198365.044016 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 95.18 sec) 22:58:00 [GPUGRID] [rr_sim] 2198460.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966836391.83G/439.78G) 22:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854181.14 22:58:00 [---] pick_jobs_to_run() start 22:58:00 [---] pick_jobs_to_run() end 22:58:00 [---] [rr_sim] end 22:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:58:00 [---] [cpu_sched_debug] preliminary job list: 22:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:58:00 [---] [cpu_sched_debug] final job list: 22:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:58:00 [---] [cpu_sched_debug] enforce_run_list: end 22:58:00 [---] choose_project(): 82680.000000 22:58:00 [WUProp@Home] doing RPC to get work 22:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352919.09 cpu 2102552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102552.90 22:58:00 [---] Got 0 tasks 22:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 22:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 22:59:00 [---] [rr_sim] doing sim: CPU sched 22:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966808801.48G/439.78G) 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198397.49 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 219839.75 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219839.749041 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978557.74 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 197855.77 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417695.523177 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780701.97 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 178070.20 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595765.719900 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602631.77 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 160263.18 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756028.896950 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442368.59 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 144236.86 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900265.756296 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298131.73 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 129813.17 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030078.929707 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168318.56 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 116831.86 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146910.785777 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051486.70 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 105148.67 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252059.456239 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946338.03 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 94633.80 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346693.259656 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851704.23 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 85170.42 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431863.682731 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766533.81 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 76653.38 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508517.063498 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689880.43 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 68988.04 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577505.106189 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620892.38 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 62089.24 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639594.344611 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558803.15 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 55880.31 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695474.659190 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502922.83 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 50292.28 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745766.942312 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452630.55 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 45263.05 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1791029.997121 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407367.49 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 40736.75 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831766.746449 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366630.74 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 36663.07 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868429.820845 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329967.67 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 32996.77 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901426.587801 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296970.90 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 29697.09 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931123.678061 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267273.81 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 26727.38 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957851.059296 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240546.43 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 24054.64 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981905.702407 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216491.79 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 21649.18 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003554.881206 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194842.61 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 19484.26 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2023039.142126 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175358.35 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 17535.83 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040574.976954 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157822.51 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 15782.25 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056357.228299 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142040.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 14204.03 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070561.254510 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127836.24 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 12783.62 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083344.878099 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115052.61 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 11505.26 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094850.139330 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103547.35 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 10354.74 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105204.874437 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93192.62 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 9319.26 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114524.136034 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83873.35 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 8387.34 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122911.471471 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75486.02 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 7548.60 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130460.073365 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67937.42 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 6793.74 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137253.815069 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61143.68 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 6114.37 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143368.182602 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55029.31 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 5502.93 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148871.113383 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49526.38 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 4952.64 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153823.751085 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44573.74 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 4457.37 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158281.125017 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40116.37 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 4011.64 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162292.761556 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36104.73 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3610.47 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165903.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32494.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169503.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28894.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173103.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25294.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176703.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21694.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180303.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18094.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183903.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14494.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187503.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10894.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191103.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7294.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194703.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3694.26 sec) 22:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198303.234441 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 94.26 sec) 22:59:00 [GPUGRID] [rr_sim] 2198397.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966808801.48G/439.78G) 22:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854178.40 22:59:00 [---] pick_jobs_to_run() start 22:59:00 [---] pick_jobs_to_run() end 22:59:00 [---] [rr_sim] end 22:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 22:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 22:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 22:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 22:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 22:59:00 [---] [cpu_sched_debug] preliminary job list: 22:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:59:00 [---] [cpu_sched_debug] final job list: 22:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 22:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 22:59:00 [---] [cpu_sched_debug] enforce_run_list: end 22:59:00 [---] choose_project(): 82740.000000 22:59:00 [WUProp@Home] doing RPC to get work 22:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352859.09 cpu 2102492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102492.90 22:59:00 [---] Got 0 tasks 23:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:00:00 [---] [rr_sim] doing sim: CPU sched 23:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966781211.12G/439.78G) 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198334.75 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 219833.48 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219833.475352 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978501.28 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 197850.13 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417683.603169 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780651.15 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 178065.12 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595748.718205 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602586.04 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 160258.60 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 756007.321737 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442327.43 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 144232.74 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900240.064915 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298094.69 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 129809.47 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030049.533776 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168285.22 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 116828.52 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146878.055751 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051456.70 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 105145.67 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1252023.725528 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946311.03 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 94631.10 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346654.828327 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851679.93 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 85167.99 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431822.820847 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766511.93 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 76651.19 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508474.014115 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689860.74 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 68986.07 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577460.088055 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620874.67 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 62087.47 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639547.554602 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558787.20 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 55878.72 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695426.274494 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502908.48 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 50290.85 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745717.122397 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452617.63 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 45261.76 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790978.885510 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407355.87 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 40735.59 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831714.472311 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366620.28 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 36662.03 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868376.500432 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329958.25 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 32995.83 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901372.325741 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296962.43 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 29696.24 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931068.568519 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267266.19 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 26726.62 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957795.187020 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240539.57 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 24053.96 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981849.143670 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216485.61 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 21648.56 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003497.704655 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194837.05 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 19483.70 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022981.409542 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175353.34 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 17535.33 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040516.743940 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157818.01 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 15781.80 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056298.544898 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142036.21 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 14203.62 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070502.165761 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127832.59 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 12783.26 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083285.424537 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115049.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 11504.93 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094790.357436 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103544.40 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 10354.44 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105144.797044 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93189.96 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 9319.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114463.792692 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83870.96 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 8387.10 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122850.888775 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75483.86 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 7548.39 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130399.275250 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67935.48 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 6793.55 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137192.823077 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61141.93 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 6114.19 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143307.016122 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55027.74 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 5502.77 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148809.789862 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49524.96 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 4952.50 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153762.286228 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44572.47 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 4457.25 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158219.532958 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40115.22 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 4011.52 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162231.055014 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36103.70 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3610.37 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165841.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32493.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169441.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28893.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173041.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25293.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176641.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21693.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180241.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18093.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183841.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14493.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187441.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10893.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191041.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7293.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194641.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3693.33 sec) 23:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198241.424865 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93.33 sec) 23:00:00 [GPUGRID] [rr_sim] 2198334.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966781211.12G/439.78G) 23:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854175.67 23:00:00 [---] pick_jobs_to_run() start 23:00:00 [---] pick_jobs_to_run() end 23:00:00 [---] [rr_sim] end 23:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:00:00 [---] [cpu_sched_debug] preliminary job list: 23:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:00:00 [---] [cpu_sched_debug] final job list: 23:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:00:00 [---] [cpu_sched_debug] enforce_run_list: end 23:00:00 [---] choose_project(): 82800.000000 23:00:00 [WUProp@Home] doing RPC to get work 23:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352799.09 cpu 2102432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102432.90 23:00:00 [---] Got 0 tasks 23:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:01:00 [---] [rr_sim] doing sim: CPU sched 23:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966753620.77G/439.78G) 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198272.02 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 219827.20 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219827.201664 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978444.81 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 197844.48 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417671.683162 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780600.33 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 178060.03 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595731.716510 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602540.30 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 160254.03 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755985.746523 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442286.27 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 144228.63 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900214.373535 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298057.64 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 129805.76 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1030020.137845 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168251.88 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 116825.19 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146845.325725 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051426.69 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 105142.67 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251987.994816 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946284.02 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 94628.40 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346616.396999 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851655.62 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 85165.56 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431781.958963 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766490.06 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 76649.01 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508430.964731 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689841.05 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 68984.11 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577415.069922 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620856.95 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 62085.69 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639500.764594 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558771.25 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 55877.13 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695377.889798 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502894.13 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 50289.41 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745667.302483 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452604.71 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 45260.47 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790927.773898 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407344.24 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 40734.42 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831662.198173 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366609.82 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 36660.98 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868323.180019 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329948.84 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 32994.88 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901318.063682 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296953.95 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 29695.40 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1931013.458977 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267258.56 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 26725.86 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957739.314744 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240532.70 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 24053.27 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981792.584933 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216479.43 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 21647.94 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003440.528104 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194831.49 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 19483.15 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022923.676958 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175348.34 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 17534.83 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040458.510926 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157813.51 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 15781.35 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056239.861498 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142032.16 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 14203.22 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070443.077012 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127828.94 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 12782.89 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083225.970975 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115046.05 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 11504.60 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094730.575541 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103541.44 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 10354.14 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105084.719651 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93187.30 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 9318.73 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114403.449350 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83868.57 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 8386.86 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122790.306079 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75481.71 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 7548.17 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130338.477135 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67933.54 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 6793.35 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137131.831086 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61140.19 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 6114.02 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143245.849641 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55026.17 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 5502.62 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148748.466341 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49523.55 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 4952.36 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153700.821371 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44571.20 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 4457.12 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158157.940898 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40114.08 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 4011.41 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162169.348472 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36102.67 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3610.27 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165779.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32492.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169379.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28892.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172979.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25292.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176579.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21692.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180179.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18092.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183779.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14492.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187379.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10892.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190979.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7292.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194579.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3692.40 sec) 23:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198179.615289 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92.40 sec) 23:01:00 [GPUGRID] [rr_sim] 2198272.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966753620.77G/439.78G) 23:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854172.93 23:01:00 [---] pick_jobs_to_run() start 23:01:00 [---] pick_jobs_to_run() end 23:01:00 [---] [rr_sim] end 23:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:01:00 [---] [cpu_sched_debug] preliminary job list: 23:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:01:00 [---] [cpu_sched_debug] final job list: 23:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:01:00 [---] [cpu_sched_debug] enforce_run_list: end 23:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:02:00 [---] [rr_sim] doing sim: CPU sched 23:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966726030.41G/439.78G) 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198209.28 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 219820.93 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219820.927976 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978388.35 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 197838.84 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417659.763154 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780549.52 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 178054.95 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595714.714815 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602494.56 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 160249.46 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755964.171309 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442245.11 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 144224.51 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900188.682154 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1298020.60 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 129802.06 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029990.741914 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168218.54 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 116821.85 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146812.595699 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051396.68 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 105139.67 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251952.264105 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946257.02 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 94625.70 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346577.965670 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851631.31 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 85163.13 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431741.097079 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766468.18 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 76646.82 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508387.915347 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689821.36 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 68982.14 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577370.051788 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620839.23 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 62083.92 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639453.974585 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558755.31 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 55875.53 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695329.505102 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502879.77 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 50287.98 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745617.482568 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452591.80 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 45259.18 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790876.662287 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407332.62 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 40733.26 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831609.924034 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366599.36 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 36659.94 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868269.859607 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329939.42 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 32993.94 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901263.801622 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296945.48 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 29694.55 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930958.349436 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267250.93 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 26725.09 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957683.442468 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240525.84 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 24052.58 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981736.026197 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216473.25 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 21647.33 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003383.351553 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194825.93 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 19482.59 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022865.944374 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175343.34 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 17534.33 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040400.277912 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157809.00 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 15780.90 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056181.178097 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142028.10 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 14202.81 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070383.988263 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127825.29 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 12782.53 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083166.517413 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115042.76 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 11504.28 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094670.793647 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103538.49 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 10353.85 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2105024.642258 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93184.64 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 9318.46 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114343.106008 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83866.17 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 8386.62 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122729.723383 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75479.56 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 7547.96 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130277.679021 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67931.60 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 6793.16 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137070.839095 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61138.44 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 6113.84 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143184.683161 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55024.60 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 5502.46 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148687.142821 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49522.14 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 4952.21 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153639.356515 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44569.92 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 4456.99 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158096.348839 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40112.93 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 4011.29 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162107.641931 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36101.64 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3610.16 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165717.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32491.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169317.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28891.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172917.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25291.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176517.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21691.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180117.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18091.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183717.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14491.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187317.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10891.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190917.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7291.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194517.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3691.47 sec) 23:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198117.805714 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 91.47 sec) 23:02:00 [GPUGRID] [rr_sim] 2198209.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966726030.41G/439.78G) 23:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854170.19 23:02:00 [---] pick_jobs_to_run() start 23:02:00 [---] pick_jobs_to_run() end 23:02:00 [---] [rr_sim] end 23:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:02:00 [---] [cpu_sched_debug] preliminary job list: 23:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:02:00 [---] [cpu_sched_debug] final job list: 23:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:02:00 [---] [cpu_sched_debug] enforce_run_list: end 23:02:00 [---] choose_project(): 82920.000000 23:02:00 [WUProp@Home] doing RPC to get work 23:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352679.09 cpu 2102312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102312.90 23:02:00 [---] Got 0 tasks 23:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:03:00 [---] [rr_sim] doing sim: CPU sched 23:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966698440.05G/439.78G) 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198146.54 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 219814.65 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219814.654288 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978331.89 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 197833.19 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417647.843147 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780498.70 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 178049.87 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595697.713120 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602448.83 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 160244.88 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755942.596095 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442203.95 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 144220.39 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900162.990773 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297983.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 129798.36 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029961.345984 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168185.20 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 116818.52 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146779.865673 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051366.68 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 105136.67 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251916.533393 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946230.01 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 94623.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346539.534342 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851607.01 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 85160.70 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431700.235195 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766446.31 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 76644.63 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508344.865963 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689801.68 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 68980.17 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577325.033654 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620821.51 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 62082.15 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639407.184577 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558739.36 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 55873.94 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695281.120407 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502865.42 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 50286.54 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745567.662654 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452578.88 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 45257.89 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790825.550676 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407320.99 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 40732.10 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831557.649896 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366588.89 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 36658.89 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868216.539194 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329930.00 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 32993.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901209.539562 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296937.00 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 29693.70 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930903.239894 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267243.30 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 26724.33 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957627.570192 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240518.97 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 24051.90 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981679.467460 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216467.08 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 21646.71 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003326.175002 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194820.37 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 19482.04 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022808.211789 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175338.33 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 17533.83 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040342.044898 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157804.50 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 15780.45 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056122.494696 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142024.05 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 14202.40 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070324.899514 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127821.64 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 12782.16 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083107.063850 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115039.48 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 11503.95 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094611.011753 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103535.53 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 10353.55 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104964.564865 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93181.98 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 9318.20 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114282.762666 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83863.78 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 8386.38 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122669.140687 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75477.40 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 7547.74 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130216.880906 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67929.66 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 6792.97 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137009.847103 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61136.70 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 6113.67 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143123.516681 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55023.03 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 5502.30 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148625.819300 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49520.72 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 4952.07 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153577.891658 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44568.65 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 4456.87 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158034.756780 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40111.79 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 4011.18 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162045.935389 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36100.61 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3610.06 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165655.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32490.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169255.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28890.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172855.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25290.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176455.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21690.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180055.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18090.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183655.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14490.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187255.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10890.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190855.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7290.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194455.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3690.55 sec) 23:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2198055.996138 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 90.55 sec) 23:03:00 [GPUGRID] [rr_sim] 2198146.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966698440.05G/439.78G) 23:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854167.46 23:03:00 [---] pick_jobs_to_run() start 23:03:00 [---] pick_jobs_to_run() end 23:03:00 [---] [rr_sim] end 23:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:03:00 [---] [cpu_sched_debug] preliminary job list: 23:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:03:00 [---] [cpu_sched_debug] final job list: 23:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:03:00 [---] [cpu_sched_debug] enforce_run_list: end 23:03:00 [---] choose_project(): 82980.000000 23:03:00 [WUProp@Home] doing RPC to get work 23:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352619.09 cpu 2102252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102252.90 23:03:00 [---] Got 0 tasks 23:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:04:00 [---] [rr_sim] doing sim: CPU sched 23:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966670849.70G/439.78G) 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198083.81 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 219808.38 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219808.380599 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978275.43 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 197827.54 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417635.923139 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780447.88 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 178044.79 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595680.711424 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602403.09 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 160240.31 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755921.020881 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442162.79 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 144216.28 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900137.299393 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297946.51 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 129794.65 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029931.950053 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168151.86 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 116815.19 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146747.135647 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051336.67 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 105133.67 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251880.802682 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946203.00 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 94620.30 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346501.103013 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851582.70 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 85158.27 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431659.373311 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766424.43 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 76642.44 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508301.816579 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689781.99 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 68978.20 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577280.015521 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620803.79 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 62080.38 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639360.394568 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558723.41 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 55872.34 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695232.735711 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502851.07 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 50285.11 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745517.842739 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452565.96 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 45256.60 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790774.439065 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407309.37 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 40730.94 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831505.375758 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366578.43 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 36657.84 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868163.218781 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329920.59 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 32992.06 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901155.277503 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296928.53 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 29692.85 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930848.130352 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267235.68 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 26723.57 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957571.697916 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240512.11 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 24051.21 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981622.908724 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216460.90 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 21646.09 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003268.998451 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194814.81 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 19481.48 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022750.479205 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175333.33 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 17533.33 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040283.811884 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157799.99 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 15780.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056063.811295 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142019.99 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 14202.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070265.810765 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127818.00 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 12781.80 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2083047.610288 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115036.20 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 11503.62 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094551.229859 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103532.58 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 10353.26 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104904.487472 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93179.32 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 9317.93 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114222.419324 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83861.39 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 8386.14 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122608.557991 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75475.25 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 7547.52 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130156.082792 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67927.72 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 6792.77 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136948.855112 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61134.95 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 6113.50 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143062.350200 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55021.46 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 5502.15 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148564.495780 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49519.31 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 4951.93 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153516.426801 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44567.38 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 4456.74 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157973.164720 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40110.64 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 4011.06 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161984.228848 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36099.58 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3609.96 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165594.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32489.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169194.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28889.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172794.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25289.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176394.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21689.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179994.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18089.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183594.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14489.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187194.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10889.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190794.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7289.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194394.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3689.62 sec) 23:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197994.186562 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 89.62 sec) 23:04:00 [GPUGRID] [rr_sim] 2198083.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966670849.70G/439.78G) 23:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854164.72 23:04:00 [---] pick_jobs_to_run() start 23:04:00 [---] pick_jobs_to_run() end 23:04:00 [---] [rr_sim] end 23:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:04:00 [---] [cpu_sched_debug] preliminary job list: 23:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:04:00 [---] [cpu_sched_debug] final job list: 23:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:04:00 [---] [cpu_sched_debug] enforce_run_list: end 23:04:00 [---] choose_project(): 83040.000000 23:04:00 [WUProp@Home] doing RPC to get work 23:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352559.09 cpu 2102192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102192.90 23:04:00 [---] Got 0 tasks 23:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:05:00 [---] [rr_sim] doing sim: CPU sched 23:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966643259.34G/439.78G) 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2198021.07 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 219802.11 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219802.106911 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978218.96 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 197821.90 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417624.003131 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780397.07 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 178039.71 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595663.709729 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602357.36 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 160235.74 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755899.445668 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442121.62 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 144212.16 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900111.608012 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297909.46 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 129790.95 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029902.554122 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168118.51 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 116811.85 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146714.405621 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051306.66 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 105130.67 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251845.071970 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946176.00 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 94617.60 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346462.671684 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851558.40 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 85155.84 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431618.511427 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766402.56 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 76640.26 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508258.767196 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689762.30 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 68976.23 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577234.997387 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620786.07 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 62078.61 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639313.604560 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558707.46 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 55870.75 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695184.351015 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502836.72 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 50283.67 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745468.022825 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452553.05 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 45255.30 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790723.327453 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407297.74 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 40729.77 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831453.101619 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366567.97 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 36656.80 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868109.898369 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329911.17 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 32991.12 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901101.015443 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296920.05 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 29692.01 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930793.020810 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267228.05 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 26722.80 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957515.825640 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240505.24 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 24050.52 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981566.349987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216454.72 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 21645.47 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003211.821900 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194809.25 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 19480.92 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022692.746621 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175328.32 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 17532.83 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040225.578870 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157795.49 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 15779.55 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2056005.127894 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142015.94 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 14201.59 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070206.722016 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127814.35 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 12781.43 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082988.156726 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115032.91 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 11503.29 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094491.447964 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103529.62 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 10352.96 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104844.410079 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93176.66 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 9317.67 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114162.075982 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83858.99 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 8385.90 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122547.975295 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75473.09 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 7547.31 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130095.284677 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67925.78 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 6792.58 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136887.863121 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61133.21 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 6113.32 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143001.183720 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55019.89 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 5501.99 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148503.172259 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49517.90 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 4951.79 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153454.961944 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44566.11 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 4456.61 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157911.572661 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40109.50 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 4010.95 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161922.522306 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36098.55 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3609.85 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165532.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32488.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169132.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28888.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172732.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25288.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176332.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21688.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179932.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18088.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183532.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14488.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187132.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10888.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190732.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7288.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194332.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3688.69 sec) 23:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197932.376987 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 88.69 sec) 23:05:00 [GPUGRID] [rr_sim] 2198021.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966643259.34G/439.78G) 23:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854161.98 23:05:00 [---] pick_jobs_to_run() start 23:05:00 [---] pick_jobs_to_run() end 23:05:00 [---] [rr_sim] end 23:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:05:00 [---] [cpu_sched_debug] preliminary job list: 23:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:05:00 [---] [cpu_sched_debug] final job list: 23:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:05:00 [---] [cpu_sched_debug] enforce_run_list: end 23:05:00 [---] choose_project(): 83100.000000 23:05:00 [WUProp@Home] doing RPC to get work 23:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352499.09 cpu 2102132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102132.90 23:05:00 [---] Got 0 tasks 23:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:06:00 [---] [rr_sim] doing sim: CPU sched 23:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966615668.98G/439.78G) 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197958.33 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 219795.83 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219795.833223 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978162.50 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 197816.25 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417612.083124 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780346.25 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 178034.62 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595646.708034 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602311.62 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 160231.16 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755877.870454 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442080.46 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 144208.05 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900085.916631 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297872.42 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 129787.24 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029873.158191 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168085.17 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 116808.52 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146681.675595 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051276.66 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 105127.67 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251809.341259 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946148.99 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 94614.90 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346424.240356 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851534.09 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 85153.41 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431577.649543 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766380.68 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 76638.07 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508215.717812 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689742.61 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 68974.26 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577189.979254 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620768.35 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 62076.84 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639266.814551 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558691.52 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 55869.15 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695135.966319 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502822.37 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 50282.24 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745418.202910 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452540.13 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 45254.01 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790672.215842 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407286.12 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 40728.61 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831400.827481 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366557.50 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 36655.75 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868056.577956 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329901.75 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 32990.18 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1901046.753383 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296911.58 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 29691.16 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930737.911268 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267220.42 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 26722.04 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957459.953364 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240498.38 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 24049.84 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981509.791251 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216448.54 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 21644.85 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003154.645349 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194803.69 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 19480.37 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022635.014037 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175323.32 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 17532.33 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040167.345856 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157790.99 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 15779.10 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055946.444493 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142011.89 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 14201.19 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070147.633267 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127810.70 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 12781.07 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082928.703163 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115029.63 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 11502.96 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094431.666070 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103526.67 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 10352.67 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104784.332686 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93174.00 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 9317.40 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114101.732640 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83856.60 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 8385.66 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122487.392599 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75470.94 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 7547.09 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130034.486562 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67923.85 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 6792.38 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136826.871129 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61131.46 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 6113.15 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142940.017239 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55018.31 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 5501.83 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148441.848738 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49516.48 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 4951.65 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153393.497087 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44564.84 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 4456.48 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157849.980602 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40108.35 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 4010.84 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161860.815765 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36097.52 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3609.75 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165470.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32487.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169070.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28887.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172670.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25287.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176270.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21687.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179870.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18087.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183470.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14487.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187070.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10887.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190670.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7287.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194270.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3687.76 sec) 23:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197870.567411 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 87.76 sec) 23:06:00 [GPUGRID] [rr_sim] 2197958.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966615668.98G/439.78G) 23:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854159.24 23:06:00 [---] pick_jobs_to_run() start 23:06:00 [---] pick_jobs_to_run() end 23:06:00 [---] [rr_sim] end 23:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:06:00 [---] [cpu_sched_debug] preliminary job list: 23:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:06:00 [---] [cpu_sched_debug] final job list: 23:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:06:00 [---] [cpu_sched_debug] enforce_run_list: end 23:06:00 [---] choose_project(): 83160.000000 23:06:00 [WUProp@Home] doing RPC to get work 23:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352439.09 cpu 2102072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102072.90 23:06:00 [---] Got 0 tasks 23:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:07:00 [---] [rr_sim] doing sim: CPU sched 23:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966588078.63G/439.78G) 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197895.60 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 219789.56 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219789.559535 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978106.04 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 197810.60 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417600.163116 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780295.43 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 178029.54 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595629.706339 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602265.89 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 160226.59 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755856.295240 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1442039.30 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 144203.93 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900060.225251 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297835.37 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 129783.54 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029843.762261 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168051.83 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 116805.18 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146648.945569 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051246.65 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 105124.66 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251773.610547 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946121.98 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 94612.20 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346385.809027 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851509.79 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 85150.98 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431536.787659 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766358.81 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 76635.88 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508172.668428 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689722.93 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 68972.29 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577144.961120 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620750.63 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 62075.06 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639220.024543 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558675.57 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 55867.56 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695087.581623 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502808.01 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 50280.80 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745368.382996 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452527.21 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 45252.72 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790621.104231 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407274.49 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 40727.45 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831348.553343 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366547.04 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 36654.70 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1868003.257543 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329892.34 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 32989.23 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900992.491324 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296903.10 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 29690.31 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930682.801726 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267212.79 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 26721.28 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957404.081088 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240491.51 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 24049.15 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981453.232514 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216442.36 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 21644.24 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003097.468797 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194798.13 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 19479.81 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022577.281453 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175318.31 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 17531.83 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040109.112842 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157786.48 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 15778.65 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055887.761093 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142007.83 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 14200.78 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070088.544518 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127807.05 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 12780.71 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082869.249601 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115026.35 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 11502.63 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094371.884176 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103523.71 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 10352.37 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104724.255293 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93171.34 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 9317.13 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2114041.389298 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83854.21 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 8385.42 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122426.809903 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75468.79 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 7546.88 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129973.688448 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67921.91 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 6792.19 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136765.879138 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61129.72 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 6112.97 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142878.850759 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55016.74 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 5501.67 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148380.525218 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49515.07 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 4951.51 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153332.032231 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44563.56 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 4456.36 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157788.388542 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40107.21 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 4010.72 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161799.109223 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36096.49 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3609.65 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165408.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32486.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169008.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28886.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172608.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25286.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176208.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21686.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179808.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18086.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183408.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14486.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187008.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10886.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190608.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7286.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194208.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3686.84 sec) 23:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197808.757835 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 86.84 sec) 23:07:00 [GPUGRID] [rr_sim] 2197895.60: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966588078.63G/439.78G) 23:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854156.51 23:07:00 [---] pick_jobs_to_run() start 23:07:00 [---] pick_jobs_to_run() end 23:07:00 [---] [rr_sim] end 23:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:07:00 [---] [cpu_sched_debug] preliminary job list: 23:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:07:00 [---] [cpu_sched_debug] final job list: 23:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:07:00 [---] [cpu_sched_debug] enforce_run_list: end 23:07:00 [---] choose_project(): 83220.000000 23:07:00 [WUProp@Home] doing RPC to get work 23:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352379.09 cpu 2102012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2102012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2102012.90 23:07:00 [---] Got 0 tasks 23:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:08:00 [---] [rr_sim] doing sim: CPU sched 23:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966560488.27G/439.78G) 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197832.86 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 219783.29 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219783.285847 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1978049.57 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 197804.96 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417588.243108 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780244.62 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 178024.46 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595612.704644 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602220.15 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 160222.02 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755834.720026 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441998.14 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 144199.81 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900034.533870 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297798.32 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 129779.83 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029814.366330 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1168018.49 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 116801.85 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146616.215543 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051216.64 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 105121.66 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251737.879836 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946094.98 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 94609.50 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346347.377699 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851485.48 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 85148.55 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431495.925775 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766336.93 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 76633.69 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508129.619044 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689703.24 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 68970.32 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577099.942986 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620732.92 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 62073.29 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639173.234534 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558659.62 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 55865.96 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1695039.196927 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502793.66 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 50279.37 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745318.563081 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452514.30 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 45251.43 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790569.992620 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407262.87 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 40726.29 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831296.279204 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366536.58 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 36653.66 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867949.937130 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329882.92 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 32988.29 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900938.229264 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296894.63 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 29689.46 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930627.692184 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267205.17 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 26720.52 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957348.208812 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240484.65 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 24048.46 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981396.673778 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216436.18 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 21643.62 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2003040.292246 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194792.57 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 19479.26 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022519.548868 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175313.31 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 17531.33 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2040050.879828 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157781.98 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 15778.20 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055829.077692 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 142003.78 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 14200.38 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2070029.455769 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127803.40 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 12780.34 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082809.796039 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115023.06 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 11502.31 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094312.102281 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103520.76 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 10352.08 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104664.177900 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93168.68 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 9316.87 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113981.045956 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83851.81 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 8385.18 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122366.227207 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75466.63 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 7546.66 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129912.890333 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67919.97 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 6792.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136704.887146 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61127.97 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 6112.80 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142817.684278 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55015.17 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 5501.52 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148319.201697 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49513.66 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 4951.37 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153270.567374 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44562.29 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 4456.23 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157726.796483 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40106.06 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 4010.61 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161737.402681 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36095.46 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3609.55 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165346.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32485.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168946.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28885.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172546.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25285.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176146.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21685.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179746.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18085.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183346.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14485.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186946.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10885.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190546.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7285.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194146.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3685.91 sec) 23:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197746.948260 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 85.91 sec) 23:08:00 [GPUGRID] [rr_sim] 2197832.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966560488.27G/439.78G) 23:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854153.77 23:08:00 [---] pick_jobs_to_run() start 23:08:00 [---] pick_jobs_to_run() end 23:08:00 [---] [rr_sim] end 23:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:08:00 [---] [cpu_sched_debug] preliminary job list: 23:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:08:00 [---] [cpu_sched_debug] final job list: 23:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:08:00 [---] [cpu_sched_debug] enforce_run_list: end 23:08:00 [---] choose_project(): 83280.000000 23:08:00 [WUProp@Home] doing RPC to get work 23:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352319.09 cpu 2101952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101952.90 23:08:00 [---] Got 0 tasks 23:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:09:00 [---] [rr_sim] doing sim: CPU sched 23:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966532897.92G/439.78G) 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197770.12 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 219777.01 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219777.012158 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977993.11 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 197799.31 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417576.323101 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780193.80 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 178019.38 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595595.702949 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602174.42 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 160217.44 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755813.144813 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441956.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 144195.70 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 900008.842490 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297761.28 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 129776.13 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029784.970399 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167985.15 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 116798.52 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146583.485517 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051186.64 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 105118.66 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251702.149124 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946067.97 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 94606.80 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346308.946370 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851461.18 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 85146.12 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431455.063891 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766315.06 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 76631.51 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508086.569661 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689683.55 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 68968.36 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577054.924853 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620715.20 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 62071.52 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639126.444526 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558643.68 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 55864.37 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694990.812232 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502779.31 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 50277.93 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745268.743167 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452501.38 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 45250.14 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790518.881008 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407251.24 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 40725.12 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831244.005066 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366526.12 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 36652.61 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867896.616718 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329873.50 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 32987.35 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900883.967204 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296886.15 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 29688.62 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930572.582642 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267197.54 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 26719.75 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957292.336536 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240477.79 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 24047.78 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981340.115041 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216430.01 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 21643.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002983.115695 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194787.01 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 19478.70 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022461.816284 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175308.31 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 17530.83 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039992.646814 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157777.47 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 15777.75 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055770.394291 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141999.73 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 14199.97 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069970.367020 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127799.75 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 12779.98 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082750.342477 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115019.78 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 11501.98 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094252.320387 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103517.80 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 10351.78 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104604.100507 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93166.02 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 9316.60 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113920.702614 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83849.42 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 8384.94 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122305.644511 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75464.48 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 7546.45 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129852.092219 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67918.03 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 6791.80 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136643.895155 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61126.23 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 6112.62 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142756.517798 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55013.60 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 5501.36 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148257.878176 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49512.24 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 4951.22 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153209.102517 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44561.02 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 4456.10 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157665.204424 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40104.92 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 4010.49 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161675.696140 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36094.43 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3609.44 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165285.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32484.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168885.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28884.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172485.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25284.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176085.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21684.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179685.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18084.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183285.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14484.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186885.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10884.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190485.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7284.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194085.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3684.98 sec) 23:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197685.138684 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84.98 sec) 23:09:00 [GPUGRID] [rr_sim] 2197770.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966532897.92G/439.78G) 23:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854151.03 23:09:00 [---] pick_jobs_to_run() start 23:09:00 [---] pick_jobs_to_run() end 23:09:00 [---] [rr_sim] end 23:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:09:00 [---] [cpu_sched_debug] preliminary job list: 23:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:09:00 [---] [cpu_sched_debug] final job list: 23:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:09:00 [---] [cpu_sched_debug] enforce_run_list: end 23:09:00 [---] choose_project(): 83340.000000 23:09:00 [WUProp@Home] doing RPC to get work 23:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352259.09 cpu 2101892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101892.90 23:09:00 [---] Got 0 tasks 23:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:10:00 [---] [rr_sim] doing sim: CPU sched 23:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966505307.56G/439.78G) 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197707.38 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 219770.74 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219770.738470 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977936.65 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 197793.66 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417564.403093 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780142.98 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 178014.30 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595578.701254 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602128.68 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 160212.87 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755791.569599 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441915.82 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 144191.58 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899983.151109 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297724.23 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 129772.42 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029755.574468 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167951.81 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 116795.18 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146550.755492 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051156.63 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 105115.66 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251666.418412 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946040.97 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 94604.10 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346270.515041 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851436.87 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 85143.69 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431414.202007 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766293.18 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 76629.32 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508043.520277 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689663.86 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 68966.39 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1577009.906719 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620697.48 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 62069.75 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639079.654517 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558627.73 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 55862.77 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694942.427536 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502764.96 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 50276.50 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745218.923252 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452488.46 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 45248.85 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790467.769397 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407239.62 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 40723.96 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831191.730928 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366515.65 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 36651.57 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867843.296305 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329864.09 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 32986.41 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900829.705145 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296877.68 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 29687.77 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930517.473100 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267189.91 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 26718.99 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957236.464260 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240470.92 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 24047.09 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981283.556304 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216423.83 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 21642.38 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002925.939144 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194781.45 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 19478.14 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022404.083700 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175303.30 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 17530.33 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039934.413800 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157772.97 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 15777.30 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055711.710890 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141995.67 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 14199.57 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069911.278271 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127796.11 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 12779.61 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082690.888914 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115016.50 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 11501.65 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094192.538493 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103514.85 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 10351.48 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104544.023114 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93163.36 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 9316.34 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113860.359273 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83847.03 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 8384.70 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122245.061815 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75462.32 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 7546.23 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129791.294104 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67916.09 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 6791.61 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136582.903164 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61124.48 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 6112.45 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142695.351317 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55012.03 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 5501.20 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148196.554656 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49510.83 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 4951.08 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153147.637660 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44559.75 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 4455.97 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157603.612364 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40103.77 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 4010.38 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161613.989598 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36093.40 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3609.34 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165223.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32484.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168823.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28884.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172423.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25284.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176023.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21684.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179623.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18084.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183223.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14484.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186823.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10884.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190423.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7284.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194023.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3684.06 sec) 23:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197623.329108 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 84.06 sec) 23:10:00 [GPUGRID] [rr_sim] 2197707.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966505307.56G/439.78G) 23:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854148.30 23:10:00 [---] pick_jobs_to_run() start 23:10:00 [---] pick_jobs_to_run() end 23:10:00 [---] [rr_sim] end 23:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:10:00 [---] [cpu_sched_debug] preliminary job list: 23:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:10:00 [---] [cpu_sched_debug] final job list: 23:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:10:00 [---] [cpu_sched_debug] enforce_run_list: end 23:10:00 [---] choose_project(): 83400.000000 23:10:00 [WUProp@Home] doing RPC to get work 23:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352199.09 cpu 2101832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101832.90 23:10:00 [---] Got 0 tasks 23:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:11:00 [---] [rr_sim] doing sim: CPU sched 23:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966477717.20G/439.78G) 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197644.65 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 219764.46 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219764.464782 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977880.18 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 197788.02 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417552.483086 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780092.16 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 178009.22 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595561.699559 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602082.95 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 160208.29 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755769.994385 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441874.65 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 144187.47 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899957.459728 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297687.19 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 129768.72 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029726.178537 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167918.47 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 116791.85 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146518.025466 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051126.62 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 105112.66 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251630.687701 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 946013.96 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 94601.40 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346232.083713 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851412.56 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 85141.26 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431373.340123 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766271.31 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 76627.13 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1508000.470893 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689644.18 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 68964.42 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576964.888586 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620679.76 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 62067.98 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1639032.864509 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558611.78 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 55861.18 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694894.042840 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502750.60 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 50275.06 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745169.103338 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452475.54 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 45247.55 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790416.657786 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407227.99 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 40722.80 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831139.456789 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366505.19 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 36650.52 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867789.975892 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329854.67 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 32985.47 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900775.443085 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296869.20 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 29686.92 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930462.363558 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267182.28 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 26718.23 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957180.591984 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240464.06 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 24046.41 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981226.997568 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216417.65 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 21641.77 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002868.762593 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194775.89 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 19477.59 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022346.351116 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175298.30 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 17529.83 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039876.180786 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157768.47 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 15776.85 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055653.027489 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141991.62 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 14199.16 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069852.189522 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127792.46 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 12779.25 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082631.435352 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115013.21 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 11501.32 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094132.756599 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103511.89 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 10351.19 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104483.945721 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93160.70 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 9316.07 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113800.015931 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83844.63 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 8384.46 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122184.479119 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75460.17 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 7546.02 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129730.495989 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67914.15 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 6791.42 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136521.911172 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61122.74 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 6112.27 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142634.184837 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55010.46 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 5501.05 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148135.231135 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49509.42 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 4950.94 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153086.172804 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44558.48 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 4455.85 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157542.020305 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40102.63 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 4010.26 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161552.283057 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36092.36 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3609.24 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165161.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32483.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168761.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28883.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172361.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25283.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175961.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21683.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179561.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18083.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183161.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14483.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186761.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10883.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190361.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7283.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193961.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3683.13 sec) 23:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197561.519533 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83.13 sec) 23:11:00 [GPUGRID] [rr_sim] 2197644.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966477717.20G/439.78G) 23:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854145.56 23:11:00 [---] pick_jobs_to_run() start 23:11:00 [---] pick_jobs_to_run() end 23:11:00 [---] [rr_sim] end 23:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:11:00 [---] [cpu_sched_debug] preliminary job list: 23:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:11:00 [---] [cpu_sched_debug] final job list: 23:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:11:00 [---] [cpu_sched_debug] enforce_run_list: end 23:11:00 [---] choose_project(): 83460.000000 23:11:00 [WUProp@Home] doing RPC to get work 23:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352139.09 cpu 2101772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101772.90 23:11:00 [---] Got 0 tasks 23:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:12:00 [---] [rr_sim] doing sim: CPU sched 23:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966450126.85G/439.78G) 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197581.91 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 219758.19 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219758.191094 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977823.72 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 197782.37 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417540.563078 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1780041.35 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 178004.13 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595544.697864 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1602037.21 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 160203.72 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755748.419171 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441833.49 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 144183.35 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899931.768348 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297650.14 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 129765.01 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029696.782607 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167885.13 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 116788.51 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146485.295440 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051096.62 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 105109.66 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251594.956989 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945986.95 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 94598.70 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346193.652384 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851388.26 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 85138.83 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431332.478239 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766249.43 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 76624.94 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507957.421509 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689624.49 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 68962.45 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576919.870452 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620662.04 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 62066.20 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638986.074500 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558595.84 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 55859.58 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694845.658144 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502736.25 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 50273.63 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745119.283423 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452462.63 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 45246.26 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790365.546175 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407216.36 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 40721.64 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831087.182651 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366494.73 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 36649.47 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867736.655480 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329845.26 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 32984.53 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900721.181025 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296860.73 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 29686.07 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930407.254016 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267174.66 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 26717.47 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957124.719708 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240457.19 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 24045.72 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981170.438831 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216411.47 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 21641.15 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002811.586042 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194770.32 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 19477.03 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022288.618531 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175293.29 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 17529.33 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039817.947772 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157763.96 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 15776.40 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055594.344088 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141987.57 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 14198.76 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069793.100773 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127788.81 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 12778.88 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082571.981790 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115009.93 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 11500.99 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094072.974704 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103508.94 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 10350.89 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104423.868328 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93158.04 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 9315.80 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113739.672589 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83842.24 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 8384.22 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122123.896423 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75458.01 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 7545.80 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129669.697875 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67912.21 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 6791.22 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136460.919181 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61120.99 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 6112.10 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142573.018357 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55008.89 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 5500.89 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148073.907615 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49508.00 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 4950.80 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153024.707947 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44557.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 4455.72 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157480.428246 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40101.48 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 4010.15 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161490.576515 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36091.33 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3609.13 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165099.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32482.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168699.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28882.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172299.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25282.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175899.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21682.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179499.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18082.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183099.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14482.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186699.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10882.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190299.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7282.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193899.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3682.20 sec) 23:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197499.709957 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 82.20 sec) 23:12:00 [GPUGRID] [rr_sim] 2197581.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966450126.85G/439.78G) 23:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854142.82 23:12:00 [---] pick_jobs_to_run() start 23:12:00 [---] pick_jobs_to_run() end 23:12:00 [---] [rr_sim] end 23:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:12:00 [---] [cpu_sched_debug] preliminary job list: 23:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:12:00 [---] [cpu_sched_debug] final job list: 23:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:12:00 [---] [cpu_sched_debug] enforce_run_list: end 23:12:00 [---] choose_project(): 83520.000000 23:12:00 [WUProp@Home] doing RPC to get work 23:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352079.09 cpu 2101712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101712.90 23:12:00 [---] Got 0 tasks 23:13:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:13:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:13:00 [---] [rr_sim] doing sim: CPU sched 23:13:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966422536.49G/439.78G) 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197519.17 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 219751.92 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219751.917405 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977767.26 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 197776.73 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417528.643070 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779990.53 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 177999.05 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595527.696169 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601991.48 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 160199.15 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755726.843957 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441792.33 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 144179.23 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899906.076967 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297613.10 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 129761.31 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029667.386676 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167851.79 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 116785.18 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146452.565414 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051066.61 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 105106.66 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251559.226278 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945959.95 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 94595.99 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346155.221056 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851363.95 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 85136.40 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431291.616355 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766227.56 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 76622.76 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507914.372125 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689604.80 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 68960.48 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576874.852318 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620644.32 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 62064.43 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638939.284492 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558579.89 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 55857.99 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694797.273448 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502721.90 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 50272.19 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745069.463509 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452449.71 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 45244.97 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790314.434563 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407204.74 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 40720.47 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1831034.908513 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366484.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 36648.43 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867683.335067 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329835.84 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 32983.58 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900666.918966 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296852.26 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 29685.23 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930352.144475 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267167.03 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 26716.70 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957068.847433 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240450.33 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 24045.03 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981113.880095 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216405.29 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 21640.53 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002754.409491 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194764.76 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 19476.48 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022230.885947 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175288.29 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 17528.83 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039759.714758 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157759.46 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 15775.95 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055535.660688 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141983.51 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 14198.35 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069734.012024 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127785.16 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 12778.52 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082512.528227 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115006.65 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 11500.66 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2094013.192810 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103505.98 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 10350.60 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104363.790935 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93155.38 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 9315.54 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113679.329247 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83839.84 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 8383.98 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122063.313727 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75455.86 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 7545.59 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129608.899760 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67910.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 6791.03 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136399.927190 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61119.25 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 6111.92 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142511.851876 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55007.32 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 5500.73 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148012.584094 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49506.59 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 4950.66 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152963.243090 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44555.93 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 4455.59 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157418.836187 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40100.34 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 4010.03 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161428.869973 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36090.30 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3609.03 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165037.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32481.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168637.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28881.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172237.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25281.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175837.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21681.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179437.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18081.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183037.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14481.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186637.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10881.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190237.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7281.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193837.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3681.27 sec) 23:13:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197437.900381 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 81.27 sec) 23:13:00 [GPUGRID] [rr_sim] 2197519.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966422536.49G/439.78G) 23:13:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854140.09 23:13:00 [---] pick_jobs_to_run() start 23:13:00 [---] pick_jobs_to_run() end 23:13:00 [---] [rr_sim] end 23:13:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:13:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:13:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:13:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:13:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:13:00 [---] [cpu_sched_debug] preliminary job list: 23:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:13:00 [---] [cpu_sched_debug] final job list: 23:13:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:13:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:13:00 [---] [cpu_sched_debug] enforce_run_list: end 23:13:00 [---] choose_project(): 83580.000000 23:13:00 [WUProp@Home] doing RPC to get work 23:13:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 352019.09 cpu 2101652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101652.90 23:13:00 [---] Got 0 tasks 23:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:14:00 [---] [rr_sim] doing sim: CPU sched 23:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966394946.13G/439.78G) 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197456.44 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 219745.64 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219745.643717 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977710.79 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 197771.08 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417516.723063 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779939.71 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 177993.97 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595510.694474 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601945.74 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 160194.57 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755705.268744 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441751.17 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 144175.12 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899880.385587 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297576.05 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 129757.61 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029637.990745 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167818.45 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 116781.84 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146419.835388 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051036.60 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 105103.66 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251523.495566 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945932.94 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 94593.29 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346116.789727 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851339.65 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 85133.96 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431250.754472 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766205.68 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 76620.57 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507871.322742 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689585.11 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 68958.51 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576829.834185 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620626.60 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 62062.66 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638892.494484 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558563.94 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 55856.39 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694748.888752 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502707.55 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 50270.75 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1745019.643594 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452436.79 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 45243.68 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790263.322952 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407193.11 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 40719.31 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830982.634374 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366473.80 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 36647.38 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867630.014654 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329826.42 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 32982.64 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900612.656906 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296843.78 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 29684.38 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930297.034933 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267159.40 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 26715.94 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1957012.975157 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240443.46 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 24044.35 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981057.321358 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216399.12 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 21639.91 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002697.232940 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194759.20 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 19475.92 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022173.153363 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175283.28 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 17528.33 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039701.481744 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157754.96 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 15775.50 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055476.977287 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141979.46 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 14197.95 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069674.923275 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127781.51 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 12778.15 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082453.074665 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115003.36 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 11500.34 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093953.410916 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103503.03 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 10350.30 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104303.713542 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93152.72 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 9315.27 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113618.985905 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83837.45 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 8383.75 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2122002.731031 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75453.71 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 7545.37 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129548.101646 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67908.34 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 6790.83 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136338.935198 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61117.50 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 6111.75 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142450.685396 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55005.75 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 5500.58 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147951.260573 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49505.18 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 4950.52 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152901.778233 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44554.66 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 4455.47 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157357.244127 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40099.19 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 4009.92 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161367.163432 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36089.27 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3608.93 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164976.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32480.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168576.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28880.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172176.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25280.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175776.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21680.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179376.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18080.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182976.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14480.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186576.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10880.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190176.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7280.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193776.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3680.35 sec) 23:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197376.090806 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 80.35 sec) 23:14:00 [GPUGRID] [rr_sim] 2197456.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966394946.13G/439.78G) 23:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854137.35 23:14:00 [---] pick_jobs_to_run() start 23:14:00 [---] pick_jobs_to_run() end 23:14:00 [---] [rr_sim] end 23:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:14:00 [---] [cpu_sched_debug] preliminary job list: 23:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:14:00 [---] [cpu_sched_debug] final job list: 23:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:14:00 [---] [cpu_sched_debug] enforce_run_list: end 23:14:00 [---] choose_project(): 83640.000000 23:14:00 [WUProp@Home] doing RPC to get work 23:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351959.09 cpu 2101592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101592.90 23:14:00 [---] Got 0 tasks 23:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:15:00 [---] [rr_sim] doing sim: CPU sched 23:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966367355.78G/439.78G) 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197393.70 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 219739.37 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219739.370029 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977654.33 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 197765.43 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417504.803055 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779888.90 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 177988.89 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595493.692779 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601900.01 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 160190.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755683.693530 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441710.01 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 144171.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899854.694206 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297539.01 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 129753.90 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029608.594814 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167785.11 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 116778.51 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146387.105362 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1051006.59 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 105100.66 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251487.764855 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945905.94 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 94590.59 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346078.358398 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851315.34 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 85131.53 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431209.892588 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766183.81 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 76618.38 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507828.273358 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689565.43 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 68956.54 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576784.816051 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620608.88 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 62060.89 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638845.704475 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558548.00 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 55854.80 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694700.504057 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502693.20 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 50269.32 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744969.823680 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452423.88 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 45242.39 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790212.211341 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407181.49 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 40718.15 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830930.360236 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366463.34 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 36646.33 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867576.694241 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329817.01 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 32981.70 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900558.394846 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296835.31 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 29683.53 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930241.925391 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267151.77 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 26715.18 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956957.102881 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240436.60 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 24043.66 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1981000.762622 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216392.94 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 21639.29 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002640.056389 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194753.64 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 19475.36 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022115.420779 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175278.28 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 17527.83 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039643.248730 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157750.45 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 15775.05 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055418.293886 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141975.41 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 14197.54 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069615.834526 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127777.87 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 12777.79 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082393.621103 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 115000.08 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 11500.01 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093893.629022 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103500.07 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 10350.01 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104243.636148 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93150.06 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 9315.01 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113558.642563 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83835.06 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 8383.51 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121942.148335 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75451.55 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 7545.16 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129487.303531 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67906.40 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 6790.64 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136277.943207 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61115.76 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 6111.58 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142389.518915 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55004.18 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 5500.42 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147889.937053 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49503.76 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 4950.38 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152840.313377 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44553.39 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 4455.34 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157295.652068 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40098.05 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 4009.80 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161305.456890 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36088.24 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3608.82 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164914.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32479.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168514.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28879.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172114.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25279.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175714.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21679.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179314.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18079.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182914.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14479.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186514.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10879.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190114.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7279.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193714.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3679.42 sec) 23:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197314.281230 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 79.42 sec) 23:15:00 [GPUGRID] [rr_sim] 2197393.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966367355.78G/439.78G) 23:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854134.61 23:15:00 [---] pick_jobs_to_run() start 23:15:00 [---] pick_jobs_to_run() end 23:15:00 [---] [rr_sim] end 23:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:15:00 [---] [cpu_sched_debug] preliminary job list: 23:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:15:00 [---] [cpu_sched_debug] final job list: 23:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:15:00 [---] [cpu_sched_debug] enforce_run_list: end 23:15:00 [---] choose_project(): 83700.000000 23:15:00 [WUProp@Home] doing RPC to get work 23:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351899.09 cpu 2101532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101532.90 23:15:00 [---] Got 0 tasks 23:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:16:00 [---] [rr_sim] doing sim: CPU sched 23:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966339765.42G/439.78G) 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197330.96 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 219733.10 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219733.096341 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977597.87 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 197759.79 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417492.883048 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779838.08 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 177983.81 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595476.691084 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601854.27 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 160185.43 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755662.118316 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441668.85 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 144166.88 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899829.002825 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297501.96 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 129750.20 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029579.198884 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167751.76 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 116775.18 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146354.375336 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050976.59 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 105097.66 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251452.034143 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945878.93 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 94587.89 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346039.927070 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851291.04 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 85129.10 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431169.030704 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766161.93 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 76616.19 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507785.223974 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689545.74 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 68954.57 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576739.797918 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620591.17 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 62059.12 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638798.914467 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558532.05 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 55853.20 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694652.119361 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502678.84 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 50267.88 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744920.003765 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452410.96 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 45241.10 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790161.099730 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407169.86 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 40716.99 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830878.086098 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366452.88 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 36645.29 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867523.373829 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329807.59 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 32980.76 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900504.132787 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296826.83 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 29682.68 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930186.815849 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267144.15 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 26714.41 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956901.230605 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240429.73 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 24042.97 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980944.203885 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216386.76 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 21638.68 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002582.879837 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194748.08 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 19474.81 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2022057.688194 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175273.28 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 17527.33 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039585.015716 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157745.95 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 15774.59 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055359.610485 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141971.35 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 14197.14 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069556.745777 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127774.22 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 12777.42 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082334.167541 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114996.80 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 11499.68 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093833.847127 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103497.12 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 10349.71 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104183.558755 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93147.40 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 9314.74 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113498.299221 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83832.66 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 8383.27 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121881.565639 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75449.40 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 7544.94 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129426.505416 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67904.46 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 6790.45 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136216.951215 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61114.01 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 6111.40 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142328.352435 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55002.61 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 5500.26 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147828.613532 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49502.35 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 4950.23 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152778.848520 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44552.11 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 4455.21 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157234.060009 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40096.90 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 4009.69 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161243.750349 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36087.21 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3608.72 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164852.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32478.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168452.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28878.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172052.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25278.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175652.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21678.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179252.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18078.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182852.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14478.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186452.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10878.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190052.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7278.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193652.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3678.49 sec) 23:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197252.471655 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 78.49 sec) 23:16:00 [GPUGRID] [rr_sim] 2197330.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966339765.42G/439.78G) 23:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854131.88 23:16:00 [---] pick_jobs_to_run() start 23:16:00 [---] pick_jobs_to_run() end 23:16:00 [---] [rr_sim] end 23:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:16:00 [---] [cpu_sched_debug] preliminary job list: 23:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:16:00 [---] [cpu_sched_debug] final job list: 23:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:16:00 [---] [cpu_sched_debug] enforce_run_list: end 23:16:00 [---] choose_project(): 83760.000000 23:16:00 [WUProp@Home] doing RPC to get work 23:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351839.09 cpu 2101472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101472.90 23:16:00 [---] Got 0 tasks 23:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:17:00 [---] [rr_sim] doing sim: CPU sched 23:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966312175.07G/439.78G) 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197268.23 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 219726.82 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219726.822653 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977541.40 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 197754.14 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417480.963040 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779787.26 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 177978.73 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595459.689389 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601808.54 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 160180.85 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755640.543102 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441627.68 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 144162.77 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899803.311445 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297464.92 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 129746.49 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029549.802953 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167718.42 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 116771.84 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146321.645310 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050946.58 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 105094.66 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251416.303432 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945851.92 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 94585.19 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1346001.495741 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851266.73 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 85126.67 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431128.168820 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766140.06 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 76614.01 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507742.174590 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689526.05 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 68952.61 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576694.779784 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620573.45 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 62057.34 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638752.124458 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558516.10 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 55851.61 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694603.734665 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502664.49 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 50266.45 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744870.183851 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452398.04 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 45239.80 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790109.988119 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407158.24 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 40715.82 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830825.811959 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366442.41 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 36644.24 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867470.053416 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329798.17 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 32979.82 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900449.870727 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296818.36 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 29681.84 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930131.706307 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267136.52 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 26713.65 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956845.358329 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240422.87 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 24042.29 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980887.645149 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216380.58 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 21638.06 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002525.703286 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194742.52 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 19474.25 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021999.955610 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175268.27 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 17526.83 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039526.782702 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157741.44 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 15774.14 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055300.927084 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141967.30 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 14196.73 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069497.657028 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127770.57 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 12777.06 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082274.713978 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114993.51 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 11499.35 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093774.065233 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103494.16 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 10349.42 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104123.481362 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93144.75 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 9314.47 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113437.955879 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83830.27 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 8383.03 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121820.982943 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75447.24 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 7544.72 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129365.707302 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67902.52 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 6790.25 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136155.959224 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61112.27 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 6111.23 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142267.185954 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55001.04 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 5500.10 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147767.290012 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49500.94 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 4950.09 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152717.383663 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44550.84 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 4455.08 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157172.467949 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40095.76 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 4009.58 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161182.043807 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36086.18 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3608.62 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164790.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32477.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168390.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28877.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171990.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25277.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175590.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21677.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179190.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18077.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182790.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14477.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186390.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10877.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189990.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7277.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193590.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3677.56 sec) 23:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197190.662079 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 77.56 sec) 23:17:00 [GPUGRID] [rr_sim] 2197268.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966312175.07G/439.78G) 23:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854129.14 23:17:00 [---] pick_jobs_to_run() start 23:17:00 [---] pick_jobs_to_run() end 23:17:00 [---] [rr_sim] end 23:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:17:00 [---] [cpu_sched_debug] preliminary job list: 23:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:17:00 [---] [cpu_sched_debug] final job list: 23:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:17:00 [---] [cpu_sched_debug] enforce_run_list: end 23:17:00 [---] choose_project(): 83820.000000 23:17:00 [WUProp@Home] doing RPC to get work 23:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351779.09 cpu 2101412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101412.90 23:17:00 [---] Got 0 tasks 23:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:18:00 [---] [rr_sim] doing sim: CPU sched 23:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966284584.71G/439.78G) 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197205.49 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 219720.55 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219720.548964 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977484.94 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 197748.49 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417469.043032 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779736.45 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 177973.64 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595442.687694 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601762.80 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 160176.28 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755618.967889 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441586.52 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 144158.65 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899777.620064 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297427.87 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 129742.79 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029520.407022 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167685.08 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 116768.51 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146288.915284 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050916.57 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 105091.66 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251380.572720 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945824.92 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 94582.49 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345963.064413 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851242.43 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 85124.24 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431087.306936 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766118.18 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 76611.82 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507699.125207 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689506.36 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 68950.64 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576649.761650 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620555.73 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 62055.57 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638705.334450 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558500.16 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 55850.02 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694555.349969 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502650.14 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 50265.01 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744820.363937 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452385.13 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 45238.51 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790058.876507 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407146.61 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 40714.66 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830773.537821 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366431.95 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 36643.20 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867416.733003 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329788.76 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 32978.88 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900395.608667 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296809.88 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 29680.99 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930076.596765 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267128.89 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 26712.89 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956789.486053 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240416.00 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 24041.60 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980831.086412 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216374.40 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 21637.44 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002468.526735 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194736.96 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 19473.70 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021942.223026 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175263.27 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 17526.33 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039468.549688 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157736.94 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 15773.69 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055242.243683 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141963.25 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 14196.32 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069438.568279 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127766.92 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 12776.69 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082215.260416 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114990.23 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 11499.02 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093714.283339 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103491.21 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 10349.12 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104063.403969 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93142.09 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 9314.21 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113377.612537 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83827.88 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 8382.79 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121760.400247 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75445.09 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 7544.51 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129304.909187 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67900.58 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 6790.06 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136094.967233 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61110.52 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 6111.05 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142206.019474 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54999.47 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 5499.95 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147705.966491 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49499.52 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 4949.95 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152655.918806 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44549.57 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 4454.96 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157110.875890 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40094.61 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 4009.46 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161120.337265 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36085.15 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3608.52 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164728.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32476.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168328.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28876.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171928.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25276.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175528.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21676.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179128.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18076.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182728.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14476.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186328.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10876.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189928.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7276.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193528.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3676.64 sec) 23:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197128.852503 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 76.64 sec) 23:18:00 [GPUGRID] [rr_sim] 2197205.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966284584.71G/439.78G) 23:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854126.40 23:18:00 [---] pick_jobs_to_run() start 23:18:00 [---] pick_jobs_to_run() end 23:18:00 [---] [rr_sim] end 23:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:18:00 [---] [cpu_sched_debug] preliminary job list: 23:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:18:00 [---] [cpu_sched_debug] final job list: 23:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:18:00 [---] [cpu_sched_debug] enforce_run_list: end 23:18:00 [---] choose_project(): 83880.000000 23:18:00 [WUProp@Home] doing RPC to get work 23:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351719.09 cpu 2101352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101352.90 23:18:00 [---] Got 0 tasks 23:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:19:00 [---] [rr_sim] doing sim: CPU sched 23:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966256994.35G/439.78G) 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197142.75 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 219714.28 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219714.275276 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977428.48 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 197742.85 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417457.123025 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779685.63 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 177968.56 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595425.685998 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601717.07 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 160171.71 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755597.392675 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441545.36 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 144154.54 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899751.928683 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297390.82 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 129739.08 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029491.011091 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167651.74 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 116765.17 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146256.185258 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050886.57 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 105088.66 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251344.842009 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945797.91 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 94579.79 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345924.633084 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851218.12 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 85121.81 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431046.445052 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766096.31 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 76609.63 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507656.075823 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689486.68 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 68948.67 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576604.743517 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620538.01 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 62053.80 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638658.544441 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558484.21 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 55848.42 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694506.965273 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502635.79 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 50263.58 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744770.544022 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452372.21 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 45237.22 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1790007.764896 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407134.99 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 40713.50 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830721.263683 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366421.49 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 36642.15 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867363.412591 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329779.34 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 32977.93 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900341.346608 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296801.41 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 29680.14 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1930021.487223 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267121.27 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 26712.13 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956733.613777 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240409.14 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 24040.91 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980774.527675 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216368.23 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 21636.82 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002411.350184 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194731.40 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 19473.14 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021884.490442 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175258.26 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 17525.83 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039410.316674 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157732.44 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 15773.24 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055183.560283 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141959.19 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 14195.92 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069379.479531 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127763.27 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 12776.33 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082155.806854 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114986.95 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 11498.69 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093654.501444 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103488.25 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 10348.83 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2104003.326576 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93139.43 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 9313.94 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113317.269195 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83825.48 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 8382.55 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121699.817551 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75442.94 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 7544.29 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129244.111072 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67898.64 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 6789.86 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136033.975241 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61108.78 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 6110.88 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142144.852993 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54997.90 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 5499.79 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147644.642970 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49498.11 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 4949.81 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152594.453949 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44548.30 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 4454.83 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157049.283831 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40093.47 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 4009.35 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161058.630724 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36084.12 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3608.41 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164667.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32475.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168267.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28875.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171867.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25275.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175467.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21675.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179067.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18075.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182667.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14475.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186267.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10875.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189867.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7275.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193467.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3675.71 sec) 23:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197067.042928 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75.71 sec) 23:19:00 [GPUGRID] [rr_sim] 2197142.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966256994.35G/439.78G) 23:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854123.67 23:19:00 [---] pick_jobs_to_run() start 23:19:00 [---] pick_jobs_to_run() end 23:19:00 [---] [rr_sim] end 23:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:19:00 [---] [cpu_sched_debug] preliminary job list: 23:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:19:00 [---] [cpu_sched_debug] final job list: 23:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:19:00 [---] [cpu_sched_debug] enforce_run_list: end 23:19:00 [---] choose_project(): 83940.000000 23:19:00 [WUProp@Home] doing RPC to get work 23:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351659.09 cpu 2101292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101292.90 23:19:00 [---] Got 0 tasks 23:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:20:00 [---] [rr_sim] doing sim: CPU sched 23:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966229404.00G/439.78G) 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197080.02 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 219708.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219708.001588 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977372.01 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 197737.20 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417445.203017 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779634.81 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 177963.48 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595408.684303 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601671.33 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 160167.13 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755575.817461 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441504.20 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 144150.42 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899726.237303 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297353.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 129735.38 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029461.615161 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167618.40 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 116761.84 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146223.455232 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050856.56 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 105085.66 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251309.111297 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945770.90 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 94577.09 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345886.201755 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851193.81 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 85119.38 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1431005.583168 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766074.43 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 76607.44 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507613.026439 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689466.99 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 68946.70 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576559.725383 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620520.29 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 62052.03 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638611.754433 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558468.26 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 55846.83 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694458.580577 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502621.44 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 50262.14 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744720.724108 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452359.29 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 45235.93 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789956.653285 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407123.36 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 40712.34 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830668.989544 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366411.03 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 36641.10 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867310.092178 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329769.92 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 32976.99 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900287.084548 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296792.93 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 29679.29 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929966.377681 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267113.64 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 26711.36 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956677.741501 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240402.27 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 24040.23 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980717.968939 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216362.05 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 21636.20 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002354.173633 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194725.84 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 19472.58 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021826.757858 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175253.26 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 17525.33 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039352.083660 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157727.93 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 15772.79 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055124.876882 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141955.14 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 14195.51 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069320.390782 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127759.63 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 12775.96 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082096.353291 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114983.66 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 11498.37 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093594.719550 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103485.30 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 10348.53 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103943.249183 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93136.77 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 9313.68 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113256.925853 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83823.09 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 8382.31 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121639.234855 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75440.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 7544.08 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129183.312958 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67896.70 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 6789.67 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135972.983250 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61107.03 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 6110.70 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142083.686513 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54996.33 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 5499.63 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147583.319450 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49496.70 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 4949.67 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152532.989093 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44547.03 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 4454.70 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156987.691771 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40092.32 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 4009.23 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160996.924182 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36083.09 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3608.31 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164605.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32474.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168205.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28874.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171805.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25274.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175405.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21674.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179005.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18074.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182605.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14474.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186205.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10874.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189805.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7274.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193405.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3674.78 sec) 23:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2197005.233352 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 74.78 sec) 23:20:00 [GPUGRID] [rr_sim] 2197080.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966229404.00G/439.78G) 23:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854120.93 23:20:00 [---] pick_jobs_to_run() start 23:20:00 [---] pick_jobs_to_run() end 23:20:00 [---] [rr_sim] end 23:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:20:00 [---] [cpu_sched_debug] preliminary job list: 23:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:20:00 [---] [cpu_sched_debug] final job list: 23:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:20:00 [---] [cpu_sched_debug] enforce_run_list: end 23:20:00 [---] choose_project(): 84000.000000 23:20:00 [WUProp@Home] doing RPC to get work 23:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351599.09 cpu 2101232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101232.90 23:20:00 [---] Got 0 tasks 23:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:21:00 [---] [rr_sim] doing sim: CPU sched 23:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966201813.64G/439.78G) 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2197017.28 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 219701.73 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219701.727900 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977315.55 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 197731.56 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417433.283010 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779584.00 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 177958.40 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595391.682608 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601625.60 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 160162.56 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755554.242247 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441463.04 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 144146.30 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899700.545922 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297316.73 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 129731.67 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029432.219230 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167585.06 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 116758.51 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146190.725207 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050826.55 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 105082.66 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251273.380586 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945743.90 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 94574.39 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345847.770427 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851169.51 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 85116.95 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430964.721284 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766052.56 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 76605.26 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507569.977055 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689447.30 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 68944.73 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576514.707249 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620502.57 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 62050.26 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638564.964424 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558452.31 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 55845.23 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694410.195882 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502607.08 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 50260.71 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744670.904193 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452346.37 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 45234.64 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789905.541674 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407111.74 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 40711.17 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830616.715406 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366400.56 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 36640.06 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867256.771765 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329760.51 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 32976.05 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900232.822488 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296784.46 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 29678.45 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929911.268139 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267106.01 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 26710.60 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956621.869225 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240395.41 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 24039.54 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980661.410202 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216355.87 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 21635.59 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002296.997082 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194720.28 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 19472.03 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021769.025273 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175248.25 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 17524.83 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039293.850646 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157723.43 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 15772.34 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055066.193481 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141951.09 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 14195.11 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069261.302033 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127755.98 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 12775.60 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2082036.899729 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114980.38 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 11498.04 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093534.937656 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103482.34 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 10348.23 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103883.171790 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93134.11 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 9313.41 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113196.582511 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83820.70 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 8382.07 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121578.652159 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75438.63 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 7543.86 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129122.514843 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67894.76 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 6789.48 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135911.991259 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61105.29 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 6110.53 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142022.520033 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54994.76 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 5499.48 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147521.995929 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49495.28 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 4949.53 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152471.524236 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44545.75 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 4454.58 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156926.099712 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40091.18 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 4009.12 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160935.217641 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36082.06 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3608.21 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164543.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32473.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168143.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28873.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171743.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25273.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175343.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21673.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178943.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18073.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182543.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14473.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186143.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10873.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189743.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7273.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193343.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3673.86 sec) 23:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196943.423776 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 73.86 sec) 23:21:00 [GPUGRID] [rr_sim] 2197017.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966201813.64G/439.78G) 23:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854118.19 23:21:00 [---] pick_jobs_to_run() start 23:21:00 [---] pick_jobs_to_run() end 23:21:00 [---] [rr_sim] end 23:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:21:00 [---] [cpu_sched_debug] preliminary job list: 23:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:21:00 [---] [cpu_sched_debug] final job list: 23:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:21:00 [---] [cpu_sched_debug] enforce_run_list: end 23:21:00 [---] choose_project(): 84060.000000 23:21:00 [WUProp@Home] doing RPC to get work 23:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351539.09 cpu 2101172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101172.90 23:21:00 [---] Got 0 tasks 23:22:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:22:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:22:00 [---] [rr_sim] doing sim: CPU sched 23:22:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966174223.29G/439.78G) 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196954.54 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 219695.45 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219695.454212 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977259.09 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 197725.91 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417421.363002 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779533.18 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 177953.32 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595374.680913 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601579.86 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 160157.99 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755532.667033 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441421.88 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 144142.19 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899674.854542 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297279.69 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 129727.97 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029402.823299 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167551.72 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 116755.17 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146157.995181 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050796.55 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 105079.65 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251237.649874 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945716.89 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 94571.69 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345809.339098 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851145.20 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 85114.52 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430923.859400 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766030.68 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 76603.07 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507526.927671 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689427.61 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 68942.76 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576469.689116 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620484.85 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 62048.49 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638518.174416 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558436.37 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 55843.64 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694361.811186 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502592.73 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 50259.27 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744621.084279 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452333.46 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 45233.35 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789854.430062 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407100.11 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 40710.01 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830564.441268 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366390.10 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 36639.01 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867203.451352 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329751.09 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 32975.11 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900178.560429 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296775.98 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 29677.60 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929856.158597 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267098.38 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 26709.84 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956565.996949 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240388.55 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 24038.85 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980604.851466 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216349.69 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 21634.97 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002239.820531 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194714.72 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 19471.47 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021711.292689 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175243.25 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 17524.32 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039235.617632 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157718.92 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 15771.89 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2055007.510080 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141947.03 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 14194.70 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069202.213284 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127752.33 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 12775.23 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081977.446167 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114977.10 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 11497.71 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093475.155762 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103479.39 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 10347.94 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103823.094397 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93131.45 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 9313.14 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113136.239169 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83818.30 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 8381.83 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121518.069463 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75436.47 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 7543.65 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129061.716729 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67892.83 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 6789.28 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135850.999267 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61103.54 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 6110.35 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141961.353552 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54993.19 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 5499.32 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147460.672408 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49493.87 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 4949.39 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152410.059379 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44544.48 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 4454.45 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156864.507653 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40090.03 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 4009.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160873.511099 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36081.03 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3608.10 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164481.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32472.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168081.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28872.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171681.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25272.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175281.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21672.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178881.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18072.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182481.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14472.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186081.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10872.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189681.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7272.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193281.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3672.93 sec) 23:22:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196881.614201 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 72.93 sec) 23:22:00 [GPUGRID] [rr_sim] 2196954.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966174223.29G/439.78G) 23:22:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854115.45 23:22:00 [---] pick_jobs_to_run() start 23:22:00 [---] pick_jobs_to_run() end 23:22:00 [---] [rr_sim] end 23:22:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:22:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:22:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:22:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:22:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:22:00 [---] [cpu_sched_debug] preliminary job list: 23:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:22:00 [---] [cpu_sched_debug] final job list: 23:22:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:22:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:22:00 [---] [cpu_sched_debug] enforce_run_list: end 23:22:00 [---] choose_project(): 84120.000000 23:22:00 [WUProp@Home] doing RPC to get work 23:22:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351479.09 cpu 2101112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101112.90 23:22:00 [---] Got 0 tasks 23:23:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:23:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:23:00 [---] [rr_sim] doing sim: CPU sched 23:23:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966146632.93G/439.78G) 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196891.81 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 219689.18 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219689.180523 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977202.62 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 197720.26 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417409.442994 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779482.36 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 177948.24 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595357.679218 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601534.13 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 160153.41 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755511.091820 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441380.71 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 144138.07 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899649.163161 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297242.64 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 129724.26 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029373.427368 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167518.38 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 116751.84 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146125.265155 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050766.54 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 105076.65 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251201.919163 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945689.89 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 94568.99 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345770.907770 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851120.90 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 85112.09 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430882.997516 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 766008.81 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 76600.88 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507483.878288 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689407.93 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 68940.79 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576424.670982 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620467.13 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 62046.71 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638471.384407 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558420.42 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 55842.04 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694313.426490 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502578.38 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 50257.84 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744571.264364 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452320.54 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 45232.05 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789803.318451 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407088.49 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 40708.85 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830512.167129 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366379.64 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 36637.96 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867150.130940 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329741.67 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 32974.17 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900124.298369 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296767.51 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 29676.75 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929801.049055 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267090.76 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 26709.08 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956510.124673 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240381.68 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 24038.17 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980548.292729 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216343.51 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 21634.35 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002182.643980 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194709.16 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 19470.92 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021653.560105 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175238.25 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 17523.82 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039177.384618 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157714.42 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 15771.44 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054948.826679 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141942.98 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 14194.30 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069143.124535 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127748.68 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 12774.87 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081917.992605 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114973.81 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 11497.38 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093415.373867 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103476.43 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 10347.64 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103763.017004 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93128.79 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 9312.88 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113075.895827 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83815.91 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 8381.59 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121457.486767 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75434.32 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 7543.43 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2129000.918614 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67890.89 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 6789.09 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135790.007276 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61101.80 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 6110.18 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141900.187072 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54991.62 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 5499.16 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147399.348888 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49492.46 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 4949.25 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152348.594522 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44543.21 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 4454.32 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156802.915593 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40088.89 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 4008.89 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160811.804557 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36080.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3608.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164419.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32472.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168019.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28872.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171619.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25272.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175219.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21672.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178819.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18072.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182419.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14472.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186019.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10872.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189619.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7272.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193219.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3672.00 sec) 23:23:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196819.804625 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 72.00 sec) 23:23:00 [GPUGRID] [rr_sim] 2196891.81: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966146632.93G/439.78G) 23:23:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854112.72 23:23:00 [---] pick_jobs_to_run() start 23:23:00 [---] pick_jobs_to_run() end 23:23:00 [---] [rr_sim] end 23:23:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:23:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:23:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:23:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:23:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:23:00 [---] [cpu_sched_debug] preliminary job list: 23:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:23:00 [---] [cpu_sched_debug] final job list: 23:23:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:23:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:23:00 [---] [cpu_sched_debug] enforce_run_list: end 23:23:00 [---] choose_project(): 84180.000000 23:23:00 [WUProp@Home] doing RPC to get work 23:23:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351419.09 cpu 2101052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2101052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2101052.90 23:23:00 [---] Got 0 tasks 23:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:24:00 [---] [rr_sim] doing sim: CPU sched 23:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966119042.57G/439.78G) 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196829.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 219682.91 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219682.906835 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977146.16 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 197714.62 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417397.522987 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779431.55 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 177943.15 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595340.677523 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601488.39 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 160148.84 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755489.516606 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441339.55 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 144133.96 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899623.471780 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297205.60 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 129720.56 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029344.031437 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167485.04 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 116748.50 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146092.535129 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050736.53 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 105073.65 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251166.188451 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945662.88 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 94566.29 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345732.476441 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851096.59 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 85109.66 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430842.135632 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765986.93 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 76598.69 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507440.828904 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689388.24 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 68938.82 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576379.652849 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620449.42 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 62044.94 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638424.594399 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558404.47 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 55840.45 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694265.041794 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502564.03 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 50256.40 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744521.444450 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452307.62 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 45230.76 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789752.206840 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407076.86 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 40707.69 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830459.892991 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366369.18 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 36636.92 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867096.810527 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329732.26 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 32973.23 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900070.036309 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296759.03 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 29675.90 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929745.939514 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267083.13 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 26708.31 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956454.252397 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240374.82 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 24037.48 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980491.733993 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216337.33 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 21633.73 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002125.467428 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194703.60 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 19470.36 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021595.827521 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175233.24 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 17523.32 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039119.151604 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157709.92 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 15770.99 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054890.143278 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141938.93 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 14193.89 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069084.035786 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127745.03 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 12774.50 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081858.539042 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114970.53 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 11497.05 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093355.591973 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103473.48 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 10347.35 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103702.939611 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93126.13 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 9312.61 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2113015.552485 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83813.52 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 8381.35 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121396.904071 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75432.16 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 7543.22 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128940.120499 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67888.95 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 6788.89 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135729.015285 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61100.05 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 6110.01 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141839.020591 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54990.05 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 5499.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147338.025367 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49491.04 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 4949.10 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152287.129666 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44541.94 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 4454.19 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156741.323534 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40087.74 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 4008.77 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160750.098016 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36078.97 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3607.90 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164357.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32471.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167957.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28871.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171557.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25271.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175157.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21671.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178757.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18071.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182357.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14471.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185957.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10871.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189557.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7271.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193157.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3671.07 sec) 23:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196757.995049 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 71.07 sec) 23:24:00 [GPUGRID] [rr_sim] 2196829.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966119042.57G/439.78G) 23:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854109.98 23:24:00 [---] pick_jobs_to_run() start 23:24:00 [---] pick_jobs_to_run() end 23:24:00 [---] [rr_sim] end 23:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:24:00 [---] [cpu_sched_debug] preliminary job list: 23:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:24:00 [---] [cpu_sched_debug] final job list: 23:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:24:00 [---] [cpu_sched_debug] enforce_run_list: end 23:24:00 [---] choose_project(): 84240.000000 23:24:00 [WUProp@Home] doing RPC to get work 23:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351359.09 cpu 2100992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100992.90 23:24:00 [---] Got 0 tasks 23:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:25:00 [---] [rr_sim] doing sim: CPU sched 23:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966091452.22G/439.78G) 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196766.33 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 219676.63 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219676.633147 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977089.70 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 197708.97 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417385.602979 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779380.73 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 177938.07 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595323.675828 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601442.66 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 160144.27 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755467.941392 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441298.39 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 144129.84 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899597.780400 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297168.55 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 129716.86 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029314.635507 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167451.70 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 116745.17 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146059.805103 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050706.53 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 105070.65 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251130.457739 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945635.87 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 94563.59 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345694.045112 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851072.29 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 85107.23 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430801.273748 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765965.06 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 76596.51 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507397.779520 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689368.55 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 68936.86 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576334.634715 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620431.70 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 62043.17 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638377.804390 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558388.53 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 55838.85 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694216.657098 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502549.67 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 50254.97 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744471.624535 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452294.71 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 45229.47 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789701.095229 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407065.24 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 40706.52 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830407.618853 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366358.71 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 36635.87 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1867043.490114 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329722.84 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 32972.28 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1900015.774250 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296750.56 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 29675.06 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929690.829972 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267075.50 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 26707.55 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956398.380121 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240367.95 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 24036.80 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980435.175256 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216331.16 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 21633.12 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002068.290877 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194698.04 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 19469.80 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021538.094936 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175228.24 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 17522.82 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039060.918590 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157705.41 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 15770.54 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054831.459878 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141934.87 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 14193.49 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2069024.947037 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127741.38 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 12774.14 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081799.085480 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114967.25 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 11496.72 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093295.810079 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103470.52 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 10347.05 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103642.862218 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93123.47 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 9312.35 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112955.209143 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83811.12 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 8381.11 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121336.321375 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75430.01 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 7543.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128879.322385 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67887.01 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 6788.70 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135668.023293 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61098.31 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 6109.83 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141777.854111 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54988.48 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 5498.85 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147276.701847 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49489.63 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 4948.96 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152225.664809 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44540.67 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 4454.07 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156679.731475 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40086.60 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 4008.66 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160688.391474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36077.94 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3607.79 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164296.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32470.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167896.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28870.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171496.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25270.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175096.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21670.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178696.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18070.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182296.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14470.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185896.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10870.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189496.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7270.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193096.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3670.15 sec) 23:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196696.185474 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 70.15 sec) 23:25:00 [GPUGRID] [rr_sim] 2196766.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966091452.22G/439.78G) 23:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854107.24 23:25:00 [---] pick_jobs_to_run() start 23:25:00 [---] pick_jobs_to_run() end 23:25:00 [---] [rr_sim] end 23:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:25:00 [---] [cpu_sched_debug] preliminary job list: 23:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:25:00 [---] [cpu_sched_debug] final job list: 23:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:25:00 [---] [cpu_sched_debug] enforce_run_list: end 23:25:00 [---] choose_project(): 84300.000000 23:25:00 [WUProp@Home] doing RPC to get work 23:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351299.09 cpu 2100932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100932.90 23:25:00 [---] Got 0 tasks 23:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:26:00 [---] [rr_sim] doing sim: CPU sched 23:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966063861.86G/439.78G) 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196703.59 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 219670.36 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219670.359459 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1977033.24 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 197703.32 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417373.682971 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779329.91 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 177932.99 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595306.674133 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601396.92 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 160139.69 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755446.366178 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441257.23 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 144125.72 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899572.089019 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297131.51 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 129713.15 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029285.239576 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167418.36 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 116741.84 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1146027.075077 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050676.52 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 105067.65 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251094.727028 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945608.87 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 94560.89 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345655.613784 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851047.98 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 85104.80 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430760.411864 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765943.18 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 76594.32 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507354.730136 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689348.86 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 68934.89 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576289.616581 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620413.98 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 62041.40 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638331.014382 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558372.58 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 55837.26 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694168.272402 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502535.32 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 50253.53 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744421.804621 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452281.79 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 45228.18 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789649.983617 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407053.61 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 40705.36 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830355.344714 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366348.25 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 36634.82 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866990.169702 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329713.42 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 32971.34 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899961.512190 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296742.08 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 29674.21 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929635.720430 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267067.87 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 26706.79 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956342.507845 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240361.09 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 24036.11 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980378.616519 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216324.98 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 21632.50 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2002011.114326 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194692.48 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 19469.25 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021480.362352 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175223.23 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 17522.32 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2039002.685576 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157700.91 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 15770.09 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054772.776477 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141930.82 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 14193.08 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068965.858288 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127737.74 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 12773.77 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081739.631918 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114963.96 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 11496.40 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093236.028185 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103467.57 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 10346.76 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103582.784825 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93120.81 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 9312.08 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112894.865801 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83808.73 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 8380.87 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121275.738679 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75427.86 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 7542.79 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128818.524270 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67885.07 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 6788.51 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135607.031302 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61096.56 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 6109.66 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141716.687630 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54986.91 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 5498.69 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147215.378326 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49488.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 4948.82 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152164.199952 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44539.39 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 4453.94 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156618.139415 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40085.46 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 4008.55 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160626.684933 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36076.91 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3607.69 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164234.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32469.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167834.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28869.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171434.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25269.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175034.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21669.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178634.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18069.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182234.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14469.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185834.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10869.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189434.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7269.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193034.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3669.22 sec) 23:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196634.375898 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 69.22 sec) 23:26:00 [GPUGRID] [rr_sim] 2196703.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966063861.86G/439.78G) 23:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854104.51 23:26:00 [---] pick_jobs_to_run() start 23:26:00 [---] pick_jobs_to_run() end 23:26:00 [---] [rr_sim] end 23:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:26:00 [---] [cpu_sched_debug] preliminary job list: 23:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:26:00 [---] [cpu_sched_debug] final job list: 23:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:26:00 [---] [cpu_sched_debug] enforce_run_list: end 23:26:00 [---] choose_project(): 84360.000000 23:26:00 [WUProp@Home] doing RPC to get work 23:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351239.09 cpu 2100872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100872.90 23:26:00 [---] Got 0 tasks 23:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:27:00 [---] [rr_sim] doing sim: CPU sched 23:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966036271.50G/439.78G) 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196640.86 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 219664.09 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219664.085770 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976976.77 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 197697.68 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417361.762964 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779279.09 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 177927.91 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595289.672438 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601351.19 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 160135.12 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755424.790965 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441216.07 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 144121.61 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899546.397639 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297094.46 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 129709.45 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029255.843645 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167385.01 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 116738.50 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145994.345051 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050646.51 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 105064.65 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251058.996316 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945581.86 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 94558.19 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345617.182455 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 851023.68 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 85102.37 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430719.549980 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765921.31 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 76592.13 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507311.680753 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689329.18 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 68932.92 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576244.598448 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620396.26 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 62039.63 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638284.224373 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558356.63 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 55835.66 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694119.887707 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502520.97 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 50252.10 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744371.984706 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452268.87 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 45226.89 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789598.872006 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407041.99 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 40704.20 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830303.070576 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366337.79 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 36633.78 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866936.849289 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329704.01 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 32970.40 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899907.250130 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296733.61 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 29673.36 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929580.610888 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267060.25 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 26706.02 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956286.635569 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240354.22 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 24035.42 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980322.057783 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216318.80 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 21631.88 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001953.937775 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194686.92 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 19468.69 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021422.629768 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175218.23 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 17521.82 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038944.452562 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157696.41 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 15769.64 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054714.093076 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141926.76 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 14192.68 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068906.769539 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127734.09 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 12773.41 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081680.178355 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114960.68 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 11496.07 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093176.246290 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103464.61 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 10346.46 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103522.707432 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93118.15 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 9311.82 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112834.522459 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83806.34 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 8380.63 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121215.155984 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75425.70 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 7542.57 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128757.726156 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67883.13 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 6788.31 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135546.039310 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61094.82 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 6109.48 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141655.521150 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54985.34 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 5498.53 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147154.054805 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49486.80 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 4948.68 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152102.735095 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44538.12 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 4453.81 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156556.547356 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40084.31 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 4008.43 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160564.978391 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36075.88 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3607.59 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164172.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32468.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167772.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28868.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171372.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25268.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174972.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21668.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178572.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18068.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182172.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14468.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185772.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10868.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189372.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7268.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192972.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3668.29 sec) 23:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196572.566322 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 68.29 sec) 23:27:00 [GPUGRID] [rr_sim] 2196640.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966036271.50G/439.78G) 23:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854101.77 23:27:00 [---] pick_jobs_to_run() start 23:27:00 [---] pick_jobs_to_run() end 23:27:00 [---] [rr_sim] end 23:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:27:00 [---] [cpu_sched_debug] preliminary job list: 23:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:27:00 [---] [cpu_sched_debug] final job list: 23:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:27:00 [---] [cpu_sched_debug] enforce_run_list: end 23:27:00 [---] choose_project(): 84420.000000 23:27:00 [WUProp@Home] doing RPC to get work 23:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351179.09 cpu 2100812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100812.90 23:27:00 [---] Got 0 tasks 23:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:28:00 [---] [rr_sim] doing sim: CPU sched 23:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (966008681.15G/439.78G) 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196578.12 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 219657.81 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219657.812082 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976920.31 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 197692.03 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417349.842956 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779228.28 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 177922.83 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595272.670743 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601305.45 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 160130.55 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755403.215751 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441174.91 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 144117.49 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899520.706258 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297057.41 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 129705.74 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029226.447714 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167351.67 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 116735.17 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145961.615025 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050616.51 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 105061.65 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1251023.265605 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945554.86 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 94555.49 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345578.751127 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850999.37 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 85099.94 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430678.688096 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765899.43 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 76589.94 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507268.631369 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689309.49 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 68930.95 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576199.580314 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620378.54 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 62037.85 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638237.434365 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558340.69 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 55834.07 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694071.503011 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502506.62 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 50250.66 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744322.164792 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452255.96 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 45225.60 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789547.760395 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407030.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 40703.04 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830250.796438 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366327.32 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 36632.73 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866883.528876 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329694.59 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 32969.46 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899852.988071 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296725.13 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 29672.51 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929525.501346 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267052.62 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 26705.26 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956230.763294 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240347.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 24034.74 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980265.499046 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216312.62 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 21631.26 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001896.761224 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194681.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 19468.14 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021364.897184 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175213.22 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 17521.32 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038886.219548 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157691.90 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 15769.19 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054655.409675 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141922.71 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 14192.27 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068847.680790 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127730.44 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 12773.04 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081620.724793 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114957.40 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 11495.74 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093116.464396 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103461.66 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 10346.17 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103462.630039 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93115.49 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 9311.55 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112774.179117 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83803.94 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 8380.39 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121154.573288 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75423.55 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 7542.35 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128696.928041 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67881.19 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 6788.12 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135485.047319 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61093.07 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 6109.31 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141594.354669 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54983.77 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 5498.38 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147092.731285 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49485.39 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 4948.54 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152041.270238 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44536.85 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 4453.69 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156494.955297 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40083.17 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 4008.32 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160503.271849 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36074.85 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3607.48 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164110.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32467.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167710.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28867.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171310.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25267.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174910.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21667.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178510.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18067.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182110.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14467.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185710.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10867.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189310.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7267.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192910.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3667.36 sec) 23:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196510.756747 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67.36 sec) 23:28:00 [GPUGRID] [rr_sim] 2196578.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (966008681.15G/439.78G) 23:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854099.03 23:28:00 [---] pick_jobs_to_run() start 23:28:00 [---] pick_jobs_to_run() end 23:28:00 [---] [rr_sim] end 23:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:28:00 [---] [cpu_sched_debug] preliminary job list: 23:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:28:00 [---] [cpu_sched_debug] final job list: 23:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:28:00 [---] [cpu_sched_debug] enforce_run_list: end 23:28:00 [---] choose_project(): 84480.000000 23:28:00 [WUProp@Home] doing RPC to get work 23:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351119.09 cpu 2100752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100752.90 23:28:00 [---] Got 0 tasks 23:29:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:29:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:29:00 [---] [rr_sim] doing sim: CPU sched 23:29:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965981090.79G/439.78G) 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196515.38 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 219651.54 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219651.538394 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976863.85 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 197686.38 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417337.922949 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779177.46 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 177917.75 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595255.669048 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601259.71 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 160125.97 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755381.640537 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441133.74 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 144113.37 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899495.014877 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1297020.37 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 129702.04 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029197.051784 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167318.33 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 116731.83 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145928.884999 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050586.50 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 105058.65 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250987.534893 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945527.85 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 94552.78 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345540.319798 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850975.06 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 85097.51 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430637.826212 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765877.56 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 76587.76 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507225.581985 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689289.80 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 68928.98 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576154.562181 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620360.82 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 62036.08 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638190.644356 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558324.74 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 55832.47 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1694023.118315 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502492.27 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 50249.23 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744272.344877 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452243.04 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 45224.30 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789496.648784 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407018.74 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 40701.87 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830198.522299 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366316.86 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 36631.69 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866830.208463 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329685.18 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 32968.52 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899798.726011 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296716.66 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 29671.67 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929470.391804 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267044.99 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 26704.50 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956174.891018 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240340.49 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 24034.05 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980208.940310 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216306.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 21630.64 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001839.584673 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194675.80 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 19467.58 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021307.164600 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175208.22 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 17520.82 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038827.986534 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157687.40 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 15768.74 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054596.726274 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141918.66 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 14191.87 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068788.592041 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127726.79 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 12772.68 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081561.271231 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114954.11 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 11495.41 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2093056.682502 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103458.70 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 10345.87 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103402.552646 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93112.83 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 9311.28 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112713.835775 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83801.55 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 8380.15 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121093.990592 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75421.39 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 7542.14 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128636.129926 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67879.25 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 6787.93 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135424.055328 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61091.33 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 6109.13 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141533.188189 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54982.20 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 5498.22 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147031.407764 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49483.98 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 4948.40 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151979.805382 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44535.58 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 4453.56 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156433.363238 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40082.02 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 4008.20 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160441.565308 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36073.82 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3607.38 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164048.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32466.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167648.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28866.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171248.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25266.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174848.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21666.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178448.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18066.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182048.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14466.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185648.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10866.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189248.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7266.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192848.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3666.44 sec) 23:29:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196448.947171 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 66.44 sec) 23:29:00 [GPUGRID] [rr_sim] 2196515.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965981090.79G/439.78G) 23:29:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854096.30 23:29:00 [---] pick_jobs_to_run() start 23:29:00 [---] pick_jobs_to_run() end 23:29:00 [---] [rr_sim] end 23:29:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:29:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:29:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:29:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:29:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:29:00 [---] [cpu_sched_debug] preliminary job list: 23:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:29:00 [---] [cpu_sched_debug] final job list: 23:29:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:29:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:29:00 [---] [cpu_sched_debug] enforce_run_list: end 23:29:00 [---] choose_project(): 84540.000000 23:29:00 [WUProp@Home] doing RPC to get work 23:29:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 351059.09 cpu 2100692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100692.90 23:29:00 [---] Got 0 tasks 23:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:30:00 [---] [rr_sim] doing sim: CPU sched 23:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965953500.44G/439.78G) 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196452.65 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 219645.26 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219645.264706 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976807.38 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 197680.74 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417326.002941 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779126.64 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 177912.66 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595238.667353 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601213.98 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 160121.40 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755360.065323 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441092.58 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 144109.26 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899469.323497 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296983.32 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 129698.33 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029167.655853 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167284.99 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 116728.50 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145896.154973 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050556.49 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 105055.65 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250951.804182 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945500.84 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 94550.08 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345501.888469 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850950.76 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 85095.08 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430596.964328 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765855.68 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 76585.57 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507182.532601 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689270.11 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 68927.01 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576109.544047 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620343.10 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 62034.31 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638143.854348 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558308.79 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 55830.88 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693974.733619 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502477.91 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 50247.79 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744222.524963 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452230.12 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 45223.01 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789445.537172 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 407007.11 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 40700.71 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830146.248161 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366306.40 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 36630.64 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866776.888051 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329675.76 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 32967.58 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899744.463951 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296708.18 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 29670.82 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929415.282262 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267037.36 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 26703.74 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956119.018742 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240333.63 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 24033.36 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980152.381573 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216300.27 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 21630.03 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001782.408122 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194670.24 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 19467.02 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021249.432015 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175203.22 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 17520.32 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038769.753520 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157682.89 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 15768.29 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054538.042873 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141914.60 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 14191.46 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068729.503292 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127723.14 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 12772.31 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081501.817668 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114950.83 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 11495.08 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092996.900607 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103455.75 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 10345.57 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103342.475252 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93110.17 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 9311.02 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112653.492433 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83799.15 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 8379.92 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2121033.407896 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75419.24 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 7541.92 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128575.331812 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67877.32 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 6787.73 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135363.063336 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61089.58 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 6108.96 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141472.021709 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54980.63 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 5498.06 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146970.084243 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49482.56 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 4948.26 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151918.340525 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44534.31 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 4453.43 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156371.771178 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40080.88 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 4008.09 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160379.858766 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36072.79 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3607.28 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163987.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32465.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167587.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28865.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171187.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25265.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174787.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21665.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178387.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18065.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181987.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14465.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185587.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10865.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189187.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7265.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192787.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3665.51 sec) 23:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196387.137595 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 65.51 sec) 23:30:00 [GPUGRID] [rr_sim] 2196452.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965953500.44G/439.78G) 23:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854093.56 23:30:00 [---] pick_jobs_to_run() start 23:30:00 [---] pick_jobs_to_run() end 23:30:00 [---] [rr_sim] end 23:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:30:00 [---] [cpu_sched_debug] preliminary job list: 23:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:30:00 [---] [cpu_sched_debug] final job list: 23:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:30:00 [---] [cpu_sched_debug] enforce_run_list: end 23:30:00 [---] choose_project(): 84600.000000 23:30:00 [WUProp@Home] doing RPC to get work 23:30:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350999.09 cpu 2100632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100632.90 23:30:00 [---] Got 0 tasks 23:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:31:00 [---] [rr_sim] doing sim: CPU sched 23:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965925910.08G/439.78G) 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196389.91 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 219638.99 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219638.991018 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976750.92 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 197675.09 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417314.082933 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779075.83 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 177907.58 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595221.665658 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601168.24 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 160116.82 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755338.490109 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441051.42 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 144105.14 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899443.632116 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296946.28 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 129694.63 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029138.259922 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167251.65 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 116725.17 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145863.424947 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050526.49 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 105052.65 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250916.073470 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945473.84 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 94547.38 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345463.457141 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850926.45 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 85092.65 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430556.102444 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765833.81 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 76583.38 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507139.483217 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689250.43 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 68925.04 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576064.525913 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620325.38 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 62032.54 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638097.064340 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558292.85 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 55829.28 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693926.348923 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502463.56 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 50246.36 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744172.705048 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452217.21 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 45221.72 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789394.425561 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406995.48 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 40699.55 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830093.974023 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366295.94 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 36629.59 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866723.567638 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329666.34 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 32966.63 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899690.201892 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296699.71 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 29669.97 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929360.172720 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267029.74 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 26702.97 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956063.146466 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240326.76 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 24032.68 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980095.822837 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216294.09 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 21629.41 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001725.231571 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194664.68 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 19466.47 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021191.699431 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175198.21 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 17519.82 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038711.520506 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157678.39 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 15767.84 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054479.359473 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141910.55 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 14191.06 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068670.414543 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127719.50 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 12771.95 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081442.364106 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114947.55 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 11494.75 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092937.118713 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103452.79 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 10345.28 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103282.397859 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93107.51 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 9310.75 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112593.149091 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83796.76 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 8379.68 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120972.825200 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75417.08 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 7541.71 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128514.533697 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67875.38 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 6787.54 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135302.071345 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61087.84 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 6108.78 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141410.855228 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54979.05 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 5497.91 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146908.760723 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49481.15 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 4948.11 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151856.875668 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44533.03 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 4453.30 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156310.179119 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40079.73 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 4007.97 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160318.152225 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36071.76 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3607.18 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163925.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32464.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167525.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28864.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171125.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25264.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174725.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21664.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178325.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18064.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181925.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14464.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185525.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10864.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189125.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7264.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192725.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3664.58 sec) 23:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196325.328020 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 64.58 sec) 23:31:00 [GPUGRID] [rr_sim] 2196389.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965925910.08G/439.78G) 23:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854090.82 23:31:00 [---] pick_jobs_to_run() start 23:31:00 [---] pick_jobs_to_run() end 23:31:00 [---] [rr_sim] end 23:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:31:00 [---] [cpu_sched_debug] preliminary job list: 23:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:31:00 [---] [cpu_sched_debug] final job list: 23:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:31:00 [---] [cpu_sched_debug] enforce_run_list: end 23:31:00 [---] choose_project(): 84660.000000 23:31:00 [WUProp@Home] doing RPC to get work 23:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350939.09 cpu 2100572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100572.90 23:31:00 [---] Got 0 tasks 23:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:32:00 [---] [rr_sim] doing sim: CPU sched 23:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965898319.72G/439.78G) 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196327.17 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 219632.72 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219632.717329 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976694.46 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 197669.45 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417302.162926 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1779025.01 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 177902.50 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595204.663963 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601122.51 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 160112.25 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755316.914896 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1441010.26 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 144101.03 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899417.940735 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296909.23 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 129690.92 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029108.863991 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167218.31 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 116721.83 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145830.694922 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050496.48 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 105049.65 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250880.342759 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945446.83 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 94544.68 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345425.025812 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850902.15 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 85090.21 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430515.240560 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765811.93 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 76581.19 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507096.433834 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689230.74 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 68923.07 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1576019.507780 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620307.67 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 62030.77 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638050.274331 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558276.90 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 55827.69 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693877.964227 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502449.21 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 50244.92 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744122.885134 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452204.29 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 45220.43 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789343.313950 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406983.86 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 40698.39 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1830041.699884 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366285.47 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 36628.55 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866670.247225 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329656.93 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 32965.69 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899635.939832 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296691.23 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 29669.12 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929305.063178 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267022.11 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 26702.21 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1956007.274190 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240319.90 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 24031.99 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1980039.264100 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216287.91 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 21628.79 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001668.055019 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194659.12 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 19465.91 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021133.966847 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175193.21 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 17519.32 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038653.287492 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157673.89 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 15767.39 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054420.676072 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141906.50 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 14190.65 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068611.325794 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127715.85 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 12771.58 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081382.910544 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114944.26 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 11494.43 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092877.336819 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103449.84 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 10344.98 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103222.320466 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93104.85 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 9310.49 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112532.805749 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83794.37 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 8379.44 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120912.242504 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75414.93 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 7541.49 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128453.735583 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67873.44 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 6787.34 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135241.079354 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61086.09 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 6108.61 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141349.688748 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54977.48 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 5497.75 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146847.437202 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49479.74 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 4947.97 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151795.410811 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44531.76 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 4453.18 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156248.587060 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40078.59 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 4007.86 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160256.445683 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36070.73 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3607.07 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163863.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32463.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167463.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28863.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171063.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25263.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174663.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21663.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178263.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18063.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181863.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14463.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185463.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10863.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189063.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7263.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192663.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3663.65 sec) 23:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196263.518444 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 63.65 sec) 23:32:00 [GPUGRID] [rr_sim] 2196327.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965898319.72G/439.78G) 23:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854088.09 23:32:00 [---] pick_jobs_to_run() start 23:32:00 [---] pick_jobs_to_run() end 23:32:00 [---] [rr_sim] end 23:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:32:00 [---] [cpu_sched_debug] preliminary job list: 23:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:32:00 [---] [cpu_sched_debug] final job list: 23:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:32:00 [---] [cpu_sched_debug] enforce_run_list: end 23:32:00 [---] choose_project(): 84720.000000 23:32:00 [WUProp@Home] doing RPC to get work 23:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350879.09 cpu 2100512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100512.90 23:32:00 [---] Got 0 tasks 23:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:33:00 [---] [rr_sim] doing sim: CPU sched 23:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965870729.37G/439.78G) 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196264.44 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 219626.44 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219626.443641 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976637.99 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 197663.80 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417290.242918 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778974.19 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 177897.42 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595187.662268 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601076.77 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 160107.68 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755295.339682 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440969.10 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 144096.91 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899392.249355 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296872.19 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 129687.22 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029079.468061 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167184.97 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 116718.50 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145797.964896 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050466.47 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 105046.65 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250844.612047 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945419.82 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 94541.98 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345386.594484 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850877.84 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 85087.78 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430474.378676 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765790.06 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 76579.01 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507053.384450 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689211.05 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 68921.11 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575974.489646 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620289.95 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 62028.99 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1638003.484323 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558260.95 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 55826.10 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693829.579531 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502434.86 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 50243.49 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744073.065219 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452191.37 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 45219.14 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789292.202339 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406972.23 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 40697.22 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829989.425746 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366275.01 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 36627.50 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866616.926813 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329647.51 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 32964.75 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899581.677772 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296682.76 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 29668.28 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929249.953636 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267014.48 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 26701.45 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955951.401914 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240313.03 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 24031.30 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979982.705364 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216281.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 21628.17 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001610.878468 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194653.56 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 19465.36 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021076.234263 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175188.20 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 17518.82 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038595.054478 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157669.38 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 15766.94 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054361.992671 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141902.44 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 14190.24 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068552.237045 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127712.20 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 12771.22 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081323.456982 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114940.98 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 11494.10 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092817.554925 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103446.88 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 10344.69 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103162.243073 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93102.19 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 9310.22 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112472.462407 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83791.97 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 8379.20 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120851.659808 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75412.78 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 7541.28 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128392.937468 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67871.50 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 6787.15 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135180.087362 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61084.35 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 6108.43 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141288.522267 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54975.91 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 5497.59 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146786.113682 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49478.32 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 4947.83 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151733.945955 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44530.49 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 4453.05 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156186.995000 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40077.44 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 4007.74 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160194.739141 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36069.70 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3606.97 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163801.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32462.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167401.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28862.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171001.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25262.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174601.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21662.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178201.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18062.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181801.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14462.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185401.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10862.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189001.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7262.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192601.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3662.73 sec) 23:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196201.708868 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 62.73 sec) 23:33:00 [GPUGRID] [rr_sim] 2196264.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965870729.37G/439.78G) 23:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854085.35 23:33:00 [---] pick_jobs_to_run() start 23:33:00 [---] pick_jobs_to_run() end 23:33:00 [---] [rr_sim] end 23:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:33:00 [---] [cpu_sched_debug] preliminary job list: 23:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:33:00 [---] [cpu_sched_debug] final job list: 23:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:33:00 [---] [cpu_sched_debug] enforce_run_list: end 23:33:00 [---] choose_project(): 84780.000000 23:33:00 [WUProp@Home] doing RPC to get work 23:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350819.09 cpu 2100452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100452.90 23:33:00 [---] Got 0 tasks 23:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:34:00 [---] [rr_sim] doing sim: CPU sched 23:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965843139.01G/439.78G) 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196201.70 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 219620.17 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219620.169953 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976581.53 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 197658.15 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417278.322911 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778923.38 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 177892.34 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595170.660572 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1601031.04 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 160103.10 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755273.764468 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440927.94 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 144092.79 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899366.557974 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296835.14 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 129683.51 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029050.072130 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167151.63 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 116715.16 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145765.234870 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050436.46 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 105043.65 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250808.881336 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945392.82 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 94539.28 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345348.163155 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850853.54 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 85085.35 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430433.516792 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765768.18 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 76576.82 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1507010.335066 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689191.36 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 68919.14 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575929.471512 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620272.23 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 62027.22 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637956.694314 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558245.01 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 55824.50 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693781.194836 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502420.50 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 50242.05 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1744023.245305 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452178.45 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 45217.85 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789241.090727 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406960.61 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 40696.06 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829937.151608 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366264.55 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 36626.45 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866563.606400 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329638.09 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 32963.81 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899527.415713 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296674.28 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 29667.43 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929194.844094 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 267006.86 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 26700.69 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955895.529638 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240306.17 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 24030.62 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979926.146627 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216275.55 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 21627.56 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001553.701917 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194648.00 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 19464.80 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2021018.501678 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175183.20 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 17518.32 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038536.821464 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157664.88 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 15766.49 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054303.309270 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141898.39 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 14189.84 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068493.148296 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127708.55 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 12770.86 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081264.003419 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114937.70 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 11493.77 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092757.773030 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103443.93 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 10344.39 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103102.165680 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93099.53 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 9309.95 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112412.119065 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83789.58 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 8378.96 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120791.077112 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75410.62 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 7541.06 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128332.139353 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67869.56 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 6786.96 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135119.095371 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61082.60 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 6108.26 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141227.355787 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54974.34 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 5497.43 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146724.790161 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49476.91 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 4947.69 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151672.481098 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44529.22 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 4452.92 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156125.402941 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40076.30 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 4007.63 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160133.032600 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36068.67 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3606.87 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163739.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32461.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167339.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28861.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170939.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25261.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174539.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21661.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178139.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18061.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181739.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14461.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185339.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10861.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188939.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7261.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192539.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3661.80 sec) 23:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196139.899293 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61.80 sec) 23:34:00 [GPUGRID] [rr_sim] 2196201.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965843139.01G/439.78G) 23:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854082.61 23:34:00 [---] pick_jobs_to_run() start 23:34:00 [---] pick_jobs_to_run() end 23:34:00 [---] [rr_sim] end 23:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:34:00 [---] [cpu_sched_debug] preliminary job list: 23:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:34:00 [---] [cpu_sched_debug] final job list: 23:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:34:00 [---] [cpu_sched_debug] enforce_run_list: end 23:34:00 [---] choose_project(): 84840.000000 23:34:00 [WUProp@Home] doing RPC to get work 23:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350759.09 cpu 2100392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100392.90 23:34:00 [---] Got 0 tasks 23:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:35:00 [---] [rr_sim] doing sim: CPU sched 23:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965815548.65G/439.78G) 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196138.96 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 219613.90 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219613.896265 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976525.07 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 197652.51 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417266.402903 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778872.56 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 177887.26 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595153.658877 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600985.30 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 160098.53 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755252.189254 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440886.77 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 144088.68 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899340.866594 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296798.10 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 129679.81 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1029020.676199 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167118.29 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 116711.83 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145732.504844 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050406.46 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 105040.65 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250773.150624 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945365.81 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 94536.58 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345309.731826 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850829.23 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 85082.92 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430392.654908 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765746.31 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 76574.63 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506967.285682 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689171.68 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 68917.17 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575884.453379 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620254.51 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 62025.45 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637909.904306 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558229.06 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 55822.91 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693732.810140 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502406.15 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 50240.62 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743973.425391 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452165.54 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 45216.55 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789189.979116 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406948.98 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 40694.90 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829884.877469 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366254.09 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 36625.41 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866510.285987 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329628.68 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 32962.87 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899473.153653 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296665.81 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 29666.58 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929139.734552 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266999.23 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 26699.92 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955839.657362 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240299.31 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 24029.93 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979869.587890 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216269.37 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 21626.94 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001496.525366 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194642.44 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 19464.24 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020960.769094 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175178.19 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 17517.82 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038478.588450 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157660.37 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 15766.04 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054244.625869 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141894.34 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 14189.43 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068434.059547 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127704.90 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 12770.49 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081204.549857 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114934.41 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 11493.44 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092697.991136 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103440.97 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 10344.10 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2103042.088287 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93096.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 9309.69 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112351.775723 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83787.19 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 8378.72 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120730.494416 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75408.47 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 7540.85 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128271.341239 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67867.62 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 6786.76 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2135058.103380 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61080.86 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 6108.09 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141166.189306 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54972.77 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 5497.28 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146663.466640 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49475.50 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 4947.55 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151611.016241 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44527.95 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 4452.79 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156063.810882 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40075.15 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 4007.52 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160071.326058 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36067.64 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3606.76 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163678.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32460.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167278.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28860.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170878.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25260.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174478.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21660.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178078.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18060.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181678.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14460.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185278.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10860.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188878.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7260.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192478.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3660.87 sec) 23:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196078.089717 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60.87 sec) 23:35:00 [GPUGRID] [rr_sim] 2196138.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965815548.65G/439.78G) 23:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854079.88 23:35:00 [---] pick_jobs_to_run() start 23:35:00 [---] pick_jobs_to_run() end 23:35:00 [---] [rr_sim] end 23:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:35:00 [---] [cpu_sched_debug] preliminary job list: 23:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:35:00 [---] [cpu_sched_debug] final job list: 23:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:35:00 [---] [cpu_sched_debug] enforce_run_list: end 23:35:00 [---] choose_project(): 84900.000000 23:35:00 [WUProp@Home] doing RPC to get work 23:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350699.09 cpu 2100332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100332.90 23:35:00 [---] Got 0 tasks 23:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:36:00 [---] [rr_sim] doing sim: CPU sched 23:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965787958.30G/439.78G) 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196076.23 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 219607.62 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219607.622576 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976468.60 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 197646.86 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417254.482895 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778821.74 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 177882.17 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595136.657182 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600939.57 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 160093.96 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755230.614041 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440845.61 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 144084.56 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899315.175213 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296761.05 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 129676.11 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028991.280268 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167084.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 116708.49 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145699.774818 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050376.45 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 105037.65 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250737.419913 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945338.81 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 94533.88 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345271.300498 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850804.93 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 85080.49 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430351.793025 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765724.43 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 76572.44 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506924.236299 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689151.99 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 68915.20 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575839.435245 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620236.79 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 62023.68 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637863.114297 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558213.11 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 55821.31 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693684.425444 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502391.80 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 50239.18 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743923.605476 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452152.62 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 45215.26 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789138.867505 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406937.36 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 40693.74 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829832.603331 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366243.62 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 36624.36 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866456.965574 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329619.26 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 32961.93 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899418.891593 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296657.33 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 29665.73 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929084.625011 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266991.60 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 26699.16 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955783.785086 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240292.44 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 24029.24 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979813.029154 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216263.20 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 21626.32 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001439.348815 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194636.88 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 19463.69 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020903.036510 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175173.19 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 17517.32 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038420.355435 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157655.87 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 15765.59 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054185.942468 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141890.28 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 14189.03 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068374.970798 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127701.25 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 12770.13 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081145.096295 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114931.13 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 11493.11 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092638.209242 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103438.02 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 10343.80 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102982.010894 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93094.21 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 9309.42 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112291.432381 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83784.79 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 8378.48 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120669.911720 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75406.31 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 7540.63 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128210.543124 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67865.68 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 6786.57 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134997.111388 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61079.11 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 6107.91 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141105.022826 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54971.20 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 5497.12 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146602.143120 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49474.08 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 4947.41 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151549.551384 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44526.67 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 4452.67 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156002.218822 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40074.01 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 4007.40 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160009.619517 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36066.61 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3606.66 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163616.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32459.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167216.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28859.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170816.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25259.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174416.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21659.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178016.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18059.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181616.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14459.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185216.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10859.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188816.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7259.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192416.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3659.95 sec) 23:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2196016.280141 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 59.95 sec) 23:36:00 [GPUGRID] [rr_sim] 2196076.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965787958.30G/439.78G) 23:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854077.14 23:36:00 [---] pick_jobs_to_run() start 23:36:00 [---] pick_jobs_to_run() end 23:36:00 [---] [rr_sim] end 23:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:36:00 [---] [cpu_sched_debug] preliminary job list: 23:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:36:00 [---] [cpu_sched_debug] final job list: 23:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:36:00 [---] [cpu_sched_debug] enforce_run_list: end 23:36:00 [---] choose_project(): 84960.000000 23:36:00 [WUProp@Home] doing RPC to get work 23:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350639.09 cpu 2100272.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100272.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100272.90 23:36:00 [---] Got 0 tasks 23:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:37:00 [---] [rr_sim] doing sim: CPU sched 23:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965760367.94G/439.78G) 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2196013.49 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 219601.35 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219601.348888 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976412.14 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 197641.21 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417242.562888 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778770.93 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 177877.09 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595119.655487 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600893.83 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 160089.38 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755209.038827 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440804.45 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 144080.45 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899289.483832 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296724.01 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 129672.40 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028961.884337 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167051.60 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 116705.16 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145667.044792 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050346.44 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 105034.64 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250701.689201 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945311.80 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 94531.18 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345232.869169 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850780.62 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 85078.06 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430310.931141 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765702.56 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 76570.26 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506881.186915 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689132.30 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 68913.23 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575794.417112 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620219.07 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 62021.91 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637816.324289 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558197.16 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 55819.72 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693636.040748 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502377.45 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 50237.74 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743873.785562 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452139.70 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 45213.97 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789087.755894 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406925.73 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 40692.57 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829780.329193 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366233.16 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 36623.32 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866403.645162 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329609.84 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 32960.98 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899364.629534 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296648.86 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 29664.89 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1929029.515469 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266983.97 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 26698.40 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955727.912810 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240285.58 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 24028.56 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979756.470417 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216257.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 21625.70 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001382.172264 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194631.32 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 19463.13 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020845.303926 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175168.18 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 17516.82 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038362.122421 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157651.37 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 15765.14 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054127.259068 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141886.23 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 14188.62 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068315.882049 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127697.61 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 12769.76 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081085.642732 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114927.85 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 11492.78 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092578.427348 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103435.06 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 10343.51 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102921.933501 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93091.56 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 9309.16 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112231.089039 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83782.40 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 8378.24 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120609.329024 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75404.16 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 7540.42 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128149.745010 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67863.74 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 6786.37 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134936.119397 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61077.37 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 6107.74 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141043.856345 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54969.63 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 5496.96 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146540.819599 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49472.67 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 4947.27 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151488.086528 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44525.40 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 4452.54 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155940.626763 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40072.86 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 4007.29 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159947.912975 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36065.58 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3606.56 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163554.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32459.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167154.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28859.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170754.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25259.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174354.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21659.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177954.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18059.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181554.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14459.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185154.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10859.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188754.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7259.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192354.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3659.02 sec) 23:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195954.470566 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 59.02 sec) 23:37:00 [GPUGRID] [rr_sim] 2196013.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965760367.94G/439.78G) 23:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854074.40 23:37:00 [---] pick_jobs_to_run() start 23:37:00 [---] pick_jobs_to_run() end 23:37:00 [---] [rr_sim] end 23:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:37:00 [---] [cpu_sched_debug] preliminary job list: 23:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:37:00 [---] [cpu_sched_debug] final job list: 23:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:37:00 [---] [cpu_sched_debug] enforce_run_list: end 23:37:00 [---] choose_project(): 85020.000000 23:37:00 [WUProp@Home] doing RPC to get work 23:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350579.09 cpu 2100212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100212.90 23:37:00 [---] Got 0 tasks 23:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:38:00 [---] [rr_sim] doing sim: CPU sched 23:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965732777.59G/439.78G) 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195950.75 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 219595.08 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219595.075200 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976355.68 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 197635.57 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417230.642880 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778720.11 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 177872.01 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595102.653792 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600848.10 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 160084.81 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755187.463613 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440763.29 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 144076.33 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899263.792452 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296686.96 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 129668.70 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028932.488407 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1167018.26 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 116701.83 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145634.314766 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050316.44 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 105031.64 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250665.958490 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945284.79 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 94528.48 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345194.437841 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850756.31 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 85075.63 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430270.069257 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765680.68 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 76568.07 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506838.137531 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689112.61 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 68911.26 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575749.398978 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620201.35 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 62020.14 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637769.534280 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558181.22 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 55818.12 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693587.656052 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502363.10 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 50236.31 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743823.965647 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452126.79 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 45212.68 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1789036.644282 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406914.11 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 40691.41 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829728.055054 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366222.70 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 36622.27 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866350.324749 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329600.43 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 32960.04 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899310.367474 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296640.38 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 29664.04 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928974.405927 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266976.35 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 26697.63 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955672.040534 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240278.71 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 24027.87 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979699.911681 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216250.84 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 21625.08 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001324.995713 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194625.76 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 19462.58 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020787.571342 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175163.18 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 17516.32 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038303.889407 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157646.86 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 15764.69 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054068.575667 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141882.18 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 14188.22 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068256.793300 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127693.96 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 12769.40 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2081026.189170 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114924.56 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 11492.46 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092518.645453 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103432.11 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 10343.21 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102861.856108 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93088.90 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 9308.89 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112170.745697 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83780.01 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 8378.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120548.746328 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75402.01 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 7540.20 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128088.946895 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67861.81 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 6786.18 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134875.127405 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61075.62 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 6107.56 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140982.689865 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54968.06 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 5496.81 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146479.496079 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49471.26 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 4947.13 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151426.621671 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44524.13 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 4452.41 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155879.034704 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40071.72 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 4007.17 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159886.206433 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36064.55 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3606.45 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163492.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32458.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167092.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28858.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170692.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25258.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174292.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21658.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177892.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18058.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181492.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14458.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185092.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10858.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188692.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7258.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192292.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3658.09 sec) 23:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195892.660990 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 58.09 sec) 23:38:00 [GPUGRID] [rr_sim] 2195950.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965732777.59G/439.78G) 23:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854071.66 23:38:00 [---] pick_jobs_to_run() start 23:38:00 [---] pick_jobs_to_run() end 23:38:00 [---] [rr_sim] end 23:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:38:00 [---] [cpu_sched_debug] preliminary job list: 23:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:38:00 [---] [cpu_sched_debug] final job list: 23:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:38:00 [---] [cpu_sched_debug] enforce_run_list: end 23:38:00 [---] choose_project(): 85080.000000 23:38:00 [WUProp@Home] doing RPC to get work 23:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350519.09 cpu 2100152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100152.90 23:38:00 [---] Got 0 tasks 23:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:39:00 [---] [rr_sim] doing sim: CPU sched 23:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965705187.23G/439.78G) 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195888.02 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 219588.80 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219588.801512 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976299.21 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 197629.92 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417218.722873 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778669.29 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 177866.93 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595085.652097 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600802.36 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 160080.24 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755165.888399 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440722.13 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 144072.21 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899238.101071 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296649.91 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 129664.99 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028903.092476 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166984.92 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 116698.49 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145601.584740 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050286.43 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 105028.64 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250630.227778 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945257.79 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 94525.78 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345156.006512 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850732.01 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 85073.20 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430229.207373 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765658.81 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 76565.88 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506795.088147 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689092.93 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 68909.29 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575704.380844 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620183.63 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 62018.36 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637722.744272 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558165.27 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 55816.53 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693539.271356 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502348.74 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 50234.87 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743774.145733 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452113.87 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 45211.39 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788985.532671 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406902.48 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 40690.25 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829675.780916 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366212.23 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 36621.22 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866297.004336 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329591.01 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 32959.10 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899256.105414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296631.91 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 29663.19 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928919.296385 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266968.72 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 26696.87 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955616.168258 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240271.85 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 24027.18 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979643.352944 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216244.66 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 21624.47 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001267.819162 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194620.20 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 19462.02 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020729.838757 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175158.18 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 17515.82 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038245.656393 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157642.36 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 15764.24 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2054009.892266 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141878.12 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 14187.81 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068197.704551 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127690.31 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 12769.03 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080966.735608 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114921.28 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 11492.13 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092458.863559 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103429.15 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 10342.92 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102801.778715 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93086.24 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 9308.62 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112110.402355 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83777.61 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 8377.76 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120488.163632 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75399.85 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 7539.99 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2128028.148780 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67859.87 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 6785.99 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134814.135414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61073.88 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 6107.39 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140921.523385 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54966.49 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 5496.65 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146418.172558 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49469.84 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 4946.98 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151365.156814 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44522.86 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 4452.29 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155817.442644 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40070.57 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 4007.06 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159824.499892 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36063.52 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3606.35 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163430.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32457.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167030.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28857.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170630.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25257.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174230.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21657.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177830.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18057.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181430.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14457.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185030.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10857.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188630.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7257.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192230.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3657.16 sec) 23:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195830.851414 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 57.16 sec) 23:39:00 [GPUGRID] [rr_sim] 2195888.02: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965705187.23G/439.78G) 23:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854068.93 23:39:00 [---] pick_jobs_to_run() start 23:39:00 [---] pick_jobs_to_run() end 23:39:00 [---] [rr_sim] end 23:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:39:00 [---] [cpu_sched_debug] preliminary job list: 23:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:39:00 [---] [cpu_sched_debug] final job list: 23:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:39:00 [---] [cpu_sched_debug] enforce_run_list: end 23:39:00 [---] choose_project(): 85140.000000 23:39:00 [WUProp@Home] doing RPC to get work 23:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350459.09 cpu 2100092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100092.90 23:39:00 [---] Got 0 tasks 23:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:40:00 [---] [rr_sim] doing sim: CPU sched 23:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965677596.87G/439.78G) 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195825.28 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 219582.53 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219582.527824 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976242.75 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 197624.28 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417206.802865 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778618.48 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 177861.85 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595068.650402 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600756.63 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 160075.66 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755144.313185 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440680.97 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 144068.10 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899212.409691 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296612.87 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 129661.29 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028873.696545 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166951.58 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 116695.16 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145568.854714 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050256.42 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 105025.64 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250594.497066 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945230.78 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 94523.08 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345117.575183 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850707.70 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 85070.77 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430188.345489 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765636.93 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 76563.69 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506752.038763 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689073.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 68907.32 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575659.362711 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620165.92 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 62016.59 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637675.954263 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558149.32 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 55814.93 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693490.886661 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502334.39 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 50233.44 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743724.325818 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452100.95 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 45210.10 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788934.421060 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406890.86 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 40689.09 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829623.506778 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366201.77 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 36620.18 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866243.683923 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329581.59 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 32958.16 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899201.843355 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296623.43 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 29662.34 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928864.186843 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266961.09 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 26696.11 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955560.295982 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240264.98 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 24026.50 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979586.794208 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216238.48 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 21623.85 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001210.642611 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194614.64 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 19461.46 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020672.106173 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175153.17 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 17515.32 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038187.423379 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157637.85 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 15763.79 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053951.208865 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141874.07 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 14187.41 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068138.615802 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127686.66 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 12768.67 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080907.282046 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114918.00 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 11491.80 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092399.081665 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103426.20 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 10342.62 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102741.701322 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93083.58 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 9308.36 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2112050.059013 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83775.22 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 8377.52 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120427.580936 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75397.70 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 7539.77 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127967.350666 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67857.93 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 6785.79 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134753.143423 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61072.13 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 6107.21 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140860.356904 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54964.92 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 5496.49 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146356.849037 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49468.43 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 4946.84 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151303.691957 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44521.59 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 4452.16 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155755.850585 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40069.43 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 4006.94 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159762.793350 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36062.48 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3606.25 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163369.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32456.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166969.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28856.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170569.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25256.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174169.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21656.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177769.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18056.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181369.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14456.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184969.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10856.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188569.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7256.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192169.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3656.24 sec) 23:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195769.041839 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 56.24 sec) 23:40:00 [GPUGRID] [rr_sim] 2195825.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965677596.87G/439.78G) 23:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854066.19 23:40:00 [---] pick_jobs_to_run() start 23:40:00 [---] pick_jobs_to_run() end 23:40:00 [---] [rr_sim] end 23:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:40:00 [---] [cpu_sched_debug] preliminary job list: 23:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:40:00 [---] [cpu_sched_debug] final job list: 23:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:40:00 [---] [cpu_sched_debug] enforce_run_list: end 23:40:00 [---] choose_project(): 85200.000000 23:40:00 [WUProp@Home] doing RPC to get work 23:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350399.09 cpu 2100032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2100032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2100032.90 23:40:00 [---] Got 0 tasks 23:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:41:00 [---] [rr_sim] doing sim: CPU sched 23:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965650006.52G/439.78G) 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195762.54 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 219576.25 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219576.254135 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976186.29 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 197618.63 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417194.882857 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778567.66 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 177856.77 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595051.648707 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600710.89 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 160071.09 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755122.737972 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440639.80 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 144063.98 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899186.718310 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296575.82 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 129657.58 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028844.300614 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166918.24 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 116691.82 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145536.124688 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050226.42 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 105022.64 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250558.766355 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945203.77 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 94520.38 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345079.143855 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850683.40 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 85068.34 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430147.483605 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765615.06 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 76561.51 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506708.989380 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689053.55 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 68905.36 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575614.344577 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620148.20 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 62014.82 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637629.164255 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558133.38 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 55813.34 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693442.501965 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502320.04 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 50232.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743674.505904 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452088.04 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 45208.80 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788883.309449 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406879.23 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 40687.92 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829571.232639 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366191.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 36619.13 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866190.363511 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329572.18 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 32957.22 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899147.581295 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296614.96 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 29661.50 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928809.077301 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266953.46 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 26695.35 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955504.423706 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240258.12 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 24025.81 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979530.235471 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216232.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 21623.23 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001153.466059 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194609.08 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 19460.91 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020614.373589 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175148.17 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 17514.82 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038129.190365 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157633.35 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 15763.34 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053892.525464 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141870.02 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 14187.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068079.527053 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127683.01 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 12768.30 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080847.828483 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114914.71 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 11491.47 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092339.299770 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103423.24 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 10342.32 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102681.623929 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93080.92 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 9308.09 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111989.715671 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83772.83 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 8377.28 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120366.998240 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75395.54 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 7539.55 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127906.552551 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67855.99 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 6785.60 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134692.151431 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61070.39 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 6107.04 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140799.190424 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54963.35 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 5496.34 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146295.525517 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49467.02 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 4946.70 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151242.227100 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44520.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 4452.03 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155694.258526 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40068.28 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 4006.83 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159701.086809 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36061.45 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3606.15 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163307.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32455.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166907.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28855.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170507.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25255.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174107.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21655.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177707.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18055.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181307.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14455.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184907.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10855.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188507.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7255.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192107.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3655.31 sec) 23:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195707.232263 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 55.31 sec) 23:41:00 [GPUGRID] [rr_sim] 2195762.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965650006.52G/439.78G) 23:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854063.45 23:41:00 [---] pick_jobs_to_run() start 23:41:00 [---] pick_jobs_to_run() end 23:41:00 [---] [rr_sim] end 23:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:41:00 [---] [cpu_sched_debug] preliminary job list: 23:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:41:00 [---] [cpu_sched_debug] final job list: 23:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:41:00 [---] [cpu_sched_debug] enforce_run_list: end 23:41:00 [---] choose_project(): 85260.000000 23:41:00 [WUProp@Home] doing RPC to get work 23:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350339.09 cpu 2099972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099972.90 23:41:00 [---] Got 0 tasks 23:42:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:42:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:42:00 [---] [rr_sim] doing sim: CPU sched 23:42:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965622416.16G/439.78G) 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195699.80 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 219569.98 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219569.980447 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976129.82 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 197612.98 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417182.962850 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778516.84 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 177851.68 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595034.647012 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600665.16 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 160066.52 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755101.162758 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440598.64 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 144059.86 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899161.026929 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296538.78 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 129653.88 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028814.904684 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166884.90 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 116688.49 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145503.394662 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050196.41 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 105019.64 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250523.035643 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945176.77 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 94517.68 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345040.712526 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850659.09 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 85065.91 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430106.621721 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765593.18 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 76559.32 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506665.939996 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689033.86 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 68903.39 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575569.326444 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620130.48 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 62013.05 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637582.374246 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558117.43 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 55811.74 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693394.117269 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502305.69 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 50230.57 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743624.685989 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452075.12 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 45207.51 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788832.197838 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406867.61 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 40686.76 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829518.958501 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366180.85 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 36618.08 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866137.043098 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329562.76 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 32956.28 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899093.319235 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296606.49 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 29660.65 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928753.967759 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266945.84 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 26694.58 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955448.551430 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240251.25 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 24025.13 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979473.676735 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216226.13 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 21622.61 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001096.289508 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194603.51 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 19460.35 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020556.641005 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175143.16 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 17514.32 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038070.957351 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157628.85 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 15762.88 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053833.842063 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141865.96 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 14186.60 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2068020.438304 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127679.37 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 12767.94 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080788.374921 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114911.43 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 11491.14 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092279.517876 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103420.29 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 10342.03 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102621.546536 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93078.26 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 9307.83 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111929.372329 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83770.43 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 8377.04 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120306.415544 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75393.39 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 7539.34 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127845.754436 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67854.05 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 6785.41 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134631.159440 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61068.65 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 6106.86 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140738.023943 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54961.78 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 5496.18 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146234.201996 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49465.60 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 4946.56 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151180.762244 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44519.04 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 4451.90 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155632.666466 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40067.14 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 4006.71 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159639.380267 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36060.42 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3606.04 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163245.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32454.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166845.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28854.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170445.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25254.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174045.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21654.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177645.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18054.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181245.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14454.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184845.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10854.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188445.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7254.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192045.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3654.38 sec) 23:42:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195645.422688 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54.38 sec) 23:42:00 [GPUGRID] [rr_sim] 2195699.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965622416.16G/439.78G) 23:42:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854060.72 23:42:00 [---] pick_jobs_to_run() start 23:42:00 [---] pick_jobs_to_run() end 23:42:00 [---] [rr_sim] end 23:42:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:42:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:42:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:42:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:42:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:42:00 [---] [cpu_sched_debug] preliminary job list: 23:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:42:00 [---] [cpu_sched_debug] final job list: 23:42:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:42:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:42:00 [---] [cpu_sched_debug] enforce_run_list: end 23:42:00 [---] choose_project(): 85320.000000 23:42:00 [WUProp@Home] doing RPC to get work 23:42:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350279.09 cpu 2099912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099912.90 23:42:00 [---] Got 0 tasks 23:43:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:43:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:43:00 [---] [rr_sim] doing sim: CPU sched 23:43:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965594825.80G/439.78G) 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195637.07 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 219563.71 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219563.706759 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976073.36 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 197607.34 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417171.042842 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778466.02 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 177846.60 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595017.645317 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600619.42 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 160061.94 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755079.587544 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440557.48 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 144055.75 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899135.335549 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296501.73 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 129650.17 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028785.508753 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166851.56 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 116685.16 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145470.664637 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050166.40 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 105016.64 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250487.304932 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945149.76 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 94514.98 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1345002.281198 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850634.79 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 85063.48 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430065.759837 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765571.31 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 76557.13 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506622.890612 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 689014.18 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 68901.42 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575524.308310 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620112.76 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 62011.28 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637535.584238 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558101.48 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 55810.15 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693345.732573 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502291.34 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 50229.13 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743574.866075 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452062.20 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 45206.22 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788781.086226 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406855.98 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 40685.60 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829466.684363 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366170.38 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 36617.04 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866083.722685 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329553.34 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 32955.33 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1899039.057176 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296598.01 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 29659.80 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928698.858217 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266938.21 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 26693.82 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955392.679154 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240244.39 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 24024.44 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979417.117998 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216219.95 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 21621.99 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2001039.112957 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194597.95 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 19459.80 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020498.908420 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175138.16 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 17513.82 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2038012.724337 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157624.34 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 15762.43 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053775.158663 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141861.91 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 14186.19 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067961.349555 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127675.72 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 12767.57 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080728.921359 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114908.15 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 11490.81 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092219.735982 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103417.33 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 10341.73 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102561.469143 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93075.60 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 9307.56 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111869.028987 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83768.04 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 8376.80 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120245.832848 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75391.23 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 7539.12 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127784.956322 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67852.11 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 6785.21 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134570.167449 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61066.90 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 6106.69 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140676.857463 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54960.21 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 5496.02 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146172.878475 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49464.19 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 4946.42 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151119.297387 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44517.77 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 4451.78 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155571.074407 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40065.99 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 4006.60 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159577.673725 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36059.39 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3605.94 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163183.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32453.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166783.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28853.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170383.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25253.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173983.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21653.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177583.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18053.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181183.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14453.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184783.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10853.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188383.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7253.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191983.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3653.45 sec) 23:43:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195583.613112 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 53.45 sec) 23:43:00 [GPUGRID] [rr_sim] 2195637.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965594825.80G/439.78G) 23:43:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854057.98 23:43:00 [---] pick_jobs_to_run() start 23:43:00 [---] pick_jobs_to_run() end 23:43:00 [---] [rr_sim] end 23:43:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:43:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:43:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:43:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:43:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:43:00 [---] [cpu_sched_debug] preliminary job list: 23:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:43:00 [---] [cpu_sched_debug] final job list: 23:43:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:43:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:43:00 [---] [cpu_sched_debug] enforce_run_list: end 23:43:00 [---] choose_project(): 85380.000000 23:43:00 [WUProp@Home] doing RPC to get work 23:43:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350219.09 cpu 2099852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099852.90 23:43:00 [---] Got 0 tasks 23:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:44:00 [---] [rr_sim] doing sim: CPU sched 23:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965567235.45G/439.78G) 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195574.33 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 219557.43 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219557.433071 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1976016.90 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 197601.69 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417159.122834 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778415.21 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 177841.52 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 595000.643622 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600573.69 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 160057.37 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755058.012330 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440516.32 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 144051.63 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899109.644168 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296464.69 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 129646.47 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028756.112822 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166818.22 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 116681.82 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145437.934611 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050136.40 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 105013.64 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250451.574220 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945122.76 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 94512.28 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344963.849869 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850610.48 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 85061.05 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1430024.897953 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765549.43 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 76554.94 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506579.841228 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688994.49 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 68899.45 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575479.290176 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620095.04 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 62009.50 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637488.794229 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558085.54 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 55808.55 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693297.347877 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502276.98 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 50227.70 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743525.046160 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452049.28 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 45204.93 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788729.974615 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406844.36 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 40684.44 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829414.410224 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366159.92 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 36615.99 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1866030.402273 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329543.93 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 32954.39 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898984.795116 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296589.54 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 29658.95 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928643.748675 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266930.58 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 26693.06 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955336.806879 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240237.52 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 24023.75 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979360.559261 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216213.77 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 21621.38 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000981.936406 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194592.39 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 19459.24 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020441.175836 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175133.15 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 17513.32 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037954.491323 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157619.84 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 15761.98 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053716.475262 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141857.86 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 14185.79 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067902.260806 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127672.07 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 12767.21 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080669.467796 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114904.86 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 11490.49 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092159.954088 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103414.38 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 10341.44 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102501.391750 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93072.94 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 9307.29 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111808.685645 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83765.65 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 8376.56 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120185.250152 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75389.08 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 7538.91 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127724.158207 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67850.17 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 6785.02 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134509.175457 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61065.16 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 6106.52 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140615.690982 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54958.64 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 5495.86 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146111.554955 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49462.78 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 4946.28 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151057.832530 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44516.50 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 4451.65 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155509.482348 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40064.85 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 4006.48 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159515.967184 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36058.36 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3605.84 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163121.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32452.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166721.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28852.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170321.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25252.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173921.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21652.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177521.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18052.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181121.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14452.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184721.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10852.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188321.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7252.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191921.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3652.53 sec) 23:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195521.803536 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 52.53 sec) 23:44:00 [GPUGRID] [rr_sim] 2195574.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965567235.45G/439.78G) 23:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854055.24 23:44:00 [---] pick_jobs_to_run() start 23:44:00 [---] pick_jobs_to_run() end 23:44:00 [---] [rr_sim] end 23:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:44:00 [---] [cpu_sched_debug] preliminary job list: 23:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:44:00 [---] [cpu_sched_debug] final job list: 23:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:44:00 [---] [cpu_sched_debug] enforce_run_list: end 23:44:00 [---] choose_project(): 85440.000000 23:44:00 [WUProp@Home] doing RPC to get work 23:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350159.09 cpu 2099792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099792.90 23:44:00 [---] Got 0 tasks 23:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:45:00 [---] [rr_sim] doing sim: CPU sched 23:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965539645.09G/439.78G) 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195511.59 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 219551.16 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219551.159383 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975960.43 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 197596.04 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417147.202827 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778364.39 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 177836.44 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594983.641927 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600527.95 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 160052.80 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755036.437117 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440475.16 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 144047.52 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899083.952787 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296427.64 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 129642.76 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028726.716891 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166784.88 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 116678.49 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145405.204585 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050106.39 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 105010.64 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250415.843509 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945095.75 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 94509.58 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344925.418540 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850586.18 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 85058.62 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429984.036069 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765527.56 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 76552.76 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506536.791845 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688974.80 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 68897.48 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575434.272043 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620077.32 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 62007.73 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637442.004221 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558069.59 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 55806.96 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693248.963181 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502262.63 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 50226.26 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743475.226246 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452036.37 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 45203.64 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788678.863004 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406832.73 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 40683.27 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829362.136086 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366149.46 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 36614.95 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865977.081860 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329534.51 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 32953.45 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898930.533056 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296581.06 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 29658.11 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928588.639133 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266922.95 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 26692.30 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955280.934603 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240230.66 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 24023.07 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979304.000525 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216207.59 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 21620.76 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000924.759855 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194586.83 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 19458.68 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020383.443252 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175128.15 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 17512.82 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037896.258309 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157615.34 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 15761.53 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053657.791861 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141853.80 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 14185.38 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067843.172057 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127668.42 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 12766.84 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080610.014234 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114901.58 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 11490.16 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092100.172193 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103411.42 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 10341.14 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102441.314357 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93070.28 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 9307.03 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111748.342303 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83763.25 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 8376.33 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120124.667456 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75386.93 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 7538.69 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127663.360093 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67848.23 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 6784.82 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134448.183466 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61063.41 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 6106.34 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140554.524502 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54957.07 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 5495.71 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146050.231434 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49461.36 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 4946.14 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150996.367673 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44515.23 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 4451.52 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155447.890289 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40063.70 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 4006.37 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159454.260642 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36057.33 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3605.73 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163059.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32451.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166659.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28851.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170259.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25251.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173859.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21651.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177459.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18051.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181059.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14451.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184659.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10851.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188259.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7251.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191859.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3651.60 sec) 23:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195459.993961 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 51.60 sec) 23:45:00 [GPUGRID] [rr_sim] 2195511.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965539645.09G/439.78G) 23:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854052.51 23:45:00 [---] pick_jobs_to_run() start 23:45:00 [---] pick_jobs_to_run() end 23:45:00 [---] [rr_sim] end 23:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:45:00 [---] [cpu_sched_debug] preliminary job list: 23:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:45:00 [---] [cpu_sched_debug] final job list: 23:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:45:00 [---] [cpu_sched_debug] enforce_run_list: end 23:45:00 [---] choose_project(): 85500.000000 23:45:00 [WUProp@Home] doing RPC to get work 23:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350099.09 cpu 2099732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099732.90 23:45:00 [---] Got 0 tasks 23:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:46:00 [---] [rr_sim] doing sim: CPU sched 23:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965512054.74G/439.78G) 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195448.86 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 219544.89 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219544.885694 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975903.97 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 197590.40 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417135.282819 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778313.57 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 177831.36 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594966.640232 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600482.22 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 160048.22 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 755014.861903 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440434.00 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 144043.40 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899058.261407 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296390.60 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 129639.06 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028697.320960 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166751.54 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 116675.15 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145372.474559 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050076.38 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 105007.64 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250380.112797 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945068.74 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 94506.87 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344886.987212 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850561.87 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 85056.19 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429943.174185 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765505.68 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 76550.57 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506493.742461 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688955.11 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 68895.51 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575389.253909 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620059.60 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 62005.96 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637395.214212 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558053.64 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 55805.36 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693200.578486 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502248.28 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 50224.83 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743425.406331 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452023.45 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 45202.35 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788627.751393 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406821.11 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 40682.11 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829309.861948 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366138.99 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 36613.90 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865923.761447 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329525.10 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 32952.51 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898876.270997 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296572.59 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 29657.26 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928533.529591 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266915.33 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 26691.53 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955225.062327 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240223.79 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 24022.38 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979247.441788 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216201.42 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 21620.14 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000867.583304 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194581.27 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 19458.13 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020325.710668 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175123.15 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 17512.31 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037838.025295 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157610.83 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 15761.08 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053599.108460 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141849.75 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 14184.97 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067784.083308 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127664.77 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 12766.48 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080550.560672 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114898.30 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 11489.83 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2092040.390299 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103408.47 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 10340.85 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102381.236963 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93067.62 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 9306.76 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111687.998961 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83760.86 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 8376.09 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120064.084760 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75384.77 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 7538.48 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127602.561978 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67846.29 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 6784.63 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134387.191475 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61061.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 6106.17 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140493.358021 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54955.50 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 5495.55 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145988.907914 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49459.95 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 4945.99 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150934.902817 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44513.95 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 4451.40 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155386.298229 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40062.56 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 4006.26 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159392.554101 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36056.30 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3605.63 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162998.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32450.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166598.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28850.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170198.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25250.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173798.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21650.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177398.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18050.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180998.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14450.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184598.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10850.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188198.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7250.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191798.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3650.67 sec) 23:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195398.184385 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 50.67 sec) 23:46:00 [GPUGRID] [rr_sim] 2195448.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965512054.74G/439.78G) 23:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854049.77 23:46:00 [---] pick_jobs_to_run() start 23:46:00 [---] pick_jobs_to_run() end 23:46:00 [---] [rr_sim] end 23:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:46:00 [---] [cpu_sched_debug] preliminary job list: 23:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:46:00 [---] [cpu_sched_debug] final job list: 23:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:46:00 [---] [cpu_sched_debug] enforce_run_list: end 23:46:00 [---] choose_project(): 85560.000000 23:46:00 [WUProp@Home] doing RPC to get work 23:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 350039.09 cpu 2099672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099672.90 23:46:00 [---] Got 0 tasks 23:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:47:00 [---] [rr_sim] doing sim: CPU sched 23:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965484464.38G/439.78G) 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195386.12 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 219538.61 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219538.612006 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975847.51 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 197584.75 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417123.362812 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778262.76 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 177826.28 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594949.638537 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600436.48 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 160043.65 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754993.286689 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440392.83 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 144039.28 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899032.570026 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296353.55 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 129635.36 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028667.925030 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166718.20 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 116671.82 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145339.744533 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050046.38 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 105004.64 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250344.382086 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945041.74 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 94504.17 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344848.555883 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850537.56 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 85053.76 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429902.312301 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765483.81 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 76548.38 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506450.693077 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688935.43 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 68893.54 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575344.235775 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620041.88 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 62004.19 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637348.424204 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558037.70 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 55803.77 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693152.193790 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502233.93 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 50223.39 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743375.586417 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 452010.53 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 45201.05 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788576.639781 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406809.48 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 40680.95 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829257.587809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366128.53 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 36612.85 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865870.441034 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329515.68 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 32951.57 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898822.008937 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296564.11 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 29656.41 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928478.420050 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266907.70 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 26690.77 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955169.190051 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240216.93 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 24021.69 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979190.883052 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216195.24 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 21619.52 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000810.406753 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194575.71 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 19457.57 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020267.978084 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175118.14 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 17511.81 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037779.792281 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157606.33 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 15760.63 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053540.425059 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141845.70 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 14184.57 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067724.994559 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127661.13 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 12766.11 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080491.107110 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114895.01 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 11489.50 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091980.608405 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103405.51 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 10340.55 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102321.159570 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93064.96 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 9306.50 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111627.655619 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83758.46 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 8375.85 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2120003.502064 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75382.62 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 7538.26 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127541.763863 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67844.36 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 6784.44 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134326.199483 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61059.92 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 6105.99 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140432.191541 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54953.93 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 5495.39 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145927.584393 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49458.54 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 4945.85 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150873.437960 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44512.68 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 4451.27 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155324.706170 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40061.41 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 4006.14 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159330.847559 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36055.27 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3605.53 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162936.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32449.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166536.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28849.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170136.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25249.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173736.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21649.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177336.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18049.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180936.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14449.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184536.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10849.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188136.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7249.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191736.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3649.75 sec) 23:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195336.374809 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49.75 sec) 23:47:00 [GPUGRID] [rr_sim] 2195386.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965484464.38G/439.78G) 23:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854047.03 23:47:00 [---] pick_jobs_to_run() start 23:47:00 [---] pick_jobs_to_run() end 23:47:00 [---] [rr_sim] end 23:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:47:00 [---] [cpu_sched_debug] preliminary job list: 23:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:47:00 [---] [cpu_sched_debug] final job list: 23:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:47:00 [---] [cpu_sched_debug] enforce_run_list: end 23:47:00 [---] choose_project(): 85620.000000 23:47:00 [WUProp@Home] doing RPC to get work 23:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349979.09 cpu 2099612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099612.90 23:47:00 [---] Got 0 tasks 23:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:48:00 [---] [rr_sim] doing sim: CPU sched 23:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965456874.02G/439.78G) 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195323.38 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 219532.34 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219532.338318 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975791.04 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 197579.10 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417111.442804 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778211.94 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 177821.19 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594932.636842 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600390.75 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 160039.07 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754971.711475 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440351.67 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 144035.17 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 899006.878646 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296316.50 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 129631.65 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028638.529099 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166684.85 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 116668.49 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145307.014507 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1050016.37 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 105001.64 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250308.651374 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 945014.73 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 94501.47 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344810.124555 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850513.26 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 85051.33 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429861.450417 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765461.93 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 76546.19 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506407.643693 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688915.74 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 68891.57 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575299.217642 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620024.17 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 62002.42 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637301.634196 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558021.75 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 55802.17 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693103.809094 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502219.57 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 50221.96 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743325.766502 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451997.62 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 45199.76 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788525.528170 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406797.86 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 40679.79 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829205.313671 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366118.07 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 36611.81 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865817.120622 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329506.26 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 32950.63 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898767.746877 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296555.64 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 29655.56 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928423.310508 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266900.07 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 26690.01 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955113.317775 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240210.07 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 24021.01 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979134.324315 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216189.06 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 21618.91 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000753.230202 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194570.15 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 19457.02 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020210.245499 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175113.14 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 17511.31 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037721.559267 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157601.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 15760.18 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053481.741658 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141841.64 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 14184.16 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067665.905810 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127657.48 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 12765.75 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080431.653547 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114891.73 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 11489.17 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091920.826511 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103402.56 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 10340.26 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102261.082177 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93062.30 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 9306.23 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111567.312278 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83756.07 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 8375.61 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119942.919368 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75380.46 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 7538.05 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127480.965749 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67842.42 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 6784.24 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134265.207492 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61058.18 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 6105.82 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140371.025061 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54952.36 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 5495.24 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145866.260872 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49457.12 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 4945.71 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150811.973103 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44511.41 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 4451.14 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155263.114111 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40060.27 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 4006.03 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159269.141017 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36054.24 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3605.42 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162874.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32448.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166474.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28848.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170074.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25248.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173674.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21648.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177274.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18048.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180874.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14448.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184474.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10848.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188074.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7248.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191674.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3648.82 sec) 23:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195274.565234 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 48.82 sec) 23:48:00 [GPUGRID] [rr_sim] 2195323.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965456874.02G/439.78G) 23:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854044.30 23:48:00 [---] pick_jobs_to_run() start 23:48:00 [---] pick_jobs_to_run() end 23:48:00 [---] [rr_sim] end 23:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:48:00 [---] [cpu_sched_debug] preliminary job list: 23:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:48:00 [---] [cpu_sched_debug] final job list: 23:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:48:00 [---] [cpu_sched_debug] enforce_run_list: end 23:48:00 [---] choose_project(): 85680.000000 23:48:00 [WUProp@Home] doing RPC to get work 23:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349919.09 cpu 2099552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099552.90 23:48:00 [---] Got 0 tasks 23:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:49:00 [---] [rr_sim] doing sim: CPU sched 23:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965429283.67G/439.78G) 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195260.65 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 219526.06 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219526.064630 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975734.58 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 197573.46 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417099.522796 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778161.12 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 177816.11 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594915.635146 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600345.01 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 160034.50 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754950.136261 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440310.51 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 144031.05 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898981.187265 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296279.46 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 129627.95 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028609.133168 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166651.51 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 116665.15 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145274.284481 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049986.36 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 104998.64 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250272.920663 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944987.73 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 94498.77 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344771.693226 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850488.95 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 85048.90 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429820.588533 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765440.06 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 76544.01 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506364.594309 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688896.05 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 68889.61 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575254.199508 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 620006.45 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 62000.64 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637254.844187 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 558005.80 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 55800.58 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693055.424398 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502205.22 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 50220.52 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743275.946588 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451984.70 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 45198.47 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788474.416559 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406786.23 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 40678.62 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829153.039533 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366107.61 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 36610.76 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865763.800209 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329496.85 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 32949.68 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898713.484818 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296547.16 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 29654.72 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928368.200966 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266892.45 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 26689.24 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955057.445499 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240203.20 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 24020.32 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979077.765579 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216182.88 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 21618.29 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000696.053650 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194564.59 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 19456.46 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020152.512915 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175108.13 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 17510.81 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037663.326253 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157597.32 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 15759.73 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053423.058258 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141837.59 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 14183.76 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067606.817062 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127653.83 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 12765.38 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080372.199985 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114888.45 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 11488.84 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091861.044616 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103399.60 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 10339.96 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102201.004784 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93059.64 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 9305.96 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111506.968936 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83753.68 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 8375.37 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119882.336672 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75378.31 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 7537.83 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127420.167634 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67840.48 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 6784.05 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134204.215500 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61056.43 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 6105.64 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140309.858580 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54950.79 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 5495.08 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145804.937352 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49455.71 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 4945.57 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150750.508246 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44510.14 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 4451.01 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155201.522051 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40059.12 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 4005.91 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159207.434476 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36053.21 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3605.32 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162812.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32447.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166412.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28847.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170012.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25247.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173612.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21647.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177212.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18047.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180812.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14447.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184412.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10847.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188012.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7247.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191612.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3647.89 sec) 23:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195212.755658 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 47.89 sec) 23:49:00 [GPUGRID] [rr_sim] 2195260.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965429283.67G/439.78G) 23:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854041.56 23:49:00 [---] pick_jobs_to_run() start 23:49:00 [---] pick_jobs_to_run() end 23:49:00 [---] [rr_sim] end 23:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:49:00 [---] [cpu_sched_debug] preliminary job list: 23:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:49:00 [---] [cpu_sched_debug] final job list: 23:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:49:00 [---] [cpu_sched_debug] enforce_run_list: end 23:49:00 [---] choose_project(): 85740.000000 23:49:00 [WUProp@Home] doing RPC to get work 23:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349859.09 cpu 2099492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099492.90 23:49:00 [---] Got 0 tasks 23:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:50:00 [---] [rr_sim] doing sim: CPU sched 23:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965401693.31G/439.78G) 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195197.91 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 219519.79 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219519.790941 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975678.12 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 197567.81 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417087.602789 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778110.31 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 177811.03 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594898.633451 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600299.28 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 160029.93 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754928.561048 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440269.35 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 144026.93 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898955.495884 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296242.41 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 129624.24 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028579.737237 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166618.17 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 116661.82 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145241.554455 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049956.35 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 104995.64 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250237.189951 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944960.72 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 94496.07 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344733.261897 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850464.65 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 85046.46 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429779.726649 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765418.18 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 76541.82 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506321.544926 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688876.36 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 68887.64 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575209.181375 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619988.73 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 61998.87 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637208.054179 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557989.86 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 55798.99 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1693007.039702 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502190.87 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 50219.09 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743226.126673 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451971.78 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 45197.18 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788423.304948 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406774.60 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 40677.46 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829100.765394 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366097.14 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 36609.71 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865710.479796 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329487.43 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 32948.74 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898659.222758 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296538.69 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 29653.87 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928313.091424 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266884.82 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 26688.48 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1955001.573223 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240196.34 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 24019.63 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1979021.206842 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216176.70 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 21617.67 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000638.877099 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194559.03 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 19455.90 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020094.780331 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175103.13 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 17510.31 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037605.093239 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157592.82 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 15759.28 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053364.374857 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141833.53 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 14183.35 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067547.728313 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127650.18 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 12765.02 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080312.746423 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114885.16 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 11488.52 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091801.262722 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103396.65 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 10339.66 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102140.927391 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93056.98 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 9305.70 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111446.625594 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83751.28 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 8375.13 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119821.753976 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75376.16 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 7537.62 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127359.369520 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67838.54 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 6783.85 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134143.223509 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61054.69 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 6105.47 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140248.692100 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54949.22 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 5494.92 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145743.613831 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49454.30 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 4945.43 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150689.043389 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44508.87 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 4450.89 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155139.929992 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40057.98 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 4005.80 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159145.727934 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36052.18 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3605.22 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162750.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32446.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166350.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28846.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169950.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25246.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173550.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21646.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177150.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18046.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180750.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14446.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184350.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10846.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187950.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7246.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191550.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3646.96 sec) 23:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195150.946082 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46.96 sec) 23:50:00 [GPUGRID] [rr_sim] 2195197.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965401693.31G/439.78G) 23:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854038.82 23:50:00 [---] pick_jobs_to_run() start 23:50:00 [---] pick_jobs_to_run() end 23:50:00 [---] [rr_sim] end 23:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:50:00 [---] [cpu_sched_debug] preliminary job list: 23:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:50:00 [---] [cpu_sched_debug] final job list: 23:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:50:00 [---] [cpu_sched_debug] enforce_run_list: end 23:50:00 [---] choose_project(): 85800.000000 23:50:00 [WUProp@Home] doing RPC to get work 23:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349799.09 cpu 2099432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099432.90 23:50:00 [---] Got 0 tasks 23:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:51:00 [---] [rr_sim] doing sim: CPU sched 23:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965374102.95G/439.78G) 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195135.17 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 219513.52 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219513.517253 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975621.66 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 197562.17 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417075.682781 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778059.49 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 177805.95 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594881.631756 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600253.54 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 160025.35 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754906.985834 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440228.19 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 144022.82 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898929.804504 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296205.37 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 129620.54 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028550.341307 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166584.83 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 116658.48 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145208.824429 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049926.35 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 104992.63 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250201.459240 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944933.71 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 94493.37 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344694.830569 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850440.34 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 85044.03 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429738.864765 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765396.31 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 76539.63 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506278.495542 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688856.68 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 68885.67 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575164.163241 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619971.01 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 61997.10 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637161.264170 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557973.91 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 55797.39 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692958.655006 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502176.52 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 50217.65 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743176.306759 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451958.87 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 45195.89 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788372.193336 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406762.98 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 40676.30 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1829048.491256 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366086.68 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 36608.67 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865657.159384 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329478.01 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 32947.80 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898604.960698 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296530.21 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 29653.02 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928257.981882 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266877.19 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 26687.72 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954945.700947 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240189.47 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 24018.95 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978964.648106 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216170.52 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 21617.05 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000581.700548 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194553.47 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 19455.35 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2020037.047747 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175098.12 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 17509.81 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037546.860225 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157588.31 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 15758.83 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053305.691456 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141829.48 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 14182.95 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067488.639564 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127646.53 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 12764.65 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080253.292860 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114881.88 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 11488.19 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091741.480828 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103393.69 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 10339.37 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102080.849998 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93054.32 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 9305.43 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111386.282252 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83748.89 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 8374.89 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119761.171280 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75374.00 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 7537.40 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127298.571405 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67836.60 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 6783.66 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134082.231518 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61052.94 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 6105.29 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140187.525619 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54947.65 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 5494.76 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145682.290311 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49452.88 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 4945.29 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150627.578533 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44507.59 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 4450.76 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155078.337933 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40056.83 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 4005.68 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159084.021393 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36051.15 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3605.12 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162689.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32446.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166289.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28846.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169889.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25246.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173489.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21646.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177089.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18046.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180689.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14446.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184289.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10846.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187889.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7246.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191489.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3646.04 sec) 23:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195089.136507 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 46.04 sec) 23:51:00 [GPUGRID] [rr_sim] 2195135.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965374102.95G/439.78G) 23:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854036.09 23:51:00 [---] pick_jobs_to_run() start 23:51:00 [---] pick_jobs_to_run() end 23:51:00 [---] [rr_sim] end 23:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:51:00 [---] [cpu_sched_debug] preliminary job list: 23:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:51:00 [---] [cpu_sched_debug] final job list: 23:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:51:00 [---] [cpu_sched_debug] enforce_run_list: end 23:51:00 [---] choose_project(): 85860.000000 23:51:00 [WUProp@Home] doing RPC to get work 23:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349739.09 cpu 2099372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099372.90 23:51:00 [---] Got 0 tasks 23:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:52:00 [---] [rr_sim] doing sim: CPU sched 23:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965346512.60G/439.78G) 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195072.44 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 219507.24 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219507.243565 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975565.19 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 197556.52 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417063.762774 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1778008.67 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 177800.87 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594864.630061 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600207.81 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 160020.78 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754885.410620 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440187.03 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 144018.70 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898904.113123 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296168.32 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 129616.83 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028520.945376 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166551.49 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 116655.15 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145176.094403 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049896.34 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 104989.63 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250165.728528 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944906.71 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 94490.67 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344656.399240 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850416.04 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 85041.60 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429698.002881 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765374.43 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 76537.44 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506235.446158 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688836.99 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 68883.70 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575119.145107 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619953.29 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 61995.33 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637114.474162 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557957.96 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 55795.80 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692910.270311 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502162.17 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 50216.22 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743126.486845 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451945.95 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 45194.59 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788321.081725 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406751.35 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 40675.14 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828996.217118 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366076.22 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 36607.62 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865603.838971 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329468.60 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 32946.86 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898550.698639 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296521.74 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 29652.17 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928202.872340 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266869.56 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 26686.96 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954889.828671 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240182.61 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 24018.26 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978908.089369 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216164.35 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 21616.43 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000524.523997 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194547.91 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 19454.79 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019979.315162 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175093.12 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 17509.31 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037488.627211 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157583.81 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 15758.38 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053247.008055 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141825.43 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 14182.54 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067429.550815 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127642.88 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 12764.29 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080193.839298 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114878.60 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 11487.86 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091681.698933 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103390.74 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 10339.07 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2102020.772605 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93051.66 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 9305.17 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111325.938910 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83746.50 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 8374.65 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119700.588584 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75371.85 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 7537.18 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127237.773290 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67834.66 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 6783.47 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2134021.239526 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61051.20 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 6105.12 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140126.359139 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54946.08 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 5494.61 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145620.966790 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49451.47 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 4945.15 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150566.113676 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44506.32 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 4450.63 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155016.745873 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40055.69 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 4005.57 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159022.314851 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36050.12 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3605.01 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162627.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32445.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166227.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28845.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169827.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25245.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173427.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21645.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177027.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18045.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180627.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14445.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184227.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10845.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187827.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7245.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191427.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3645.11 sec) 23:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2195027.326931 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 45.11 sec) 23:52:00 [GPUGRID] [rr_sim] 2195072.44: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965346512.60G/439.78G) 23:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854033.35 23:52:00 [---] pick_jobs_to_run() start 23:52:00 [---] pick_jobs_to_run() end 23:52:00 [---] [rr_sim] end 23:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:52:00 [---] [cpu_sched_debug] preliminary job list: 23:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:52:00 [---] [cpu_sched_debug] final job list: 23:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:52:00 [---] [cpu_sched_debug] enforce_run_list: end 23:52:00 [---] choose_project(): 85920.000000 23:52:00 [WUProp@Home] doing RPC to get work 23:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349679.09 cpu 2099312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099312.90 23:52:00 [---] Got 0 tasks 23:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:53:00 [---] [rr_sim] doing sim: CPU sched 23:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965318922.24G/439.78G) 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2195009.70 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 219500.97 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219500.969877 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975508.73 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 197550.87 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417051.842766 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777957.86 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 177795.79 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594847.628366 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600162.07 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 160016.21 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754863.835406 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440145.86 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 144014.59 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898878.421743 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296131.28 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 129613.13 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028491.549445 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166518.15 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 116651.81 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145143.364377 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049866.33 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 104986.63 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250129.997816 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944879.70 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 94487.97 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344617.967912 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850391.73 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 85039.17 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429657.140997 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765352.56 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 76535.26 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506192.396774 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688817.30 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 68881.73 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575074.126974 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619935.57 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 61993.56 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637067.684153 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557942.01 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 55794.20 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692861.885615 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502147.81 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 50214.78 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743076.666930 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451933.03 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 45193.30 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788269.970114 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406739.73 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 40673.97 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828943.942979 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366065.76 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 36606.58 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865550.518558 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329459.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 32945.92 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898496.436579 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296513.26 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 29651.33 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928147.762798 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266861.94 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 26686.19 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954833.956395 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240175.74 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 24017.57 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978851.530632 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216158.17 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 21615.82 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000467.347446 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194542.35 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 19454.24 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019921.582578 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175088.12 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 17508.81 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037430.394197 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157579.30 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 15757.93 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053188.324654 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141821.37 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 14182.14 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067370.462066 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127639.24 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 12763.92 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080134.385736 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114875.31 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 11487.53 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091621.917039 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103387.78 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 10338.78 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101960.695212 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93049.00 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 9304.90 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111265.595568 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83744.10 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 8374.41 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119640.005888 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75369.69 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 7536.97 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127176.975176 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67832.72 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 6783.27 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133960.247535 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61049.45 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 6104.95 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140065.192658 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54944.51 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 5494.45 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145559.643269 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49450.06 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 4945.01 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150504.648819 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44505.05 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 4450.50 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154955.153814 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40054.54 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 4005.45 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158960.608309 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36049.09 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3604.91 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162565.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32444.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166165.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28844.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169765.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25244.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173365.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21644.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176965.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18044.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180565.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14444.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184165.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10844.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187765.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7244.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191365.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3644.18 sec) 23:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194965.517355 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44.18 sec) 23:53:00 [GPUGRID] [rr_sim] 2195009.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965318922.24G/439.78G) 23:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854030.61 23:53:00 [---] pick_jobs_to_run() start 23:53:00 [---] pick_jobs_to_run() end 23:53:00 [---] [rr_sim] end 23:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:53:00 [---] [cpu_sched_debug] preliminary job list: 23:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:53:00 [---] [cpu_sched_debug] final job list: 23:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:53:00 [---] [cpu_sched_debug] enforce_run_list: end 23:53:00 [---] choose_project(): 85980.000000 23:53:00 [WUProp@Home] doing RPC to get work 23:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349619.09 cpu 2099252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099252.90 23:53:00 [---] Got 0 tasks 23:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:54:00 [---] [rr_sim] doing sim: CPU sched 23:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965291331.89G/439.78G) 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194946.96 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 219494.70 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219494.696189 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975452.27 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 197545.23 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417039.922758 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777907.04 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 177790.70 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594830.626671 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600116.34 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 160011.63 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754842.260193 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440104.70 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 144010.47 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898852.730362 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296094.23 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 129609.42 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028462.153514 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166484.81 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 116648.48 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145110.634352 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049836.33 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 104983.63 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250094.267105 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944852.69 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 94485.27 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344579.536583 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850367.43 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 85036.74 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429616.279113 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765330.68 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 76533.07 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506149.347391 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688797.61 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 68879.76 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1575029.108840 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619917.85 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 61991.79 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1637020.894145 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557926.07 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 55792.61 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692813.500919 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502133.46 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 50213.35 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1743026.847016 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451920.11 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 45192.01 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788218.858503 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406728.10 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 40672.81 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828891.668841 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366055.29 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 36605.53 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865497.198145 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329449.76 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 32944.98 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898442.174520 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296504.79 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 29650.48 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928092.653256 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266854.31 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 26685.43 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954778.084119 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240168.88 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 24016.89 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978794.971896 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216151.99 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 21615.20 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000410.170895 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194536.79 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 19453.68 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019863.849994 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175083.11 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 17508.31 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037372.161183 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157574.80 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 15757.48 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053129.641253 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141817.32 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 14181.73 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067311.373317 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127635.59 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 12763.56 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080074.932174 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114872.03 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 11487.20 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091562.135145 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103384.83 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 10338.48 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101900.617819 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93046.34 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 9304.63 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111205.252226 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83741.71 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 8374.17 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119579.423192 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75367.54 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 7536.75 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127116.177061 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67830.78 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 6783.08 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133899.255544 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61047.71 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 6104.77 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2140004.026178 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54942.94 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 5494.29 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145498.319749 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49448.64 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 4944.86 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150443.183962 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44503.78 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 4450.38 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154893.561755 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40053.40 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 4005.34 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158898.901768 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36048.06 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3604.81 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162503.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32443.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166103.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28843.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169703.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25243.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173303.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21643.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176903.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18043.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180503.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14443.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184103.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10843.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187703.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7243.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191303.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3643.25 sec) 23:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194903.707780 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 43.25 sec) 23:54:00 [GPUGRID] [rr_sim] 2194946.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965291331.89G/439.78G) 23:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854027.87 23:54:00 [---] pick_jobs_to_run() start 23:54:00 [---] pick_jobs_to_run() end 23:54:00 [---] [rr_sim] end 23:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:54:00 [---] [cpu_sched_debug] preliminary job list: 23:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:54:00 [---] [cpu_sched_debug] final job list: 23:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:54:00 [---] [cpu_sched_debug] enforce_run_list: end 23:54:00 [---] choose_project(): 86040.000000 23:54:00 [WUProp@Home] doing RPC to get work 23:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349559.09 cpu 2099192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099192.90 23:54:00 [---] Got 0 tasks 23:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:55:00 [---] [rr_sim] doing sim: CPU sched 23:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965263741.53G/439.78G) 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194884.23 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 219488.42 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219488.422500 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975395.80 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 197539.58 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417028.002751 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777856.22 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 177785.62 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594813.624976 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600070.60 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 160007.06 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754820.684979 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440063.54 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 144006.35 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898827.038981 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296057.19 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 129605.72 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028432.757584 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166451.47 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 116645.15 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145077.904326 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049806.32 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 104980.63 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250058.536393 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944825.69 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 94482.57 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344541.105254 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850343.12 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 85034.31 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429575.417229 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765308.81 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 76530.88 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506106.298007 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688777.93 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 68877.79 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574984.090707 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619900.13 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 61990.01 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636974.104136 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557910.12 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 55791.01 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692765.116223 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502119.11 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 50211.91 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742977.027101 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451907.20 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 45190.72 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788167.746891 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406716.48 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 40671.65 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828839.394703 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366044.83 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 36604.48 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865443.877733 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329440.35 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 32944.03 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898387.912460 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296496.31 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 29649.63 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1928037.543714 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266846.68 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 26684.67 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954722.211843 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240162.01 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 24016.20 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978738.413159 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216145.81 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 21614.58 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000352.994344 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194531.23 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 19453.12 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019806.117410 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175078.11 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 17507.81 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037313.928169 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157570.30 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 15757.03 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053070.957853 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141813.27 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 14181.33 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067252.284568 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127631.94 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 12763.19 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2080015.478611 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114868.75 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 11486.87 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091502.353251 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103381.87 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 10338.19 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101840.540426 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93043.68 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 9304.37 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111144.908884 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83739.32 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 8373.93 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119518.840496 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75365.38 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 7536.54 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2127055.378947 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67828.85 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 6782.88 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133838.263552 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61045.96 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 6104.60 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139942.859697 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54941.37 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 5494.14 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145436.996228 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49447.23 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 4944.72 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150381.719106 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44502.51 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 4450.25 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154831.969695 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40052.26 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 4005.23 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158837.195226 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36047.03 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3604.70 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162441.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32442.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166041.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28842.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169641.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25242.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173241.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21642.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176841.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18042.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180441.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14442.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184041.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10842.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187641.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7242.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191241.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3642.33 sec) 23:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194841.898204 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 42.33 sec) 23:55:00 [GPUGRID] [rr_sim] 2194884.23: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965263741.53G/439.78G) 23:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854025.14 23:55:00 [---] pick_jobs_to_run() start 23:55:00 [---] pick_jobs_to_run() end 23:55:00 [---] [rr_sim] end 23:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:55:00 [---] [cpu_sched_debug] preliminary job list: 23:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:55:00 [---] [cpu_sched_debug] final job list: 23:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:55:00 [---] [cpu_sched_debug] enforce_run_list: end 23:55:00 [---] choose_project(): 86100.000000 23:55:00 [WUProp@Home] doing RPC to get work 23:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349499.09 cpu 2099132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099132.90 23:55:00 [---] Got 0 tasks 23:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:56:00 [---] [rr_sim] doing sim: CPU sched 23:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965236151.17G/439.78G) 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194821.49 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 219482.15 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219482.148812 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975339.34 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 197533.93 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417016.082743 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777805.41 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 177780.54 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594796.623281 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1600024.86 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 160002.49 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754799.109765 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1440022.38 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 144002.24 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898801.347601 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1296020.14 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 129602.01 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028403.361653 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166418.13 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 116641.81 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145045.174300 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049776.31 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 104977.63 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1250022.805682 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944798.68 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 94479.87 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344502.673926 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850318.81 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 85031.88 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429534.555345 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765286.93 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 76528.69 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506063.248623 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688758.24 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 68875.82 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574939.072573 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619882.42 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 61988.24 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636927.314128 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557894.17 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 55789.42 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692716.731527 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502104.76 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 50210.48 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742927.207187 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451894.28 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 45189.43 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788116.635280 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406704.85 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 40670.49 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828787.120564 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366034.37 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 36603.44 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865390.557320 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329430.93 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 32943.09 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898333.650400 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296487.84 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 29648.78 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927982.434172 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266839.05 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 26683.91 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954666.339567 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240155.15 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 24015.51 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978681.854423 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216139.63 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 21613.96 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000295.817793 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194525.67 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 19452.57 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019748.384826 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175073.10 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 17507.31 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037255.695155 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157565.79 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 15756.58 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2053012.274452 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141809.21 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 14180.92 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067193.195819 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127628.29 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 12762.83 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079956.025049 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114865.46 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 11486.55 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091442.571356 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103378.92 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 10337.89 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101780.463033 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93041.03 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 9304.10 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111084.565542 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83736.92 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 8373.69 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119458.257800 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75363.23 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 7536.32 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126994.580832 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67826.91 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 6782.69 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133777.271561 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61044.22 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 6104.42 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139881.693217 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54939.79 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 5493.98 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145375.672707 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49445.82 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 4944.58 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150320.254249 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44501.23 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 4450.12 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154770.377636 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40051.11 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 4005.11 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158775.488685 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36046.00 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3604.60 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162380.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32441.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165980.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28841.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169580.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25241.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173180.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21641.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176780.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18041.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180380.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14441.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183980.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10841.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187580.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7241.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191180.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3641.40 sec) 23:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194780.088628 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 41.40 sec) 23:56:00 [GPUGRID] [rr_sim] 2194821.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965236151.17G/439.78G) 23:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854022.40 23:56:00 [---] pick_jobs_to_run() start 23:56:00 [---] pick_jobs_to_run() end 23:56:00 [---] [rr_sim] end 23:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:56:00 [---] [cpu_sched_debug] preliminary job list: 23:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:56:00 [---] [cpu_sched_debug] final job list: 23:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:56:00 [---] [cpu_sched_debug] enforce_run_list: end 23:56:00 [---] choose_project(): 86160.000000 23:56:00 [WUProp@Home] doing RPC to get work 23:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349439.09 cpu 2099072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099072.90 23:56:00 [---] Got 0 tasks 23:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:57:00 [---] [rr_sim] doing sim: CPU sched 23:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965208560.82G/439.78G) 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194758.75 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 219475.88 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219475.875124 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975282.88 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 197528.29 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 417004.162736 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777754.59 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 177775.46 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594779.621586 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599979.13 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 159997.91 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754777.534551 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439981.22 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 143998.12 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898775.656220 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295983.10 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 129598.31 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028373.965722 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166384.79 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 116638.48 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1145012.444274 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049746.31 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 104974.63 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249987.074970 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944771.68 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 94477.17 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344464.242597 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850294.51 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 85029.45 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429493.693461 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765265.06 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 76526.51 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1506020.199239 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688738.55 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 68873.86 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574894.054439 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619864.70 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 61986.47 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636880.524119 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557878.23 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 55787.82 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692668.346831 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502090.40 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 50209.04 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742877.387272 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451881.36 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 45188.14 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788065.523669 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406693.23 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 40669.32 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828734.846426 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366023.90 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 36602.39 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865337.236907 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329421.51 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 32942.15 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898279.388341 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296479.36 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 29647.94 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927927.324630 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266831.43 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 26683.14 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954610.467291 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240148.28 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 24014.83 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978625.295686 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216133.46 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 21613.35 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000238.641241 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194520.11 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 19452.01 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019690.652241 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175068.10 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 17506.81 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037197.462141 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157561.29 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 15756.13 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052953.591051 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141805.16 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 14180.52 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067134.107070 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127624.64 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 12762.46 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079896.571487 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114862.18 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 11486.22 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091382.789462 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103375.96 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 10337.60 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101720.385640 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93038.37 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 9303.84 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2111024.222200 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83734.53 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 8373.45 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119397.675104 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75361.08 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 7536.11 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126933.782717 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67824.97 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 6782.50 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133716.279570 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61042.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 6104.25 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139820.526737 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54938.22 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 5493.82 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145314.349187 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49444.40 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 4944.44 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150258.789392 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44499.96 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 4450.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154708.785577 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40049.97 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 4005.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158713.782143 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36044.97 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3604.50 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162318.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32440.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165918.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28840.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169518.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25240.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173118.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21640.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176718.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18040.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180318.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14440.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183918.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10840.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187518.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7240.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191118.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3640.47 sec) 23:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194718.279053 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40.47 sec) 23:57:00 [GPUGRID] [rr_sim] 2194758.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965208560.82G/439.78G) 23:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854019.66 23:57:00 [---] pick_jobs_to_run() start 23:57:00 [---] pick_jobs_to_run() end 23:57:00 [---] [rr_sim] end 23:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:57:00 [---] [cpu_sched_debug] preliminary job list: 23:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: yes) 23:57:00 [---] [cpu_sched_debug] final job list: 23:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 23:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:57:00 [---] [cpu_sched_debug] enforce_run_list: end 23:57:00 [---] choose_project(): 86220.000000 23:57:00 [WUProp@Home] doing RPC to get work 23:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349379.09 cpu 2099012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2099012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2099012.90 23:57:00 [---] Got 0 tasks 23:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:58:00 [---] [rr_sim] doing sim: CPU sched 23:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965180970.46G/439.78G) 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194696.01 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 219469.60 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219469.601436 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975226.41 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 197522.64 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416992.242728 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777703.77 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 177770.38 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594762.619891 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599933.39 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 159993.34 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754755.959337 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439940.06 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 143994.01 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898749.964839 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295946.05 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 129594.60 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028344.569791 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166351.44 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 116635.14 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144979.714248 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049716.30 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 104971.63 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249951.344259 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944744.67 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 94474.47 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344425.811269 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850270.20 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 85027.02 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429452.831578 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765243.18 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 76524.32 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505977.149856 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688718.86 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 68871.89 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574849.036306 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619846.98 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 61984.70 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636833.734111 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557862.28 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 55786.23 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692619.962136 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502076.05 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 50207.61 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742827.567358 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451868.45 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 45186.84 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1788014.412058 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406681.60 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 40668.16 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828682.572288 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366013.44 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 36601.34 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865283.916495 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329412.10 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 32941.21 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898225.126281 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296470.89 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 29647.09 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927872.215089 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266823.80 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 26682.38 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954554.595015 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240141.42 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 24014.14 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978568.736950 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216127.28 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 21612.73 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000181.464690 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194514.55 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 19451.45 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019632.919657 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175063.09 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 17506.31 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037139.229127 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157556.79 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 15755.68 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052894.907650 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141801.11 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 14180.11 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067075.018321 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127621.00 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 12762.10 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079837.117924 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114858.90 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 11485.89 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091323.007568 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103373.01 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 10337.30 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101660.308247 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93035.71 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 9303.57 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110963.878858 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83732.14 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 8373.21 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119337.092408 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75358.92 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 7535.89 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126872.984603 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67823.03 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 6782.30 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133655.287578 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61040.73 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 6104.07 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139759.360256 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54936.65 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 5493.67 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145253.025666 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49442.99 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 4944.30 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150197.324535 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44498.69 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 4449.87 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154647.193518 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40048.82 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 4004.88 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158652.075601 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36043.94 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3604.39 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162256.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32439.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165856.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28839.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169456.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25239.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173056.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21639.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176656.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18039.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180256.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14439.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183856.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10839.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187456.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7239.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2191056.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3639.54 sec) 23:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194656.469477 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39.54 sec) 23:58:00 [GPUGRID] [rr_sim] 2194696.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965180970.46G/439.78G) 23:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854016.93 23:58:00 [---] pick_jobs_to_run() start 23:58:00 [---] pick_jobs_to_run() end 23:58:00 [---] [rr_sim] end 23:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:58:00 [---] [cpu_sched_debug] preliminary job list: 23:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 23:58:00 [---] [cpu_sched_debug] final job list: 23:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 23:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:58:00 [---] [cpu_sched_debug] enforce_run_list: end 23:58:00 [---] choose_project(): 86280.000000 23:58:00 [WUProp@Home] doing RPC to get work 23:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349319.09 cpu 2098952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098952.90 23:58:00 [---] Got 0 tasks 23:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 23:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 23:59:00 [---] [rr_sim] doing sim: CPU sched 23:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965153380.11G/439.78G) 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194633.28 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 219463.33 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219463.327748 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975169.95 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 197516.99 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416980.322720 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777652.95 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 177765.30 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594745.618196 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599887.66 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 159988.77 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754734.384124 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439898.89 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 143989.89 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898724.273459 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295909.00 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 129590.90 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028315.173860 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166318.10 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 116631.81 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144946.984222 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049686.29 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 104968.63 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249915.613547 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944717.66 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 94471.77 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344387.379940 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850245.90 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 85024.59 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429411.969694 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765221.31 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 76522.13 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505934.100472 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688699.18 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 68869.92 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574804.018172 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619829.26 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 61982.93 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636786.944102 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557846.33 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 55784.63 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692571.577440 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502061.70 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 50206.17 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742777.747443 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451855.53 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 45185.55 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787963.300446 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406669.98 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 40667.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828630.298149 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 366002.98 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 36600.30 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865230.596082 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329402.68 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 32940.27 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898170.864221 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296462.41 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 29646.24 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927817.105547 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266816.17 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 26681.62 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954498.722739 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240134.55 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 24013.46 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978512.178213 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216121.10 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 21612.11 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000124.288139 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194508.99 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 19450.90 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019575.187073 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175058.09 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 17505.81 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037080.996113 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157552.28 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 15755.23 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052836.224249 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141797.05 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 14179.71 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2067015.929572 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127617.35 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 12761.73 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079777.664362 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114855.61 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 11485.56 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091263.225673 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103370.05 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 10337.01 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101600.230854 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93033.05 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 9303.30 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110903.535516 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83729.74 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 8372.97 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119276.509712 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75356.77 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 7535.68 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126812.186488 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67821.09 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 6782.11 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133594.295587 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61038.98 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 6103.90 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139698.193776 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54935.08 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 5493.51 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145191.702146 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49441.58 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 4944.16 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150135.859679 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44497.42 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 4449.74 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154585.601458 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40047.68 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 4004.77 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158590.369060 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36042.91 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3604.29 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162194.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32438.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165794.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28838.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169394.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25238.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172994.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21638.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176594.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18038.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180194.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14438.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183794.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10838.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187394.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7238.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190994.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3638.62 sec) 23:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194594.659901 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 38.62 sec) 23:59:00 [GPUGRID] [rr_sim] 2194633.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965153380.11G/439.78G) 23:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854014.19 23:59:00 [---] pick_jobs_to_run() start 23:59:00 [---] pick_jobs_to_run() end 23:59:00 [---] [rr_sim] end 23:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 23:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 23:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 23:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 23:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 23:59:00 [---] [cpu_sched_debug] preliminary job list: 23:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 23:59:00 [---] [cpu_sched_debug] final job list: 23:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 23:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 23:59:00 [---] [cpu_sched_debug] enforce_run_list: end 23:59:00 [---] choose_project(): 86340.000000 23:59:00 [WUProp@Home] doing RPC to get work 23:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349259.09 cpu 2098892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098892.90 23:59:00 [---] Got 0 tasks 24:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 24:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 24:00:00 [---] [rr_sim] doing sim: CPU sched 24:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965125789.75G/439.78G) 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194570.54 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 219457.05 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219457.054059 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975113.49 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 197511.35 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416968.402713 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777602.14 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 177760.21 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594728.616501 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599841.92 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 159984.19 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754712.808910 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439857.73 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 143985.77 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898698.582078 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295871.96 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 129587.20 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028285.777930 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166284.76 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 116628.48 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144914.254196 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049656.29 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 104965.63 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249879.882836 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944690.66 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 94469.07 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344348.948611 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850221.59 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 85022.16 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429371.107810 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765199.43 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 76519.94 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505891.051088 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688679.49 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 68867.95 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574759.000038 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619811.54 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 61981.15 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636740.154094 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557830.39 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 55783.04 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692523.192744 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502047.35 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 50204.73 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742727.927529 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451842.61 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 45184.26 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787912.188835 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406658.35 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 40665.84 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828578.024011 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365992.52 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 36599.25 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865177.275669 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329393.26 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 32939.33 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898116.602162 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296453.94 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 29645.39 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927761.996005 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266808.54 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 26680.85 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954442.850464 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240127.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 24012.77 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978455.619476 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216114.92 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 21611.49 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000067.111588 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194503.43 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 19450.34 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019517.454489 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175053.09 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 17505.31 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2037022.763099 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157547.78 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 15754.78 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052777.540848 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141793.00 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 14179.30 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066956.840823 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127613.70 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 12761.37 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079718.210800 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114852.33 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 11485.23 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091203.443779 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103367.10 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 10336.71 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101540.153461 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93030.39 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 9303.04 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110843.192174 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83727.35 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 8372.73 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119215.927016 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75354.61 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 7535.46 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126751.388373 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67819.15 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 6781.92 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133533.303595 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61037.24 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 6103.72 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139637.027295 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54933.51 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 5493.35 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145130.378625 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49440.16 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 4944.02 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150074.394822 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44496.15 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 4449.61 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154524.009399 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40046.53 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 4004.65 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158528.662518 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36041.88 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3604.19 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162132.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32437.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165732.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28837.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169332.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25237.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172932.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21637.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176532.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18037.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180132.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14437.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183732.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10837.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187332.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7237.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190932.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3637.69 sec) 24:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194532.850326 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 37.69 sec) 24:00:00 [GPUGRID] [rr_sim] 2194570.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965125789.75G/439.78G) 24:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854011.45 24:00:00 [---] pick_jobs_to_run() start 24:00:00 [---] pick_jobs_to_run() end 24:00:00 [---] [rr_sim] end 24:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 24:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 24:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 24:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 24:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 24:00:00 [---] [cpu_sched_debug] preliminary job list: 24:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 24:00:00 [---] [cpu_sched_debug] final job list: 24:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 24:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 24:00:00 [---] [cpu_sched_debug] enforce_run_list: end 24:00:00 [---] choose_project(): 86400.000000 24:00:00 [WUProp@Home] doing RPC to get work 24:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349199.09 cpu 2098832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098832.90 24:00:00 [---] Got 0 tasks 1 day 00:01:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:01:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:01:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:01:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965098199.39G/439.78G) 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194507.80 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 219450.78 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219450.780371 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975057.02 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 197505.70 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416956.482705 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777551.32 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 177755.13 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594711.614806 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599796.19 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 159979.62 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754691.233696 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439816.57 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 143981.66 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898672.890698 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295834.91 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 129583.49 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028256.381999 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166251.42 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 116625.14 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144881.524170 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049626.28 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 104962.63 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249844.152124 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944663.65 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 94466.37 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344310.517283 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850197.29 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 85019.73 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429330.245926 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765177.56 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 76517.76 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505848.001704 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688659.80 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 68865.98 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574713.981905 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619793.82 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 61979.38 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636693.364085 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557814.44 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 55781.44 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692474.808048 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502033.00 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 50203.30 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742678.107614 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451829.70 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 45182.97 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787861.077224 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406646.73 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 40664.67 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828525.749873 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365982.05 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 36598.21 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865123.955256 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329383.85 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 32938.38 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898062.340102 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296445.46 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 29644.55 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927706.886463 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266800.92 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 26680.09 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954386.978188 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240120.83 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 24012.08 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978399.060740 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216108.74 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 21610.87 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2000009.935037 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194497.87 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 19449.79 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019459.721904 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175048.08 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 17504.81 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036964.530085 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157543.27 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 15754.33 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052718.857448 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141788.95 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 14178.89 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066897.752074 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127610.05 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 12761.01 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079658.757238 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114849.05 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 11484.90 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091143.661885 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103364.14 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 10336.41 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101480.076068 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93027.73 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 9302.77 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110782.848832 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83724.95 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 8372.50 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119155.344320 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75352.46 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 7535.25 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126690.590259 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67817.21 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 6781.72 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133472.311604 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61035.49 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 6103.55 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139575.860815 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54931.94 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 5493.19 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145069.055104 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49438.75 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 4943.87 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2150012.929965 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44494.87 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 4449.49 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154462.417340 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40045.39 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 4004.54 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158466.955977 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36040.85 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3604.08 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162071.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32436.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165671.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28836.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169271.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25236.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172871.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21636.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176471.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18036.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180071.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14436.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183671.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10836.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187271.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7236.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190871.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3636.76 sec) 1 day 00:01:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194471.040750 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36.76 sec) 1 day 00:01:00 [GPUGRID] [rr_sim] 2194507.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965098199.39G/439.78G) 1 day 00:01:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854008.72 1 day 00:01:00 [---] pick_jobs_to_run() start 1 day 00:01:00 [---] pick_jobs_to_run() end 1 day 00:01:00 [---] [rr_sim] end 1 day 00:01:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:01:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:01:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:01:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:01:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:01:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:01:00 [---] [cpu_sched_debug] final job list: 1 day 00:01:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:01:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:01:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:01:00 [---] choose_project(): 86460.000000 1 day 00:01:00 [WUProp@Home] doing RPC to get work 1 day 00:01:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349139.09 cpu 2098772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098772.90 1 day 00:01:00 [---] Got 0 tasks 1 day 00:02:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:02:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:02:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:02:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965070609.04G/439.78G) 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194445.07 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 219444.51 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219444.506683 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1975000.56 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 197500.06 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416944.562697 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777500.50 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 177750.05 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594694.613111 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599750.45 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 159975.05 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754669.658482 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439775.41 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 143977.54 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898647.199317 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295797.87 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 129579.79 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028226.986068 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166218.08 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 116621.81 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144848.794144 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049596.27 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 104959.63 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249808.421413 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944636.65 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 94463.66 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344272.085954 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850172.98 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 85017.30 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429289.384042 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765155.68 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 76515.57 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505804.952320 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688640.11 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 68864.01 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574668.963771 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619776.10 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 61977.61 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636646.574077 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557798.49 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 55779.85 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692426.423352 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502018.64 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 50201.86 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742628.287700 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451816.78 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 45181.68 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787809.965613 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406635.10 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 40663.51 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828473.475734 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365971.59 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 36597.16 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865070.634844 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329374.43 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 32937.44 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1898008.078042 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296436.99 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 29643.70 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927651.776921 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266793.29 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 26679.33 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954331.105912 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240113.96 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 24011.40 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978342.502003 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216102.56 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 21610.26 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999952.758486 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194492.31 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 19449.23 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019401.989320 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175043.08 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 17504.31 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036906.297071 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157538.77 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 15753.88 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052660.174047 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141784.89 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 14178.49 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066838.663325 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127606.40 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 12760.64 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079599.303675 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114845.76 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 11484.58 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091083.879991 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103361.19 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 10336.12 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101419.998674 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93025.07 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 9302.51 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110722.505490 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83722.56 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 8372.26 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119094.761624 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75350.31 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 7535.03 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126629.792144 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67815.27 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 6781.53 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133411.319613 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61033.75 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 6103.37 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139514.694334 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54930.37 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 5493.04 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2145007.731584 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49437.34 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 4943.73 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149951.465108 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44493.60 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 4449.36 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154400.825280 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40044.24 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 4004.42 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158405.249435 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36039.82 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3603.98 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162009.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32435.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165609.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28835.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169209.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25235.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172809.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21635.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176409.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18035.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180009.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14435.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183609.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10835.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187209.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7235.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190809.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3635.84 sec) 1 day 00:02:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194409.231174 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35.84 sec) 1 day 00:02:00 [GPUGRID] [rr_sim] 2194445.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965070609.04G/439.78G) 1 day 00:02:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854005.98 1 day 00:02:00 [---] pick_jobs_to_run() start 1 day 00:02:00 [---] pick_jobs_to_run() end 1 day 00:02:00 [---] [rr_sim] end 1 day 00:02:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:02:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:02:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:02:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:02:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:02:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:02:00 [---] [cpu_sched_debug] final job list: 1 day 00:02:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:02:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:02:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:02:00 [---] choose_project(): 86520.000000 1 day 00:02:00 [WUProp@Home] doing RPC to get work 1 day 00:02:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349079.09 cpu 2098712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098712.90 1 day 00:02:00 [---] Got 0 tasks 1 day 00:03:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:03:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:03:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:03:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965043018.68G/439.78G) 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194382.33 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 219438.23 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219438.232995 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974944.10 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 197494.41 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416932.642690 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777449.69 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 177744.97 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594677.611416 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599704.72 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 159970.47 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754648.083269 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439734.25 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 143973.42 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898621.507936 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295760.82 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 129576.08 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028197.590137 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166184.74 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 116618.47 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144816.064118 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049566.27 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 104956.63 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249772.690701 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944609.64 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 94460.96 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344233.654626 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850148.68 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 85014.87 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429248.522158 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765133.81 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 76513.38 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505761.902937 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688620.43 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 68862.04 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574623.945638 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619758.38 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 61975.84 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636599.784069 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557782.55 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 55778.25 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692378.038656 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 502004.29 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 50200.43 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742578.467785 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451803.86 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 45180.39 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787758.854001 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406623.48 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 40662.35 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828421.201596 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365961.13 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 36596.11 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1865017.314431 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329365.02 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 32936.50 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897953.815983 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296428.51 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 29642.85 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927596.667379 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266785.66 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 26678.57 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954275.233636 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240107.10 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 24010.71 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978285.943267 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216096.39 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 21609.64 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999895.581935 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194486.75 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 19448.67 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019344.256736 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175038.07 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 17503.81 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036848.064057 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157534.27 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 15753.43 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052601.490646 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141780.84 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 14178.08 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066779.574576 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127602.76 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 12760.28 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079539.850113 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114842.48 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 11484.25 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2091024.098096 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103358.23 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 10335.82 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101359.921281 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93022.41 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 9302.24 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110662.162148 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83720.17 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 8372.02 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2119034.178928 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75348.15 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 7534.82 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126568.994030 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67813.34 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 6781.33 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133350.327621 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61032.00 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 6103.20 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139453.527854 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54928.80 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 5492.88 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144946.408063 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49435.92 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 4943.59 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149890.000251 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44492.33 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 4449.23 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154339.233221 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40043.10 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 4004.31 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158343.542894 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36038.79 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3603.88 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161947.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32434.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165547.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28834.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169147.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25234.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172747.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21634.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176347.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18034.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179947.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14434.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183547.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10834.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187147.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7234.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190747.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3634.91 sec) 1 day 00:03:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194347.421599 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 34.91 sec) 1 day 00:03:00 [GPUGRID] [rr_sim] 2194382.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965043018.68G/439.78G) 1 day 00:03:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854003.24 1 day 00:03:00 [---] pick_jobs_to_run() start 1 day 00:03:00 [---] pick_jobs_to_run() end 1 day 00:03:00 [---] [rr_sim] end 1 day 00:03:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:03:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:03:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:03:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:03:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:03:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:03:00 [---] [cpu_sched_debug] final job list: 1 day 00:03:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:03:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:03:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:03:00 [---] choose_project(): 86580.000000 1 day 00:03:00 [WUProp@Home] doing RPC to get work 1 day 00:03:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 349019.09 cpu 2098652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098652.90 1 day 00:03:00 [---] Got 0 tasks 1 day 00:04:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:04:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:04:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:04:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (965015428.32G/439.78G) 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194319.59 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 219431.96 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219431.959306 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974887.63 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 197488.76 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416920.722682 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777398.87 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 177739.89 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594660.609720 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599658.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 159965.90 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754626.508055 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439693.09 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 143969.31 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898595.816556 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295723.78 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 129572.38 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028168.194207 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166151.40 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 116615.14 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144783.334092 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049536.26 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 104953.63 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249736.959990 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944582.63 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 94458.26 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344195.223297 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850124.37 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 85012.44 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429207.660274 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765111.93 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 76511.19 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505718.853553 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688600.74 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 68860.07 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574578.927504 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619740.67 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 61974.07 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636552.994060 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557766.60 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 55776.66 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692329.653960 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501989.94 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 50198.99 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742528.647871 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451790.95 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 45179.09 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787707.742390 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406611.85 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 40661.19 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828368.927458 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365950.67 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 36595.07 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864963.994018 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329355.60 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 32935.56 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897899.553923 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296420.04 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 29642.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927541.557837 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266778.04 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 26677.80 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954219.361360 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240100.23 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 24010.02 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978229.384530 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216090.21 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 21609.02 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999838.405384 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194481.19 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 19448.12 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019286.524152 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175033.07 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 17503.31 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036789.831043 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157529.76 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 15752.98 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052542.807245 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141776.79 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 14177.68 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066720.485827 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127599.11 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 12759.91 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079480.396551 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114839.20 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 11483.92 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090964.316202 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103355.28 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 10335.53 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101299.843888 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93019.75 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 9301.97 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110601.818806 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83717.77 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 8371.78 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118973.596232 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75346.00 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 7534.60 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126508.195915 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67811.40 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 6781.14 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133289.335630 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61030.26 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 6103.03 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139392.361373 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54927.23 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 5492.72 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144885.084543 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49434.51 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 4943.45 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149828.535395 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44491.06 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 4449.11 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154277.641162 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40041.95 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 4004.20 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158281.836352 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36037.76 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3603.78 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161885.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32433.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165485.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28833.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169085.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25233.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172685.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21633.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176285.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18033.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179885.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14433.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183485.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10833.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187085.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7233.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190685.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3633.98 sec) 1 day 00:04:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194285.612023 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33.98 sec) 1 day 00:04:00 [GPUGRID] [rr_sim] 2194319.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (965015428.32G/439.78G) 1 day 00:04:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854000.51 1 day 00:04:00 [---] pick_jobs_to_run() start 1 day 00:04:00 [---] pick_jobs_to_run() end 1 day 00:04:00 [---] [rr_sim] end 1 day 00:04:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:04:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:04:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:04:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:04:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:04:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:04:00 [---] [cpu_sched_debug] final job list: 1 day 00:04:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:04:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:04:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:04:00 [---] choose_project(): 86640.000000 1 day 00:04:00 [WUProp@Home] doing RPC to get work 1 day 00:04:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348959.09 cpu 2098592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098592.90 1 day 00:04:00 [---] Got 0 tasks 1 day 00:05:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:05:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:05:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:05:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964987837.97G/439.78G) 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194256.86 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 219425.69 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219425.685618 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974831.17 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 197483.12 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416908.802675 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777348.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 177734.81 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594643.608025 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599613.25 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 159961.32 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754604.932841 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439651.92 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 143965.19 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898570.125175 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295686.73 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 129568.67 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028138.798276 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166118.06 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 116611.81 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144750.604066 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049506.25 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 104950.63 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249701.229278 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944555.63 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 94455.56 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344156.791968 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850100.06 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 85010.01 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429166.798390 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765090.06 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 76509.01 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505675.804169 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688581.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 68858.11 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574533.909370 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619722.95 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 61972.29 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636506.204052 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557750.65 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 55775.07 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692281.269265 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501975.59 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 50197.56 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742478.827956 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451778.03 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 45177.80 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787656.630779 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406600.23 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 40660.02 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828316.653319 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365940.20 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 36594.02 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864910.673606 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329346.18 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 32934.62 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897845.291863 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296411.56 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 29641.16 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927486.448295 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266770.41 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 26677.04 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954163.489084 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240093.37 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 24009.34 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978172.825794 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216084.03 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 21608.40 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999781.228833 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194475.63 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 19447.56 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019228.791567 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175028.06 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 17502.81 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036731.598029 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157525.26 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 15752.53 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052484.123844 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141772.73 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 14177.27 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066661.397078 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127595.46 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 12759.55 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079420.942988 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114835.91 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 11483.59 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090904.534308 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103352.32 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 10335.23 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101239.766495 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93017.09 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 9301.71 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110541.475464 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83715.38 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 8371.54 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118913.013536 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75343.84 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 7534.38 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126447.397800 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67809.46 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 6780.95 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133228.343639 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61028.51 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 6102.85 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139331.194893 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54925.66 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 5492.57 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144823.761022 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49433.10 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 4943.31 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149767.070538 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44489.79 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 4448.98 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154216.049102 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40040.81 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 4004.08 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158220.129810 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36036.73 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3603.67 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161823.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32433.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165423.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28833.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169023.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25233.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172623.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21633.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176223.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18033.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179823.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14433.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183423.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10833.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187023.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7233.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190623.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3633.05 sec) 1 day 00:05:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194223.802448 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 33.05 sec) 1 day 00:05:00 [GPUGRID] [rr_sim] 2194256.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964987837.97G/439.78G) 1 day 00:05:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853997.77 1 day 00:05:00 [---] pick_jobs_to_run() start 1 day 00:05:00 [---] pick_jobs_to_run() end 1 day 00:05:00 [---] [rr_sim] end 1 day 00:05:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:05:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:05:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:05:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:05:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:05:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:05:00 [---] [cpu_sched_debug] final job list: 1 day 00:05:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:05:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:05:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:05:00 [---] choose_project(): 86700.000000 1 day 00:05:00 [WUProp@Home] doing RPC to get work 1 day 00:05:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348899.09 cpu 2098532.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098532.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098532.90 1 day 00:05:00 [---] Got 0 tasks 1 day 00:06:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:06:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:06:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:06:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964960247.61G/439.78G) 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194194.12 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 219419.41 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219419.411930 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974774.71 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 197477.47 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416896.882667 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777297.24 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 177729.72 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594626.606330 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599567.51 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 159956.75 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754583.357627 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439610.76 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 143961.08 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898544.433795 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295649.69 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 129564.97 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028109.402345 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166084.72 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 116608.47 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144717.874041 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049476.25 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 104947.62 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249665.498567 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944528.62 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 94452.86 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344118.360640 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850075.76 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 85007.58 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429125.936506 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765068.18 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 76506.82 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505632.754785 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688561.36 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 68856.14 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574488.891237 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619705.23 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 61970.52 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636459.414043 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557734.71 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 55773.47 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692232.884569 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501961.23 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 50196.12 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742429.008042 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451765.11 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 45176.51 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787605.519168 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406588.60 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 40658.86 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828264.379181 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365929.74 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 36592.97 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864857.353193 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329336.77 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 32933.68 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897791.029804 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296403.09 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 29640.31 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927431.338753 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266762.78 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 26676.28 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954107.616808 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240086.50 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 24008.65 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978116.267057 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216077.85 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 21607.79 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999724.052281 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194470.07 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 19447.01 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019171.058983 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175023.06 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 17502.31 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036673.365015 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157520.75 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 15752.08 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052425.440443 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141768.68 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 14176.87 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066602.308329 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127591.81 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 12759.18 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079361.489426 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114832.63 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 11483.26 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090844.752414 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103349.37 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 10334.94 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101179.689102 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93014.43 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 9301.44 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110481.132122 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83712.99 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 8371.30 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118852.430840 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75341.69 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 7534.17 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126386.599686 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67807.52 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 6780.75 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133167.351647 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61026.77 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 6102.68 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139270.028413 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54924.09 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 5492.41 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144762.437501 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49431.68 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 4943.17 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149705.605681 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44488.51 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 4448.85 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154154.457043 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40039.66 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 4003.97 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158158.423269 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36035.70 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3603.57 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161761.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32432.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165361.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28832.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168961.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25232.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172561.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21632.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176161.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18032.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179761.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14432.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183361.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10832.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186961.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7232.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190561.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3632.13 sec) 1 day 00:06:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194161.992872 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32.13 sec) 1 day 00:06:00 [GPUGRID] [rr_sim] 2194194.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964960247.61G/439.78G) 1 day 00:06:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853995.03 1 day 00:06:00 [---] pick_jobs_to_run() start 1 day 00:06:00 [---] pick_jobs_to_run() end 1 day 00:06:00 [---] [rr_sim] end 1 day 00:06:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:06:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:06:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:06:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:06:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:06:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:06:00 [---] [cpu_sched_debug] final job list: 1 day 00:06:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:06:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:06:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:06:00 [---] choose_project(): 86760.000000 1 day 00:06:00 [WUProp@Home] doing RPC to get work 1 day 00:06:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348839.09 cpu 2098472.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098472.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098472.90 1 day 00:06:00 [---] Got 0 tasks 1 day 00:07:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:07:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:07:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:07:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964932657.26G/439.78G) 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194131.38 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 219413.14 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219413.138242 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974718.24 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 197471.82 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416884.962659 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777246.42 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 177724.64 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594609.604635 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599521.78 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 159952.18 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754561.782413 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439569.60 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 143956.96 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898518.742414 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295612.64 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 129561.26 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028080.006414 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166051.38 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 116605.14 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144685.144015 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049446.24 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 104944.62 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249629.767855 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944501.61 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 94450.16 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344079.929311 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850051.45 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 85005.15 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429085.074622 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765046.31 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 76504.63 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505589.705402 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688541.68 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 68854.17 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574443.873103 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619687.51 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 61968.75 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636412.624035 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557718.76 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 55771.88 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692184.499873 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501946.88 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 50194.69 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742379.188127 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451752.19 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 45175.22 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787554.407556 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406576.97 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 40657.70 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828212.105043 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365919.28 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 36591.93 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864804.032780 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329327.35 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 32932.73 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897736.767744 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296394.61 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 29639.46 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927376.229211 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266755.15 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 26675.52 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1954051.744532 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240079.64 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 24007.96 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978059.708321 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216071.67 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 21607.17 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999666.875730 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194464.51 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 19446.45 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019113.326399 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175018.06 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 17501.81 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036615.132001 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157516.25 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 15751.63 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052366.757043 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141764.63 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 14176.46 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066543.219580 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127588.16 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 12758.82 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079302.035864 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114829.35 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 11482.93 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090784.970519 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103346.41 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 10334.64 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101119.611709 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93011.77 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 9301.18 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110420.788780 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83710.59 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 8371.06 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118791.848144 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75339.53 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 7533.95 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126325.801571 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67805.58 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 6780.56 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133106.359656 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61025.02 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 6102.50 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139208.861932 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54922.52 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 5492.25 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144701.113981 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49430.27 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 4943.03 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149644.140824 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44487.24 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 4448.72 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154092.864984 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40038.52 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 4003.85 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158096.716727 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36034.67 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3603.47 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161700.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32431.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165300.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28831.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168900.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25231.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172500.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21631.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176100.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18031.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179700.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14431.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183300.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10831.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186900.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7231.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190500.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3631.20 sec) 1 day 00:07:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194100.183296 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 31.20 sec) 1 day 00:07:00 [GPUGRID] [rr_sim] 2194131.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964932657.26G/439.78G) 1 day 00:07:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853992.30 1 day 00:07:00 [---] pick_jobs_to_run() start 1 day 00:07:00 [---] pick_jobs_to_run() end 1 day 00:07:00 [---] [rr_sim] end 1 day 00:07:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:07:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:07:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:07:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:07:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:07:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:07:00 [---] [cpu_sched_debug] final job list: 1 day 00:07:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:07:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:07:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:07:00 [---] choose_project(): 86820.000000 1 day 00:07:00 [WUProp@Home] doing RPC to get work 1 day 00:07:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348779.09 cpu 2098412.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098412.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098412.90 1 day 00:07:00 [---] Got 0 tasks 1 day 00:08:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:08:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:08:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:08:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964905066.90G/439.78G) 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194068.65 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 219406.86 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219406.864554 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974661.78 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 197466.18 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416873.042652 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777195.60 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 177719.56 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594592.602940 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599476.04 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 159947.60 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754540.207200 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439528.44 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 143952.84 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898493.051033 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295575.59 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 129557.56 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028050.610484 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1166018.04 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 116601.80 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144652.413989 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049416.23 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 104941.62 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249594.037143 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944474.61 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 94447.46 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344041.497983 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850027.15 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 85002.71 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429044.212738 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765024.43 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 76502.44 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505546.656018 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688521.99 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 68852.20 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574398.854970 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619669.79 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 61966.98 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636365.834026 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557702.81 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 55770.28 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692136.115177 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501932.53 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 50193.25 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742329.368213 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451739.28 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 45173.93 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787503.295945 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406565.35 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 40656.53 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828159.830904 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365908.81 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 36590.88 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864750.712367 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329317.93 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 32931.79 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897682.505684 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296386.14 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 29638.61 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927321.119669 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266747.53 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 26674.75 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953995.872256 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240072.77 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 24007.28 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1978003.149584 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216065.50 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 21606.55 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999609.699179 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194458.95 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 19445.89 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2019055.593815 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175013.05 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 17501.31 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036556.898987 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157511.75 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 15751.17 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052308.073642 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141760.57 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 14176.06 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066484.130831 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127584.51 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 12758.45 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079242.582302 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114826.06 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 11482.61 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090725.188625 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103343.46 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 10334.35 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2101059.534316 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93009.11 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 9300.91 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110360.445438 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83708.20 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 8370.82 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118731.265448 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75337.38 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 7533.74 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126265.003457 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67803.64 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 6780.36 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2133045.367665 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61023.28 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 6102.33 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139147.695452 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54920.95 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 5492.10 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144639.790460 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49428.86 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 4942.89 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149582.675968 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44485.97 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 4448.60 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2154031.272924 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40037.37 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 4003.74 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158035.010186 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36033.64 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3603.36 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161638.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32430.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165238.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28830.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168838.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25230.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172438.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21630.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176038.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18030.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179638.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14430.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183238.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10830.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186838.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7230.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190438.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3630.27 sec) 1 day 00:08:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2194038.373721 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 30.27 sec) 1 day 00:08:00 [GPUGRID] [rr_sim] 2194068.65: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964905066.90G/439.78G) 1 day 00:08:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853989.56 1 day 00:08:00 [---] pick_jobs_to_run() start 1 day 00:08:00 [---] pick_jobs_to_run() end 1 day 00:08:00 [---] [rr_sim] end 1 day 00:08:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:08:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:08:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:08:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:08:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:08:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:08:00 [---] [cpu_sched_debug] final job list: 1 day 00:08:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:08:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:08:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:08:00 [---] choose_project(): 86880.000000 1 day 00:08:00 [WUProp@Home] doing RPC to get work 1 day 00:08:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348719.09 cpu 2098352.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098352.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098352.90 1 day 00:08:00 [---] Got 0 tasks 1 day 00:09:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:09:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:09:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:09:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964877476.54G/439.78G) 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2194005.91 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 219400.59 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219400.590865 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974605.32 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 197460.53 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416861.122644 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777144.79 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 177714.48 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594575.601245 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599430.31 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 159943.03 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754518.631986 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439487.28 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 143948.73 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898467.359653 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295538.55 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 129553.85 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1028021.214553 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165984.69 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 116598.47 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144619.683963 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049386.22 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 104938.62 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249558.306432 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944447.60 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 94444.76 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1344003.066654 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 850002.84 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 85000.28 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1429003.350854 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 765002.56 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 76500.26 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505503.606634 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688502.30 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 68850.23 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574353.836836 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619652.07 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 61965.21 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636319.044018 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557686.86 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 55768.69 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692087.730481 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501918.18 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 50191.82 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742279.548298 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451726.36 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 45172.64 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787452.184334 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406553.72 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 40655.37 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828107.556766 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365898.35 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 36589.84 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864697.391955 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329308.52 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 32930.85 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897628.243625 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296377.67 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 29637.77 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927266.010127 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266739.90 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 26673.99 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953939.999980 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240065.91 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 24006.59 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977946.590847 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216059.32 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 21605.93 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999552.522628 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194453.39 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 19445.34 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018997.861231 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175008.05 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 17500.80 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036498.665973 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157507.24 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 15750.72 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052249.390241 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141756.52 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 14175.65 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066425.042082 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127580.87 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 12758.09 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079183.128739 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114822.78 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 11482.28 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090665.406731 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103340.50 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 10334.05 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100999.456923 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93006.45 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 9300.65 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110300.102096 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83705.81 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 8370.58 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118670.682752 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75335.23 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 7533.52 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126204.205342 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67801.70 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 6780.17 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132984.375673 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61021.53 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 6102.15 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139086.528971 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54919.38 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 5491.94 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144578.466939 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49427.44 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 4942.74 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149521.211111 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44484.70 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 4448.47 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153969.680865 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40036.23 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 4003.62 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157973.303644 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36032.61 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3603.26 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161576.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32429.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165176.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28829.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168776.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25229.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172376.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21629.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175976.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18029.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179576.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14429.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183176.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10829.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186776.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7229.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190376.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3629.34 sec) 1 day 00:09:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193976.564145 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 29.34 sec) 1 day 00:09:00 [GPUGRID] [rr_sim] 2194005.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964877476.54G/439.78G) 1 day 00:09:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853986.82 1 day 00:09:00 [---] pick_jobs_to_run() start 1 day 00:09:00 [---] pick_jobs_to_run() end 1 day 00:09:00 [---] [rr_sim] end 1 day 00:09:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:09:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:09:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:09:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:09:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:09:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:09:00 [---] [cpu_sched_debug] final job list: 1 day 00:09:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:09:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:09:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:09:00 [---] choose_project(): 86940.000000 1 day 00:09:00 [WUProp@Home] doing RPC to get work 1 day 00:09:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348659.09 cpu 2098292.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098292.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098292.90 1 day 00:09:00 [---] Got 0 tasks 1 day 00:10:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:10:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:10:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:10:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964849886.19G/439.78G) 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193943.17 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 219394.32 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219394.317177 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974548.85 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 197454.89 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416849.202637 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777093.97 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 177709.40 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594558.599550 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599384.57 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 159938.46 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754497.056772 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439446.11 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 143944.61 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898441.668272 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295501.50 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 129550.15 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027991.818622 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165951.35 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 116595.14 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144586.953937 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049356.22 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 104935.62 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249522.575720 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944420.60 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 94442.06 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343964.635325 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849978.54 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 84997.85 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428962.488970 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764980.68 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 76498.07 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505460.557250 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688482.61 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 68848.26 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574308.818702 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619634.35 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 61963.44 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636272.254009 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557670.92 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 55767.09 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1692039.345785 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501903.83 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 50190.38 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742229.728384 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451713.44 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 45171.34 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787401.072723 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406542.10 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 40654.21 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828055.282628 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365887.89 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 36588.79 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864644.071542 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329299.10 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 32929.91 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897573.981565 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296369.19 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 29636.92 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927210.900586 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266732.27 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 26673.23 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953884.127704 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240059.04 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 24005.90 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977890.032111 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216053.14 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 21605.31 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999495.346077 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194447.83 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 19444.78 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018940.128646 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 175003.04 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 17500.30 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036440.432959 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157502.74 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 15750.27 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052190.706840 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141752.46 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 14175.25 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066365.953333 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127577.22 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 12757.72 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079123.675177 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114819.50 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 11481.95 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090605.624836 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103337.55 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 10333.75 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100939.379530 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93003.79 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 9300.38 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110239.758754 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83703.41 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 8370.34 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118610.100056 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75333.07 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 7533.31 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126143.407227 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67799.76 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 6779.98 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132923.383682 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61019.79 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 6101.98 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2139025.362491 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54917.81 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 5491.78 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144517.143419 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49426.03 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 4942.60 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149459.746254 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44483.43 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 4448.34 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153908.088806 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40035.08 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 4003.51 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157911.597102 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36031.57 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3603.16 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161514.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32428.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165114.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28828.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168714.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25228.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172314.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21628.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175914.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18028.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179514.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14428.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183114.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10828.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186714.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7228.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190314.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3628.42 sec) 1 day 00:10:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193914.754569 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28.42 sec) 1 day 00:10:00 [GPUGRID] [rr_sim] 2193943.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964849886.19G/439.78G) 1 day 00:10:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853984.08 1 day 00:10:00 [---] pick_jobs_to_run() start 1 day 00:10:00 [---] pick_jobs_to_run() end 1 day 00:10:00 [---] [rr_sim] end 1 day 00:10:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:10:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:10:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:10:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:10:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:10:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:10:00 [---] [cpu_sched_debug] final job list: 1 day 00:10:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:10:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:10:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:10:00 [---] choose_project(): 87000.000000 1 day 00:10:00 [WUProp@Home] doing RPC to get work 1 day 00:10:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348599.09 cpu 2098232.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098232.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098232.90 1 day 00:10:00 [---] Got 0 tasks 1 day 00:11:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:11:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:11:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:11:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964822295.83G/439.78G) 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193880.43 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 219388.04 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219388.043489 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974492.39 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 197449.24 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416837.282629 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1777043.15 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 177704.32 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594541.597855 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599338.84 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 159933.88 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754475.481558 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439404.95 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 143940.50 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898415.976891 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295464.46 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 129546.45 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027962.422691 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165918.01 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 116591.80 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144554.223911 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049326.21 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 104932.62 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249486.845009 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944393.59 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 94439.36 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343926.203997 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849954.23 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 84995.42 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428921.627086 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764958.81 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 76495.88 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505417.507866 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688462.93 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 68846.29 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574263.800569 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619616.63 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 61961.66 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636225.464001 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557654.97 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 55765.50 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691990.961090 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501889.47 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 50188.95 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742179.908470 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451700.53 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 45170.05 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787349.961112 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406530.47 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 40653.05 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1828003.008489 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365877.43 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 36587.74 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864590.751129 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329289.68 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 32928.97 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897519.719505 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296360.72 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 29636.07 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927155.791044 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266724.64 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 26672.46 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953828.255428 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240052.18 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 24005.22 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977833.473374 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216046.96 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 21604.70 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999438.169526 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194442.27 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 19444.23 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018882.396062 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174998.04 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 17499.80 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036382.199945 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157498.23 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 15749.82 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052132.023439 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141748.41 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 14174.84 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066306.864584 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127573.57 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 12757.36 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079064.221615 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114816.21 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 11481.62 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090545.842942 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103334.59 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 10333.46 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100879.302137 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 93001.13 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 9300.11 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110179.415412 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83701.02 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 8370.10 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118549.517360 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75330.92 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 7533.09 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126082.609113 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67797.83 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 6779.78 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132862.391690 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61018.04 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 6101.80 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138964.196010 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54916.24 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 5491.62 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144455.819898 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49424.61 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 4942.46 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149398.281397 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44482.15 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 4448.22 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153846.496746 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40033.94 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 4003.39 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157849.890561 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36030.54 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3603.05 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161452.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32427.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165052.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28827.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168652.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25227.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172252.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21627.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175852.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18027.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179452.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14427.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183052.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10827.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186652.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7227.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190252.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3627.49 sec) 1 day 00:11:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193852.944994 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 27.49 sec) 1 day 00:11:00 [GPUGRID] [rr_sim] 2193880.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964822295.83G/439.78G) 1 day 00:11:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853981.35 1 day 00:11:00 [---] pick_jobs_to_run() start 1 day 00:11:00 [---] pick_jobs_to_run() end 1 day 00:11:00 [---] [rr_sim] end 1 day 00:11:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:11:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:11:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:11:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:11:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:11:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:11:00 [---] [cpu_sched_debug] final job list: 1 day 00:11:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:11:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:11:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:11:00 [---] choose_project(): 87060.000000 1 day 00:11:00 [WUProp@Home] doing RPC to get work 1 day 00:11:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348539.09 cpu 2098172.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098172.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098172.90 1 day 00:11:00 [---] Got 0 tasks 1 day 00:12:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:12:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:12:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:12:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964794705.47G/439.78G) 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193817.70 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 219381.77 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219381.769801 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974435.93 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 197443.59 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416825.362621 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776992.34 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 177699.23 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594524.596160 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599293.10 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 159929.31 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754453.906345 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439363.79 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 143936.38 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898390.285511 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295427.41 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 129542.74 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027933.026760 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165884.67 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 116588.47 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144521.493885 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049296.20 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 104929.62 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249451.114297 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944366.58 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 94436.66 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343887.772668 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849929.93 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 84992.99 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428880.765202 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764936.93 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 76493.69 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505374.458483 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688443.24 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 68844.32 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574218.782435 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619598.92 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 61959.89 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636178.673992 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557639.02 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 55763.90 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691942.576394 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501875.12 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 50187.51 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742130.088555 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451687.61 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 45168.76 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787298.849500 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406518.85 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 40651.88 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827950.734351 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365866.96 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 36586.70 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864537.430717 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329280.27 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 32928.03 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897465.457446 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296352.24 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 29635.22 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927100.681502 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266717.02 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 26671.70 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953772.383152 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240045.31 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 24004.53 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977776.914638 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216040.78 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 21604.08 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999380.992975 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194436.71 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 19443.67 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018824.663478 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174993.03 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 17499.30 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036323.966931 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157493.73 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 15749.37 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052073.340038 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141744.36 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 14174.44 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066247.775835 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127569.92 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 12756.99 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2079004.768052 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114812.93 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 11481.29 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090486.061048 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103331.64 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 10333.16 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100819.224744 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92998.47 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 9299.85 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110119.072070 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83698.63 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 8369.86 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118488.934664 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75328.76 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 7532.88 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2126021.810998 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67795.89 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 6779.59 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132801.399699 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61016.30 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 6101.63 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138903.029530 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54914.67 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 5491.47 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144394.496378 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49423.20 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 4942.32 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149336.816541 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44480.88 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 4448.09 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153784.904687 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40032.79 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 4003.28 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157788.184019 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36029.51 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3602.95 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161391.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32426.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164991.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28826.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168591.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25226.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172191.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21626.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175791.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18026.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179391.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14426.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182991.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10826.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186591.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7226.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190191.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3626.56 sec) 1 day 00:12:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193791.135418 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 26.56 sec) 1 day 00:12:00 [GPUGRID] [rr_sim] 2193817.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964794705.47G/439.78G) 1 day 00:12:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1853978.61 1 day 00:12:00 [---] pick_jobs_to_run() start 1 day 00:12:00 [---] pick_jobs_to_run() end 1 day 00:12:00 [---] [rr_sim] end 1 day 00:12:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:12:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:12:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:12:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:12:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:12:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:12:00 [---] [cpu_sched_debug] final job list: 1 day 00:12:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:12:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:12:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:12:00 [---] choose_project(): 87120.000000 1 day 00:12:00 [WUProp@Home] doing RPC to get work 1 day 00:12:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348479.09 cpu 2098112.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098112.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098112.90 1 day 00:12:00 [---] Got 0 tasks 1 day 00:14:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:14:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:14:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:14:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964767115.12G/439.78G) 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193754.96 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 219375.50 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219375.496112 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974379.47 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 197437.95 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416813.442614 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776941.52 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 177694.15 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594507.594465 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599247.37 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 159924.74 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754432.331131 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439322.63 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 143932.26 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898364.594130 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295390.37 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 129539.04 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027903.630830 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165851.33 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 116585.13 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144488.763859 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049266.20 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 104926.62 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249415.383586 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944339.58 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 94433.96 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343849.341340 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849905.62 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 84990.56 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428839.903318 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764915.06 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 76491.51 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505331.409099 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688423.55 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 68842.36 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574173.764301 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619581.20 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 61958.12 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636131.883984 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557623.08 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 55762.31 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691894.191698 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501860.77 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 50186.08 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742080.268641 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451674.69 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 45167.47 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787247.737889 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406507.22 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 40650.72 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827898.460213 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365856.50 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 36585.65 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864484.110304 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329270.85 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 32927.09 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897411.195386 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296343.77 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 29634.38 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1927045.571960 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266709.39 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 26670.94 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953716.510876 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240038.45 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 24003.85 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977720.355901 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216034.61 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 21603.46 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999323.816424 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194431.14 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 19443.11 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018766.930894 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174988.03 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 17498.80 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036265.733917 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157489.23 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 15748.92 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2052014.656638 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141740.30 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 14174.03 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066188.687086 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127566.27 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 12756.63 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078945.314490 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114809.65 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 11480.96 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090426.279154 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103328.68 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 10332.87 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100759.147351 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92995.81 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 9299.58 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2110058.728728 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83696.23 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 8369.62 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118428.351968 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75326.61 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 7532.66 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125961.012884 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67793.95 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 6779.39 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132740.407708 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61014.55 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 6101.46 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138841.863049 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54913.10 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 5491.31 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144333.172857 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49421.79 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 4942.18 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149275.351684 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44479.61 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 4447.96 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153723.312628 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40031.65 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 4003.16 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157726.477478 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36028.48 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3602.85 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161329.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32425.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164929.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28825.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168529.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25225.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172129.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21625.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175729.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18025.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179329.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14425.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182929.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10825.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186529.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7225.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190129.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3625.64 sec) 1 day 00:14:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193729.325842 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25.64 sec) 1 day 00:14:00 [GPUGRID] [rr_sim] 2193754.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964767115.12G/439.78G) 1 day 00:14:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854035.87 1 day 00:14:00 [---] pick_jobs_to_run() start 1 day 00:14:00 [---] pick_jobs_to_run() end 1 day 00:14:00 [---] [rr_sim] end 1 day 00:14:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:14:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:14:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:14:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:14:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:14:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:14:00 [---] [cpu_sched_debug] final job list: 1 day 00:14:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:14:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:14:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:14:00 [---] choose_project(): 87240.000000 1 day 00:14:00 [WUProp@Home] doing RPC to get work 1 day 00:14:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348359.09 cpu 2098052.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2098052.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2098052.90 1 day 00:14:00 [---] Got 0 tasks 1 day 00:15:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:15:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:15:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:15:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964739524.76G/439.78G) 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193692.22 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 219369.22 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219369.222424 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974323.00 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 197432.30 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416801.522606 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776890.70 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 177689.07 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594490.592770 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599201.63 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 159920.16 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754410.755917 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439281.47 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 143928.15 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898338.902750 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295353.32 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 129535.33 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027874.234899 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165817.99 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 116581.80 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144456.033833 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049236.19 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 104923.62 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249379.652874 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944312.57 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 94431.26 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343810.910011 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849881.31 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 84988.13 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428799.041434 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764893.18 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 76489.32 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505288.359715 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688403.86 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 68840.39 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574128.746168 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619563.48 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 61956.35 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636085.093975 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557607.13 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 55760.71 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691845.807002 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501846.42 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 50184.64 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1742030.448726 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451661.78 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 45166.18 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787196.626278 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406495.60 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 40649.56 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827846.186074 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365846.04 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 36584.60 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864430.789891 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329261.43 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 32926.14 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897356.933326 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296335.29 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 29633.53 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926990.462418 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266701.76 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 26670.18 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953660.638600 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240031.59 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 24003.16 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977663.797165 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216028.43 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 21602.84 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999266.639872 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194425.58 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 19442.56 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018709.198309 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174983.03 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 17498.30 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036207.500903 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157484.72 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 15748.47 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051955.973237 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141736.25 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 14173.63 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066129.598337 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127562.63 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 12756.26 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078885.860928 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114806.36 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 11480.64 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090366.497259 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103325.73 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 10332.57 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100699.069958 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92993.15 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 9299.32 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109998.385386 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83693.84 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 8369.38 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118367.769272 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75324.45 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 7532.45 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125900.214769 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67792.01 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 6779.20 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132679.415716 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61012.81 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 6101.28 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138780.696569 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54911.53 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 5491.15 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144271.849336 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49420.37 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 4942.04 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149213.886827 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44478.34 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 4447.83 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153661.720569 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40030.50 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 4003.05 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157664.770936 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36027.45 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3602.75 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161267.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32424.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164867.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28824.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168467.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25224.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172067.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21624.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175667.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18024.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179267.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14424.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182867.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10824.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186467.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7224.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190067.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3624.71 sec) 1 day 00:15:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193667.516267 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 24.71 sec) 1 day 00:15:00 [GPUGRID] [rr_sim] 2193692.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964739524.76G/439.78G) 1 day 00:15:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854033.14 1 day 00:15:00 [---] pick_jobs_to_run() start 1 day 00:15:00 [---] pick_jobs_to_run() end 1 day 00:15:00 [---] [rr_sim] end 1 day 00:15:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:15:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:15:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:15:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:15:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:15:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:15:00 [---] [cpu_sched_debug] final job list: 1 day 00:15:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:15:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:15:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:15:00 [---] choose_project(): 87300.000000 1 day 00:15:00 [WUProp@Home] doing RPC to get work 1 day 00:15:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348299.09 cpu 2097992.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097992.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097992.90 1 day 00:15:00 [---] Got 0 tasks 1 day 00:16:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:16:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:16:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:16:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964711934.41G/439.78G) 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193629.49 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 219362.95 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219362.948736 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974266.54 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 197426.65 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416789.602599 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776839.88 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 177683.99 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594473.591075 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599155.90 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 159915.59 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754389.180703 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439240.31 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 143924.03 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898313.211369 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295316.28 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 129531.63 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027844.838968 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165784.65 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 116578.46 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144423.303807 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049206.18 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 104920.62 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249343.922163 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944285.57 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 94428.56 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343772.478682 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849857.01 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 84985.70 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428758.179550 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764871.31 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 76487.13 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505245.310331 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688384.18 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 68838.42 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574083.728034 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619545.76 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 61954.58 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1636038.303967 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557591.18 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 55759.12 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691797.422306 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501832.07 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 50183.21 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741980.628812 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451648.86 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 45164.89 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787145.514667 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406483.97 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 40648.40 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827793.911936 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365835.58 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 36583.56 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864377.469478 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329252.02 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 32925.20 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897302.671267 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296326.82 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 29632.68 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926935.352876 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266694.13 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 26669.41 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953604.766324 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240024.72 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 24002.47 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977607.238428 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216022.25 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 21602.22 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999209.463321 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194420.02 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 19442.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018651.465725 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174978.02 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 17497.80 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036149.267889 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157480.22 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 15748.02 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051897.289836 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141732.20 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 14173.22 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066070.509588 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127558.98 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 12755.90 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078826.407366 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114803.08 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 11480.31 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090306.715365 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103322.77 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 10332.28 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100638.992565 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92990.49 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 9299.05 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109938.042044 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83691.45 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 8369.14 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118307.186576 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75322.30 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 7532.23 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125839.416654 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67790.07 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 6779.01 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132618.423725 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61011.06 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 6101.11 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138719.530089 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54909.96 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 5491.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144210.525816 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49418.96 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 4941.90 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149152.421970 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44477.07 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 4447.71 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153600.128509 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40029.36 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 4002.94 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157603.064394 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36026.42 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3602.64 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161205.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32423.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164805.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28823.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168405.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25223.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2172005.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21623.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175605.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18023.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179205.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14423.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182805.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10823.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186405.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7223.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2190005.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3623.78 sec) 1 day 00:16:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193605.706691 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 23.78 sec) 1 day 00:16:00 [GPUGRID] [rr_sim] 2193629.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964711934.41G/439.78G) 1 day 00:16:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854030.40 1 day 00:16:00 [---] pick_jobs_to_run() start 1 day 00:16:00 [---] pick_jobs_to_run() end 1 day 00:16:00 [---] [rr_sim] end 1 day 00:16:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:16:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:16:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:16:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:16:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:16:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:16:00 [---] [cpu_sched_debug] final job list: 1 day 00:16:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:16:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:16:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:16:00 [---] choose_project(): 87360.000000 1 day 00:16:00 [WUProp@Home] doing RPC to get work 1 day 00:16:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348239.09 cpu 2097932.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097932.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097932.90 1 day 00:16:00 [---] Got 0 tasks 1 day 00:17:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:17:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:17:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:17:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964684344.05G/439.78G) 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193566.75 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 219356.68 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219356.675048 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974210.08 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 197421.01 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416777.682591 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776789.07 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 177678.91 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594456.589380 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599110.16 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 159911.02 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754367.605490 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439199.14 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 143919.91 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898287.519988 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295279.23 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 129527.92 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027815.443037 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165751.31 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 116575.13 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144390.573781 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049176.18 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 104917.62 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249308.191451 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944258.56 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 94425.86 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343734.047354 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849832.70 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 84983.27 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428717.317666 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764849.43 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 76484.94 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505202.260948 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688364.49 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 68836.45 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1574038.709901 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619528.04 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 61952.80 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635991.513958 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557575.24 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 55757.52 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691749.037610 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501817.71 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 50181.77 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741930.808897 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451635.94 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 45163.59 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787094.403055 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406472.35 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 40647.23 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827741.637798 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365825.11 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 36582.51 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864324.149066 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329242.60 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 32924.26 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897248.409207 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296318.34 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 29631.83 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926880.243334 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266686.51 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 26668.65 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953548.894049 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240017.86 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 24001.79 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977550.679692 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216016.07 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 21601.61 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999152.286770 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194414.46 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 19441.45 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018593.733141 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174973.02 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 17497.30 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036091.034875 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157475.72 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 15747.57 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051838.606435 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141728.14 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 14172.81 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2066011.420839 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127555.33 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 12755.53 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078766.953803 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114799.80 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 11479.98 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090246.933471 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103319.82 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 10331.98 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100578.915172 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92987.84 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 9298.78 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109877.698702 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83689.05 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 8368.91 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118246.603880 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75320.15 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 7532.01 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125778.618540 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67788.13 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 6778.81 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132557.431734 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61009.32 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 6100.93 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138658.363608 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54908.39 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 5490.84 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144149.202295 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49417.55 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 4941.75 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149090.957113 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44475.79 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 4447.58 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153538.536450 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40028.21 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 4002.82 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157541.357853 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36025.39 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3602.54 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161143.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32422.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164743.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28822.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168343.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25222.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171943.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21622.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175543.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18022.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179143.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14422.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182743.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10822.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186343.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7222.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189943.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3622.85 sec) 1 day 00:17:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193543.897115 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 22.85 sec) 1 day 00:17:00 [GPUGRID] [rr_sim] 2193566.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964684344.05G/439.78G) 1 day 00:17:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854027.66 1 day 00:17:00 [---] pick_jobs_to_run() start 1 day 00:17:00 [---] pick_jobs_to_run() end 1 day 00:17:00 [---] [rr_sim] end 1 day 00:17:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:17:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:17:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:17:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:17:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:17:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:17:00 [---] [cpu_sched_debug] final job list: 1 day 00:17:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:17:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:17:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:17:00 [---] choose_project(): 87420.000000 1 day 00:17:00 [WUProp@Home] doing RPC to get work 1 day 00:17:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348179.09 cpu 2097872.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097872.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097872.90 1 day 00:17:00 [---] Got 0 tasks 1 day 00:18:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:18:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:18:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:18:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964656753.69G/439.78G) 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193504.01 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 219350.40 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219350.401360 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974153.61 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 197415.36 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416765.762583 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776738.25 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 177673.83 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594439.587685 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599064.43 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 159906.44 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754346.030276 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439157.98 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 143915.80 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898261.828608 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295242.18 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 129524.22 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027786.047107 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165717.97 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 116571.80 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144357.843756 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049146.17 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 104914.62 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249272.460740 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944231.55 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 94423.16 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343695.616025 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849808.40 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 84980.84 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428676.455782 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764827.56 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 76482.76 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505159.211564 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688344.80 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 68834.48 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573993.691767 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619510.32 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 61951.03 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635944.723950 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557559.29 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 55755.93 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691700.652915 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501803.36 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 50180.34 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741880.988983 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451623.02 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 45162.30 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1787043.291444 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406460.72 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 40646.07 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827689.363659 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365814.65 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 36581.46 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864270.828653 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329233.18 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 32923.32 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897194.147147 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296309.87 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 29630.99 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926825.133792 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266678.88 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 26667.89 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953493.021773 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240010.99 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 24001.10 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977494.120955 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216009.89 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 21600.99 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999095.110219 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194408.90 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 19440.89 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018536.000557 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174968.01 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 17496.80 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2036032.801861 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157471.21 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 15747.12 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051779.923034 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141724.09 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 14172.41 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065952.332090 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127551.68 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 12755.17 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078707.500241 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114796.51 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 11479.65 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090187.151577 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103316.86 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 10331.69 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100518.837779 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92985.18 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 9298.52 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109817.355360 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83686.66 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 8368.67 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118186.021184 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75317.99 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 7531.80 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125717.820425 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67786.19 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 6778.62 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132496.439742 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61007.57 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 6100.76 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138597.197128 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54906.82 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 5490.68 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144087.878774 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49416.13 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 4941.61 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2149029.492257 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44474.52 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 4447.45 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153476.944391 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40027.07 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 4002.71 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157479.651311 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36024.36 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3602.44 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161082.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32421.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164682.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28821.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168282.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25221.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171882.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21621.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175482.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18021.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179082.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14421.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182682.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10821.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186282.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7221.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189882.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3621.93 sec) 1 day 00:18:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193482.087540 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21.93 sec) 1 day 00:18:00 [GPUGRID] [rr_sim] 2193504.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964656753.69G/439.78G) 1 day 00:18:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854024.93 1 day 00:18:00 [---] pick_jobs_to_run() start 1 day 00:18:00 [---] pick_jobs_to_run() end 1 day 00:18:00 [---] [rr_sim] end 1 day 00:18:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:18:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:18:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:18:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:18:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:18:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:18:00 [---] [cpu_sched_debug] final job list: 1 day 00:18:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:18:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:18:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:18:00 [---] choose_project(): 87480.000000 1 day 00:18:00 [WUProp@Home] doing RPC to get work 1 day 00:18:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348119.09 cpu 2097812.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097812.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097812.90 1 day 00:18:00 [---] Got 0 tasks 1 day 00:19:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:19:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:19:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:19:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964629163.34G/439.78G) 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193441.28 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 219344.13 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219344.127671 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974097.15 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 197409.71 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416753.842576 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776687.43 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 177668.74 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594422.585990 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1599018.69 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 159901.87 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754324.455062 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439116.82 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 143911.68 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898236.137227 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295205.14 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 129520.51 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027756.651176 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165684.63 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 116568.46 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144325.113730 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049116.16 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 104911.62 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249236.730028 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944204.55 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 94420.45 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343657.184697 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849784.09 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 84978.41 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428635.593898 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764805.68 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 76480.57 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505116.162180 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688325.11 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 68832.51 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573948.673633 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619492.60 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 61949.26 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635897.933941 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557543.34 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 55754.33 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691652.268219 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501789.01 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 50178.90 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741831.169068 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451610.11 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 45161.01 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786992.179833 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406449.10 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 40644.91 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827637.089521 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365804.19 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 36580.42 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864217.508240 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329223.77 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 32922.38 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897139.885088 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296301.39 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 29630.14 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926770.024250 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266671.25 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 26667.13 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953437.149497 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 240004.13 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 24000.41 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977437.562218 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 216003.71 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 21600.37 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1999037.933668 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194403.34 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 19440.33 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018478.267973 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174963.01 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 17496.30 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035974.568847 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157466.71 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 15746.67 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051721.239633 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141720.04 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 14172.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065893.243342 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127548.03 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 12754.80 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078648.046679 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114793.23 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 11479.32 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090127.369682 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103313.91 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 10331.39 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100458.760385 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92982.52 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 9298.25 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109757.012018 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83684.26 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 8368.43 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118125.438488 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75315.84 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 7531.58 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125657.022311 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67784.25 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 6778.43 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132435.447751 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61005.83 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 6100.58 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138536.030647 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54905.25 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 5490.52 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2144026.555254 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49414.72 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 4941.47 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148968.027400 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44473.25 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 4447.32 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153415.352331 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40025.92 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 4002.59 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157417.944770 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36023.33 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3602.33 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2161020.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32421.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164620.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28821.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168220.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25221.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171820.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21621.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175420.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18021.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2179020.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14421.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182620.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10821.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186220.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7221.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189820.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3621.00 sec) 1 day 00:19:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193420.277964 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21.00 sec) 1 day 00:19:00 [GPUGRID] [rr_sim] 2193441.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964629163.34G/439.78G) 1 day 00:19:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854022.19 1 day 00:19:00 [---] pick_jobs_to_run() start 1 day 00:19:00 [---] pick_jobs_to_run() end 1 day 00:19:00 [---] [rr_sim] end 1 day 00:19:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:19:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:19:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:19:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:19:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:19:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:19:00 [---] [cpu_sched_debug] final job list: 1 day 00:19:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:19:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:19:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:19:00 [---] choose_project(): 87540.000000 1 day 00:19:00 [WUProp@Home] doing RPC to get work 1 day 00:19:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 348059.09 cpu 2097752.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097752.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097752.90 1 day 00:19:00 [---] Got 0 tasks 1 day 00:20:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:20:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:20:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:20:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964601572.98G/439.78G) 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193378.54 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 219337.85 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219337.853983 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1974040.69 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 197404.07 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416741.922568 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776636.62 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 177663.66 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594405.584294 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598972.96 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 159897.30 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754302.879848 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439075.66 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 143907.57 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898210.445847 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295168.09 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 129516.81 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027727.255245 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165651.28 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 116565.13 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144292.383704 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049086.16 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 104908.62 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249200.999317 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944177.54 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 94417.75 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343618.753368 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849759.79 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 84975.98 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428594.732014 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764783.81 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 76478.38 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505073.112796 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688305.43 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 68830.54 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573903.655500 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619474.88 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 61947.49 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635851.143933 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557527.40 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 55752.74 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691603.883523 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501774.66 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 50177.47 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741781.349154 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451597.19 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 45159.72 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786941.068222 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406437.47 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 40643.75 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827584.815383 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365793.72 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 36579.37 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864164.187828 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329214.35 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 32921.44 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897085.623028 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296292.92 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 29629.29 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926714.914708 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266663.63 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 26666.36 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953381.277221 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239997.26 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 23999.73 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977381.003482 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215997.54 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 21599.75 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998980.757117 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194397.78 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 19439.78 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018420.535388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174958.00 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 17495.80 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035916.335833 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157462.20 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 15746.22 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051662.556233 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141715.98 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 14171.60 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065834.154593 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127544.39 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 12754.44 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078588.593116 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114789.95 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 11478.99 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090067.587788 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103310.95 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 10331.10 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100398.682992 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92979.86 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 9297.99 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109696.668676 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83681.87 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 8368.19 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118064.855792 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75313.68 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 7531.37 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125596.224196 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67782.32 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 6778.23 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132374.455759 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61004.08 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 6100.41 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138474.864167 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54903.68 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 5490.37 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143965.231733 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49413.31 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 4941.33 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148906.562543 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44471.98 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 4447.20 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153353.760272 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40024.78 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 4002.48 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157356.238228 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36022.30 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3602.23 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160958.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32420.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164558.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28820.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168158.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25220.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171758.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21620.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175358.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18020.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178958.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14420.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182558.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10820.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186158.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7220.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189758.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3620.07 sec) 1 day 00:20:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193358.468388 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 20.07 sec) 1 day 00:20:00 [GPUGRID] [rr_sim] 2193378.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964601572.98G/439.78G) 1 day 00:20:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854019.45 1 day 00:20:00 [---] pick_jobs_to_run() start 1 day 00:20:00 [---] pick_jobs_to_run() end 1 day 00:20:00 [---] [rr_sim] end 1 day 00:20:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:20:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:20:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:20:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:20:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:20:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:20:00 [---] [cpu_sched_debug] final job list: 1 day 00:20:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:20:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:20:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:20:00 [---] choose_project(): 87600.000000 1 day 00:20:00 [WUProp@Home] doing RPC to get work 1 day 00:20:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347999.09 cpu 2097692.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097692.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097692.90 1 day 00:20:00 [---] Got 0 tasks 1 day 00:21:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:21:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:21:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:21:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964573982.62G/439.78G) 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193315.80 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 219331.58 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219331.580295 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973984.22 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 197398.42 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416730.002560 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776585.80 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 177658.58 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594388.582599 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598927.22 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 159892.72 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754281.304634 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1439034.50 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 143903.45 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898184.754466 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295131.05 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 129513.10 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027697.859314 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165617.94 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 116561.79 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144259.653678 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049056.15 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 104905.61 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249165.268605 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944150.53 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 94415.05 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343580.322040 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849735.48 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 84973.55 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428553.870131 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764761.93 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 76476.19 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1505030.063412 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688285.74 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 68828.57 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573858.637366 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619457.17 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 61945.72 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635804.353925 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557511.45 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 55751.14 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691555.498827 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501760.30 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 50176.03 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741731.529239 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451584.27 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 45158.43 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786889.956610 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406425.85 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 40642.58 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827532.541244 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365783.26 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 36578.33 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864110.867415 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329204.94 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 32920.49 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1897031.360968 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296284.44 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 29628.44 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926659.805166 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266656.00 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 26665.60 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953325.404945 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239990.40 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 23999.04 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977324.444745 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215991.36 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 21599.14 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998923.580566 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194392.22 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 19439.22 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018362.802804 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174953.00 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 17495.30 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035858.102819 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157457.70 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 15745.77 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051603.872832 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141711.93 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 14171.19 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065775.065844 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127540.74 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 12754.07 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078529.139554 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114786.66 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 11478.67 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2090007.805894 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103308.00 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 10330.80 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100338.605599 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92977.20 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 9297.72 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109636.325334 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83679.48 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 8367.95 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2118004.273096 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75311.53 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 7531.15 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125535.426081 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67780.38 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 6778.04 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132313.463768 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61002.34 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 6100.23 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138413.697686 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54902.11 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 5490.21 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143903.908213 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49411.89 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 4941.19 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148845.097686 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44470.71 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 4447.07 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153292.168213 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40023.63 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 4002.36 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157294.531686 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36021.27 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3602.13 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160896.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32419.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164496.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28819.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168096.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25219.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171696.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21619.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175296.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18019.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178896.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14419.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182496.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10819.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186096.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7219.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189696.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3619.14 sec) 1 day 00:21:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193296.658813 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 19.14 sec) 1 day 00:21:00 [GPUGRID] [rr_sim] 2193315.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964573982.62G/439.78G) 1 day 00:21:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854016.72 1 day 00:21:00 [---] pick_jobs_to_run() start 1 day 00:21:00 [---] pick_jobs_to_run() end 1 day 00:21:00 [---] [rr_sim] end 1 day 00:21:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:21:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:21:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:21:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:21:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:21:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:21:00 [---] [cpu_sched_debug] final job list: 1 day 00:21:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:21:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:21:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:21:00 [---] choose_project(): 87660.000000 1 day 00:21:00 [WUProp@Home] doing RPC to get work 1 day 00:21:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347939.09 cpu 2097632.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097632.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097632.90 1 day 00:21:00 [---] Got 0 tasks 1 day 00:24:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:24:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:24:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:24:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964546392.27G/439.78G) 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193253.07 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 219325.31 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219325.306607 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973927.76 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 197392.78 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416718.082553 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776534.98 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 177653.50 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594371.580904 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598881.49 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 159888.15 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754259.729421 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438993.34 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 143899.33 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898159.063085 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295094.00 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 129509.40 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027668.463384 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165584.60 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 116558.46 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144226.923652 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1049026.14 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 104902.61 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249129.537894 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944123.53 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 94412.35 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343541.890711 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849711.18 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 84971.12 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428513.008247 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764740.06 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 76474.01 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504987.014029 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688266.05 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 68826.61 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573813.619233 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619439.45 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 61943.94 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635757.563916 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557495.50 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 55749.55 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691507.114131 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501745.95 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 50174.60 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741681.709325 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451571.36 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 45157.14 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786838.844999 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406414.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 40641.42 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827480.267106 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365772.80 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 36577.28 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864057.547002 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329195.52 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 32919.55 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896977.098909 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296275.97 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 29627.60 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926604.695625 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266648.37 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 26664.84 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953269.532669 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239983.53 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 23998.35 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977267.886009 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215985.18 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 21598.52 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998866.404015 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194386.66 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 19438.67 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018305.070220 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174948.00 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 17494.80 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035799.869805 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157453.20 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 15745.32 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051545.189431 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141707.88 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 14170.79 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065715.977095 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127537.09 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 12753.71 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078469.685992 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114783.38 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 11478.34 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089948.023999 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103305.04 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 10330.50 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100278.528206 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92974.54 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 9297.45 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109575.981992 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83677.08 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 8367.71 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117943.690400 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75309.38 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 7530.94 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125474.627967 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67778.44 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 6777.84 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132252.471777 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 61000.59 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 6100.06 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138352.531206 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54900.53 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 5490.05 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143842.584692 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49410.48 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 4941.05 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148783.632830 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44469.43 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 4446.94 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153230.576153 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40022.49 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 4002.25 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157232.825145 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36020.24 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3602.02 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160834.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32418.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164434.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28818.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2168034.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25218.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171634.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21618.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175234.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18018.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178834.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14418.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182434.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10818.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2186034.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7218.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189634.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3618.22 sec) 1 day 00:24:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193234.849237 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18.22 sec) 1 day 00:24:00 [GPUGRID] [rr_sim] 2193253.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964546392.27G/439.78G) 1 day 00:24:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854133.98 1 day 00:24:00 [---] pick_jobs_to_run() start 1 day 00:24:00 [---] pick_jobs_to_run() end 1 day 00:24:00 [---] [rr_sim] end 1 day 00:24:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:24:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:24:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:24:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:24:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:24:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:24:00 [---] [cpu_sched_debug] final job list: 1 day 00:24:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:24:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:24:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:24:00 [---] choose_project(): 87840.000000 1 day 00:24:00 [WUProp@Home] doing RPC to get work 1 day 00:24:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347759.09 cpu 2097572.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097572.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097572.90 1 day 00:24:00 [---] Got 0 tasks 1 day 00:25:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:25:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:25:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:25:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964518801.91G/439.78G) 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193190.33 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 219319.03 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219319.032919 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973871.30 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 197387.13 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416706.162545 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776484.17 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 177648.42 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594354.579209 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598835.75 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 159883.57 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754238.154207 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438952.17 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 143895.22 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898133.371705 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295056.96 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 129505.70 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027639.067453 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165551.26 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 116555.13 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144194.193626 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048996.14 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 104899.61 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249093.807182 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944096.52 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 94409.65 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343503.459382 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849686.87 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 84968.69 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428472.146363 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764718.18 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 76471.82 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504943.964645 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688246.36 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 68824.64 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573768.601099 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619421.73 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 61942.17 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635710.773908 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557479.56 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 55747.96 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691458.729435 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501731.60 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 50173.16 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741631.889410 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451558.44 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 45155.84 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786787.733388 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406402.60 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 40640.26 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827427.992968 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365762.34 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 36576.23 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1864004.226589 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329186.10 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 32918.61 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896922.836849 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296267.49 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 29626.75 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926549.586083 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266640.74 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 26664.07 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953213.660393 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239976.67 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 23997.67 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977211.327272 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215979.00 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 21597.90 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998809.227463 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194381.10 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 19438.11 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018247.337636 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174942.99 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 17494.30 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035741.636791 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157448.69 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 15744.87 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051486.506030 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141703.82 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 14170.38 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065656.888346 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127533.44 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 12753.34 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078410.232430 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114780.10 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 11478.01 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089888.242105 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103302.09 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 10330.21 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100218.450813 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92971.88 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 9297.19 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109515.638650 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83674.69 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 8367.47 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117883.107704 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75307.22 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 7530.72 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125413.829852 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67776.50 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 6777.65 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132191.479785 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60998.85 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 6099.88 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138291.364725 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54898.96 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 5489.90 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143781.261171 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49409.07 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 4940.91 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148722.167973 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44468.16 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 4446.82 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153168.984094 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40021.35 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 4002.13 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157171.118603 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36019.21 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3601.92 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160773.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32417.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164373.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28817.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167973.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25217.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171573.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21617.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175173.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18017.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178773.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14417.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182373.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10817.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185973.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7217.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189573.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3617.29 sec) 1 day 00:25:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193173.039661 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17.29 sec) 1 day 00:25:00 [GPUGRID] [rr_sim] 2193190.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964518801.91G/439.78G) 1 day 00:25:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854131.24 1 day 00:25:00 [---] pick_jobs_to_run() start 1 day 00:25:00 [---] pick_jobs_to_run() end 1 day 00:25:00 [---] [rr_sim] end 1 day 00:25:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:25:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:25:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:25:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:25:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:25:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:25:00 [---] [cpu_sched_debug] final job list: 1 day 00:25:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:25:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:25:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:25:00 [---] choose_project(): 87900.000000 1 day 00:25:00 [WUProp@Home] doing RPC to get work 1 day 00:25:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347699.09 cpu 2097512.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097512.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097512.90 1 day 00:25:00 [---] Got 0 tasks 1 day 00:26:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:26:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:26:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:26:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964491211.56G/439.78G) 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193127.59 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 219312.76 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219312.759230 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973814.83 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 197381.48 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416694.242538 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776433.35 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 177643.33 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594337.577514 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598790.01 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 159879.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754216.578993 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438911.01 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 143891.10 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898107.680324 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1295019.91 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 129501.99 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027609.671522 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165517.92 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 116551.79 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144161.463600 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048966.13 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 104896.61 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249058.076470 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944069.52 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 94406.95 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343465.028054 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849662.56 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 84966.26 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428431.284479 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764696.31 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 76469.63 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504900.915261 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688226.68 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 68822.67 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573723.582965 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619404.01 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 61940.40 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635663.983899 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557463.61 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 55746.36 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691410.344740 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501717.25 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 50171.72 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741582.069496 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451545.52 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 45154.55 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786736.621777 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406390.97 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 40639.10 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827375.718829 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365751.87 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 36575.19 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863950.906177 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329176.69 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 32917.67 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896868.574789 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296259.02 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 29625.90 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926494.476541 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266633.12 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 26663.31 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953157.788117 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239969.80 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 23996.98 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977154.768536 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215972.82 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 21597.28 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998752.050912 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194375.54 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 19437.55 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018189.605051 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174937.99 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 17493.80 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035683.403777 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157444.19 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 15744.42 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051427.822629 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141699.77 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 14169.98 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065597.799597 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127529.79 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 12752.98 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078350.778867 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114776.81 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 11477.68 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089828.460211 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103299.13 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 10329.91 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100158.373420 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92969.22 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 9296.92 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109455.295308 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83672.30 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 8367.23 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117822.525008 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75305.07 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 7530.51 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125353.031737 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67774.56 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 6777.46 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132130.487794 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60997.10 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 6099.71 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138230.198245 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54897.39 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 5489.74 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143719.937651 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49407.65 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 4940.77 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148660.703116 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44466.89 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 4446.69 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153107.392035 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40020.20 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 4002.02 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157109.412062 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36018.18 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3601.82 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160711.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32416.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164311.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28816.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167911.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25216.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171511.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21616.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175111.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18016.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178711.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14416.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182311.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10816.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185911.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7216.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189511.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3616.36 sec) 1 day 00:26:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193111.230086 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 16.36 sec) 1 day 00:26:00 [GPUGRID] [rr_sim] 2193127.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964491211.56G/439.78G) 1 day 00:26:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854128.51 1 day 00:26:00 [---] pick_jobs_to_run() start 1 day 00:26:00 [---] pick_jobs_to_run() end 1 day 00:26:00 [---] [rr_sim] end 1 day 00:26:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:26:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:26:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:26:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:26:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:26:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:26:00 [---] [cpu_sched_debug] final job list: 1 day 00:26:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:26:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:26:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:26:00 [---] choose_project(): 87960.000000 1 day 00:26:00 [WUProp@Home] doing RPC to get work 1 day 00:26:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347639.09 cpu 2097452.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097452.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097452.90 1 day 00:26:00 [---] Got 0 tasks 1 day 00:27:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:27:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:27:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:27:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964463621.20G/439.78G) 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193064.86 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 219306.49 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219306.485542 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973758.37 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 197375.84 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416682.322530 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776382.53 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 177638.25 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594320.575819 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598744.28 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 159874.43 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754195.003779 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438869.85 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 143886.99 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898081.988943 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294982.87 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 129498.29 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027580.275591 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165484.58 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 116548.46 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144128.733574 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048936.12 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 104893.61 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1249022.345759 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944042.51 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 94404.25 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343426.596725 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849638.26 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 84963.83 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428390.422595 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764674.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 76467.44 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504857.865877 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688206.99 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 68820.70 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573678.564832 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619386.29 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 61938.63 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635617.193891 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557447.66 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 55744.77 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691361.960044 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501702.90 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 50170.29 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741532.249581 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451532.61 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 45153.26 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786685.510165 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406379.35 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 40637.93 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827323.444691 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365741.41 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 36574.14 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863897.585764 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329167.27 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 32916.73 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896814.312730 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296250.54 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 29625.05 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926439.366999 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266625.49 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 26662.55 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953101.915841 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239962.94 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 23996.29 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977098.209799 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215966.65 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 21596.66 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998694.874361 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194369.98 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 19437.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018131.872467 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174932.98 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 17493.30 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035625.170763 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157439.68 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 15743.97 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051369.139228 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141695.72 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 14169.57 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065538.710848 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127526.14 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 12752.61 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078291.325305 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114773.53 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 11477.35 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089768.678317 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103296.18 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 10329.62 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100098.296027 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92966.56 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 9296.66 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109394.951966 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83669.90 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 8366.99 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117761.942312 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75302.91 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 7530.29 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125292.233623 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67772.62 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 6777.26 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132069.495803 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60995.36 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 6099.54 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138169.031765 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54895.82 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 5489.58 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143658.614130 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49406.24 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 4940.62 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148599.238259 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44465.62 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 4446.56 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2153045.799975 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40019.06 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 4001.91 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2157047.705520 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36017.15 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3601.71 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160649.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32415.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164249.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28815.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167849.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25215.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171449.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21615.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2175049.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18015.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178649.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14415.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182249.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10815.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185849.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7215.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189449.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3615.43 sec) 1 day 00:27:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2193049.420510 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 15.43 sec) 1 day 00:27:00 [GPUGRID] [rr_sim] 2193064.86: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964463621.20G/439.78G) 1 day 00:27:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854125.77 1 day 00:27:00 [---] pick_jobs_to_run() start 1 day 00:27:00 [---] pick_jobs_to_run() end 1 day 00:27:00 [---] [rr_sim] end 1 day 00:27:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:27:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:27:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:27:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:27:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:27:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:27:00 [---] [cpu_sched_debug] final job list: 1 day 00:27:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:27:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:27:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:27:00 [---] choose_project(): 88020.000000 1 day 00:27:00 [WUProp@Home] doing RPC to get work 1 day 00:27:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347579.09 cpu 2097392.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097392.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097392.90 1 day 00:27:00 [---] Got 0 tasks 1 day 00:28:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:28:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:28:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:28:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964436030.84G/439.78G) 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2193002.12 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 219300.21 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219300.211854 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973701.91 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 197370.19 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416670.402522 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776331.72 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 177633.17 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594303.574124 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598698.54 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 159869.85 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754173.428566 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438828.69 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 143882.87 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898056.297563 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294945.82 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 129494.58 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027550.879660 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165451.24 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 116545.12 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144096.003548 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048906.11 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 104890.61 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248986.615047 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 944015.50 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 94401.55 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343388.165397 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849613.95 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 84961.40 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428349.560711 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764652.56 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 76465.26 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504814.816494 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688187.30 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 68818.73 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573633.546698 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619368.57 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 61936.86 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635570.403882 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557431.71 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 55743.17 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691313.575348 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501688.54 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 50168.85 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741482.429667 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451519.69 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 45151.97 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786634.398554 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406367.72 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 40636.77 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827271.170553 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365730.95 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 36573.09 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863844.265351 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329157.85 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 32915.79 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896760.050670 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296242.07 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 29624.21 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926384.257457 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266617.86 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 26661.79 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1953046.043565 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239956.07 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 23995.61 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1977041.651062 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215960.47 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 21596.05 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998637.697810 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194364.42 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 19436.44 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018074.139883 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174927.98 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 17492.80 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035566.937749 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157435.18 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 15743.52 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051310.455828 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141691.66 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 14169.17 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065479.622099 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127522.50 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 12752.25 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078231.871743 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114770.25 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 11477.02 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089708.896422 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103293.22 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 10329.32 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2100038.218634 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92963.90 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 9296.39 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109334.608625 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83667.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 8366.75 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117701.359616 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75300.76 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 7530.08 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125231.435508 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67770.68 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 6777.07 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2132008.503811 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60993.61 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 6099.36 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138107.865284 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54894.25 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 5489.43 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143597.290610 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49404.83 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 4940.48 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148537.773402 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44464.35 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 4446.43 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152984.207916 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40017.91 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 4001.79 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156985.998978 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36016.12 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3601.61 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160587.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32414.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164187.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28814.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167787.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25214.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171387.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21614.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174987.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18014.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178587.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14414.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182187.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10814.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185787.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7214.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189387.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3614.51 sec) 1 day 00:28:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192987.610934 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14.51 sec) 1 day 00:28:00 [GPUGRID] [rr_sim] 2193002.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964436030.84G/439.78G) 1 day 00:28:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854123.03 1 day 00:28:00 [---] pick_jobs_to_run() start 1 day 00:28:00 [---] pick_jobs_to_run() end 1 day 00:28:00 [---] [rr_sim] end 1 day 00:28:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:28:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:28:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:28:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:28:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:28:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:28:00 [---] [cpu_sched_debug] final job list: 1 day 00:28:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:28:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:28:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:28:00 [---] choose_project(): 88080.000000 1 day 00:28:00 [WUProp@Home] doing RPC to get work 1 day 00:28:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347519.09 cpu 2097332.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097332.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097332.90 1 day 00:28:00 [---] Got 0 tasks 1 day 00:30:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:30:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:30:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:30:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964408440.49G/439.78G) 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192939.38 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 219293.94 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219293.938166 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973645.44 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 197364.54 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416658.482515 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776280.90 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 177628.09 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594286.572429 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598652.81 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 159865.28 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754151.853352 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438787.53 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 143878.75 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898030.606182 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294908.78 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 129490.88 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027521.483730 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165417.90 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 116541.79 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144063.273522 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048876.11 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 104887.61 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248950.884336 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943988.50 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 94398.85 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343349.734068 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849589.65 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 84958.96 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428308.698827 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764630.68 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 76463.07 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504771.767110 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688167.61 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 68816.76 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573588.528564 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619350.85 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 61935.09 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635523.613874 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557415.77 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 55741.58 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691265.190652 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501674.19 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 50167.42 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741432.609752 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451506.77 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 45150.68 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786583.286943 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406356.09 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 40635.61 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827218.896414 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365720.49 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 36572.05 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863790.944939 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329148.44 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 32914.84 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896705.788610 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296233.59 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 29623.36 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926329.147915 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266610.23 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 26661.02 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952990.171289 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239949.21 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 23994.92 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976985.092326 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215954.29 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 21595.43 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998580.521259 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194358.86 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 19435.89 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2018016.407299 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174922.97 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 17492.30 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035508.704735 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157430.68 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 15743.07 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051251.772427 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141687.61 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 14168.76 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065420.533350 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127518.85 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 12751.88 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078172.418180 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114766.96 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 11476.70 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089649.114528 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103290.27 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 10329.03 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099978.141241 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92961.24 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 9296.12 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109274.265283 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83665.12 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 8366.51 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117640.776920 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75298.60 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 7529.86 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125170.637394 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67768.74 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 6776.87 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131947.511820 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60991.87 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 6099.19 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2138046.698804 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54892.68 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 5489.27 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143535.967089 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49403.41 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 4940.34 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148476.308546 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44463.07 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 4446.31 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152922.615857 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40016.77 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 4001.68 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156924.292437 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36015.09 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3601.51 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160525.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32413.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164125.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28813.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167725.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25213.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171325.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21613.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174925.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18013.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178525.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14413.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182125.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10813.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185725.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7213.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189325.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3613.58 sec) 1 day 00:30:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192925.801359 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 13.58 sec) 1 day 00:30:00 [GPUGRID] [rr_sim] 2192939.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964408440.49G/439.78G) 1 day 00:30:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854180.29 1 day 00:30:00 [---] pick_jobs_to_run() start 1 day 00:30:00 [---] pick_jobs_to_run() end 1 day 00:30:00 [---] [rr_sim] end 1 day 00:30:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:30:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:30:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:30:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:30:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:30:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:30:00 [---] [cpu_sched_debug] final job list: 1 day 00:30:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:30:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:30:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:31:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:31:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:31:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:31:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964380850.13G/439.78G) 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192876.64 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 219287.66 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219287.664477 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973588.98 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 197358.90 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416646.562507 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776230.08 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 177623.01 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594269.570734 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598607.07 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 159860.71 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754130.278138 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438746.37 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 143874.64 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 898004.914802 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294871.73 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 129487.17 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027492.087799 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165384.56 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 116538.46 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1144030.543496 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048846.10 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 104884.61 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248915.153624 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943961.49 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 94396.15 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343311.302739 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849565.34 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 84956.53 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428267.836943 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764608.81 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 76460.88 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504728.717726 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688147.93 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 68814.79 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573543.510431 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619333.13 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 61933.31 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635476.823865 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557399.82 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 55739.98 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691216.805956 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501659.84 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 50165.98 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741382.789838 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451493.85 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 45149.39 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786532.175332 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406344.47 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 40634.45 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827166.622276 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365710.02 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 36571.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863737.624526 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329139.02 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 32913.90 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896651.526551 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296225.12 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 29622.51 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926274.038373 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266602.61 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 26660.26 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952934.299013 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239942.35 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 23994.23 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976928.533589 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215948.11 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 21594.81 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998523.344708 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194353.30 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 19435.33 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017958.674715 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174917.97 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 17491.80 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035450.471721 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157426.17 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 15742.62 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051193.089026 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141683.56 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 14168.36 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065361.444601 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127515.20 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 12751.52 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078112.964618 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114763.68 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 11476.37 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089589.332634 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103287.31 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 10328.73 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099918.063848 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92958.58 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 9295.86 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109213.921941 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83662.72 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 8366.27 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117580.194224 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75296.45 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 7529.65 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125109.839279 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67766.81 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 6776.68 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131886.519829 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60990.12 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 6099.01 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137985.532323 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54891.11 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 5489.11 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143474.643568 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49402.00 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 4940.20 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148414.843689 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44461.80 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 4446.18 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152861.023797 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40015.62 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 4001.56 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156862.585895 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36014.06 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3601.41 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160463.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32412.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164063.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28812.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167663.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25212.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171263.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21612.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174863.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18012.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178463.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14412.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182063.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10812.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185663.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7212.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189263.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3612.65 sec) 1 day 00:31:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192863.991783 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 12.65 sec) 1 day 00:31:00 [GPUGRID] [rr_sim] 2192876.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964380850.13G/439.78G) 1 day 00:31:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854177.56 1 day 00:31:00 [---] pick_jobs_to_run() start 1 day 00:31:00 [---] pick_jobs_to_run() end 1 day 00:31:00 [---] [rr_sim] end 1 day 00:31:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:31:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:31:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:31:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:31:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:31:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:31:00 [---] [cpu_sched_debug] final job list: 1 day 00:31:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:31:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:31:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:31:00 [---] choose_project(): 88260.000000 1 day 00:31:00 [WUProp@Home] doing RPC to get work 1 day 00:31:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347339.09 cpu 2097212.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097212.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097212.90 1 day 00:31:00 [---] Got 0 tasks 1 day 00:32:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:32:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:32:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:32:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964353259.78G/439.78G) 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192813.91 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 219281.39 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219281.390789 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973532.52 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 197353.25 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416634.642500 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776179.27 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 177617.93 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594252.569039 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598561.34 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 159856.13 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754108.702924 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438705.20 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 143870.52 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897979.223421 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294834.68 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 129483.47 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027462.691868 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165351.22 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 116535.12 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143997.813471 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048816.09 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 104881.61 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248879.422913 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943934.48 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 94393.45 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343272.871411 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849541.04 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 84954.10 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428226.975059 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764586.93 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 76458.69 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504685.668342 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688128.24 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 68812.82 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573498.492297 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619315.42 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 61931.54 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635430.033857 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557383.87 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 55738.39 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691168.421260 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501645.49 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 50164.55 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741332.969924 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451480.94 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 45148.09 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786481.063720 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406332.84 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 40633.28 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827114.348138 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365699.56 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 36569.96 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863684.304113 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329129.60 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 32912.96 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896597.264491 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296216.64 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 29621.66 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926218.928831 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266594.98 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 26659.50 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952878.426737 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239935.48 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 23993.55 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976871.974853 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215941.93 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 21594.19 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998466.168157 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194347.74 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 19434.77 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017900.942130 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174912.97 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 17491.30 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035392.238707 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157421.67 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 15742.17 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051134.405625 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141679.50 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 14167.95 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065302.355852 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127511.55 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 12751.16 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2078053.511056 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114760.40 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 11476.04 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089529.550740 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103284.36 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 10328.44 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099857.986455 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92955.92 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 9295.59 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109153.578599 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83660.33 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 8366.03 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117519.611528 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75294.30 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 7529.43 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2125049.041164 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67764.87 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 6776.49 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131825.527837 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60988.38 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 6098.84 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137924.365843 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54889.54 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 5488.95 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143413.320048 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49400.59 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 4940.06 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148353.378832 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44460.53 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 4446.05 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152799.431738 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40014.48 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 4001.45 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156800.879354 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36013.03 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3601.30 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160402.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32411.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2164002.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28811.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167602.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25211.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171202.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21611.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174802.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18011.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178402.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14411.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2182002.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10811.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185602.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7211.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189202.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3611.73 sec) 1 day 00:32:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192802.182207 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 11.73 sec) 1 day 00:32:00 [GPUGRID] [rr_sim] 2192813.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964353259.78G/439.78G) 1 day 00:32:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854174.82 1 day 00:32:00 [---] pick_jobs_to_run() start 1 day 00:32:00 [---] pick_jobs_to_run() end 1 day 00:32:00 [---] [rr_sim] end 1 day 00:32:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:32:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:32:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:32:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:32:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:32:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:32:00 [---] [cpu_sched_debug] final job list: 1 day 00:32:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:32:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:32:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:32:00 [---] choose_project(): 88320.000000 1 day 00:32:00 [WUProp@Home] doing RPC to get work 1 day 00:32:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347279.09 cpu 2097152.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097152.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097152.90 1 day 00:32:00 [---] Got 0 tasks 1 day 00:33:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:33:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:33:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:33:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964325669.42G/439.78G) 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192751.17 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 219275.12 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219275.117101 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973476.05 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 197347.61 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416622.722492 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776128.45 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 177612.84 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594235.567344 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598515.60 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 159851.56 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754087.127710 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438664.04 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 143866.40 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897953.532040 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294797.64 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 129479.76 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027433.295937 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165317.88 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 116531.79 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143965.083445 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048786.09 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 104878.61 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248843.692201 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943907.48 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 94390.75 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343234.440082 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849516.73 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 84951.67 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428186.113175 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764565.06 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 76456.51 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504642.618958 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688108.55 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 68810.86 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573453.474164 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619297.70 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 61929.77 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635383.243848 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557367.93 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 55736.79 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691120.036564 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501631.13 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 50163.11 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741283.150009 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451468.02 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 45146.80 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786429.952109 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406321.22 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 40632.12 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827062.073999 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365689.10 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 36568.91 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863630.983700 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329120.19 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 32912.02 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896543.002431 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296208.17 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 29620.82 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926163.819289 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266587.35 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 26658.74 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952822.554461 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239928.62 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 23992.86 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976815.416116 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215935.75 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 21593.58 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998408.991606 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194342.18 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 19434.22 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017843.209546 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174907.96 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 17490.80 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035334.005693 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157417.17 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 15741.72 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051075.722224 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141675.45 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 14167.54 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065243.267103 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127507.90 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 12750.79 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077994.057494 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114757.11 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 11475.71 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089469.768845 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103281.40 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 10328.14 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099797.909062 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92953.26 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 9295.33 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109093.235257 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83657.94 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 8365.79 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117459.028832 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75292.14 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 7529.21 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124988.243050 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67762.93 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 6776.29 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131764.535846 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60986.64 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 6098.66 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137863.199362 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54887.97 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 5488.80 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143351.996527 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49399.17 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 4939.92 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148291.913975 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44459.26 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 4445.93 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152737.839679 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40013.33 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 4001.33 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156739.172812 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36012.00 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3601.20 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160340.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32410.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163940.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28810.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167540.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25210.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171140.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21610.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174740.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18010.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178340.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14410.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181940.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10810.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185540.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7210.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189140.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3610.80 sec) 1 day 00:33:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192740.372632 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10.80 sec) 1 day 00:33:00 [GPUGRID] [rr_sim] 2192751.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964325669.42G/439.78G) 1 day 00:33:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854172.08 1 day 00:33:00 [---] pick_jobs_to_run() start 1 day 00:33:00 [---] pick_jobs_to_run() end 1 day 00:33:00 [---] [rr_sim] end 1 day 00:33:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:33:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:33:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:33:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:33:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:33:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:33:00 [---] [cpu_sched_debug] final job list: 1 day 00:33:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:33:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:33:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:33:00 [---] choose_project(): 88380.000000 1 day 00:33:00 [WUProp@Home] doing RPC to get work 1 day 00:33:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347219.09 cpu 2097092.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097092.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097092.90 1 day 00:33:00 [---] Got 0 tasks 1 day 00:34:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:34:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:34:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:34:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964298079.06G/439.78G) 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192688.43 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 219268.84 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219268.843413 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973419.59 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 197341.96 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416610.802484 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776077.63 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 177607.76 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594218.565649 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598469.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 159846.99 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754065.552497 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438622.88 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 143862.29 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897927.840660 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294760.59 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 129476.06 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027403.900007 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165284.53 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 116528.45 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143932.353419 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048756.08 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 104875.61 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248807.961490 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943880.47 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 94388.05 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343196.008754 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849492.43 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 84949.24 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428145.251291 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764543.18 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 76454.32 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504599.569575 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688088.86 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 68808.89 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573408.456030 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619279.98 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 61928.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635336.453840 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557351.98 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 55735.20 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691071.651869 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501616.78 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 50161.68 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741233.330095 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451455.10 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 45145.51 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786378.840498 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406309.59 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 40630.96 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1827009.799861 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365678.63 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 36567.86 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863577.663288 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329110.77 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 32911.08 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896488.740372 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296199.69 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 29619.97 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926108.709747 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266579.72 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 26657.97 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952766.682185 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239921.75 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 23992.18 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976758.857380 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215929.58 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 21592.96 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998351.815055 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194336.62 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 19433.66 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017785.476962 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174902.96 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 17490.30 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035275.772678 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157412.66 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 15741.27 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2051017.038823 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141671.40 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 14167.14 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065184.178354 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127504.26 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 12750.43 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077934.603931 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114753.83 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 11475.38 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089409.986951 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103278.45 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 10327.84 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099737.831669 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92950.60 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 9295.06 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2109032.891915 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83655.54 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 8365.55 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117398.446136 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75289.99 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 7529.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124927.444935 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67760.99 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 6776.10 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131703.543854 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60984.89 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 6098.49 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137802.032882 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54886.40 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 5488.64 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143290.673006 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49397.76 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 4939.78 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148230.449119 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44457.99 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 4445.80 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152676.247620 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40012.19 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 4001.22 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156677.466270 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36010.97 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3601.10 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160278.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32409.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163878.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28809.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167478.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25209.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171078.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21609.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174678.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18009.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178278.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14409.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181878.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10809.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185478.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7209.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189078.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3609.87 sec) 1 day 00:34:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192678.563056 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 9.87 sec) 1 day 00:34:00 [GPUGRID] [rr_sim] 2192688.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964298079.06G/439.78G) 1 day 00:34:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854169.35 1 day 00:34:00 [---] pick_jobs_to_run() start 1 day 00:34:00 [---] pick_jobs_to_run() end 1 day 00:34:00 [---] [rr_sim] end 1 day 00:34:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:34:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:34:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:34:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:34:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:34:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:34:00 [---] [cpu_sched_debug] final job list: 1 day 00:34:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:34:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:34:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:34:00 [---] choose_project(): 88440.000000 1 day 00:34:00 [WUProp@Home] doing RPC to get work 1 day 00:34:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347159.09 cpu 2097032.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2097032.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2097032.90 1 day 00:34:00 [---] Got 0 tasks 1 day 00:35:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:35:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:35:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:35:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964270488.71G/439.78G) 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192625.70 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 219262.57 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219262.569725 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973363.13 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 197336.31 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416598.882477 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1776026.81 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 177602.68 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594201.563954 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598424.13 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 159842.41 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754043.977283 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438581.72 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 143858.17 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897902.149279 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294723.55 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 129472.35 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027374.504076 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165251.19 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 116525.12 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143899.623393 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048726.07 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 104872.61 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248772.230778 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943853.47 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 94385.35 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343157.577425 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849468.12 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 84946.81 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428104.389407 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764521.31 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 76452.13 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504556.520191 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688069.18 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 68806.92 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573363.437896 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619262.26 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 61926.23 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635289.663831 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557336.03 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 55733.60 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1691023.267173 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501602.43 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 50160.24 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741183.510180 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451442.19 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 45144.22 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786327.728887 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406297.97 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 40629.80 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826957.525723 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365668.17 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 36566.82 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863524.342875 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329101.35 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 32910.14 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896434.478312 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296191.22 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 29619.12 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1926053.600205 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266572.10 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 26657.21 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952710.809909 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239914.89 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 23991.49 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976702.298643 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215923.40 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 21592.34 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998294.638503 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194331.06 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 19433.11 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017727.744378 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174897.95 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 17489.80 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035217.539664 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157408.16 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 15740.82 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050958.355423 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141667.34 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 14166.73 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065125.089605 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127500.61 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 12750.06 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077875.150369 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114750.55 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 11475.05 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089350.205057 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103275.49 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 10327.55 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099677.754276 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92947.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 9294.79 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108972.548573 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83653.15 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 8365.31 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117337.863440 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75287.83 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 7528.78 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124866.646821 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67759.05 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 6775.91 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131642.551863 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60983.15 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 6098.31 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137740.866401 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54884.83 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 5488.48 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143229.349486 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49396.35 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 4939.63 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148168.984262 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44456.71 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 4445.67 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152614.655560 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40011.04 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 4001.10 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156615.759729 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36009.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.99 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160216.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32408.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163816.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28808.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167416.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25208.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2171016.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21608.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174616.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18008.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178216.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14408.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181816.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10808.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185416.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7208.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2189016.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3608.94 sec) 1 day 00:35:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192616.753481 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8.94 sec) 1 day 00:35:00 [GPUGRID] [rr_sim] 2192625.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964270488.71G/439.78G) 1 day 00:35:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854166.61 1 day 00:35:00 [---] pick_jobs_to_run() start 1 day 00:35:00 [---] pick_jobs_to_run() end 1 day 00:35:00 [---] [rr_sim] end 1 day 00:35:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:35:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:35:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:35:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:35:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:35:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:35:00 [---] [cpu_sched_debug] final job list: 1 day 00:35:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:35:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:35:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:35:00 [---] choose_project(): 88500.000000 1 day 00:35:00 [WUProp@Home] doing RPC to get work 1 day 00:35:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347099.09 cpu 2096972.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096972.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096972.90 1 day 00:35:00 [---] Got 0 tasks 1 day 00:36:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:36:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:36:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:36:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964242898.35G/439.78G) 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192562.96 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 219256.30 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219256.296036 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973306.66 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 197330.67 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416586.962469 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775976.00 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 177597.60 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594184.562259 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598378.40 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 159837.84 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754022.402069 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438540.56 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 143854.06 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897876.457899 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294686.50 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 129468.65 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027345.108145 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165217.85 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 116521.79 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143866.893367 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048696.07 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 104869.61 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248736.500067 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943826.46 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 94382.65 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343119.146096 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849443.81 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 84944.38 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428063.527523 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764499.43 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 76449.94 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504513.470807 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688049.49 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 68804.95 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573318.419763 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619244.54 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 61924.45 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635242.873823 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557320.09 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 55732.01 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690974.882477 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501588.08 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 50158.81 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741133.690266 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451429.27 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 45142.93 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786276.617275 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406286.34 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 40628.63 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826905.251584 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365657.71 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 36565.77 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863471.022462 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329091.94 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 32909.19 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896380.216252 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296182.74 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 29618.27 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925998.490664 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266564.47 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 26656.45 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952654.937634 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239908.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 23990.80 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976645.739907 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215917.22 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 21591.72 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998237.461952 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194325.50 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 19432.55 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017670.011793 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174892.95 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 17489.29 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035159.306650 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157403.65 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 15740.37 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050899.672022 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141663.29 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 14166.33 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065066.000856 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127496.96 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 12749.70 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077815.696807 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114747.26 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 11474.73 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089290.423162 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103272.54 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 10327.25 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099617.676883 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92945.28 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 9294.53 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108912.205231 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83650.76 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 8365.08 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117277.280744 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75285.68 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 7528.57 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124805.848706 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67757.11 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 6775.71 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131581.559872 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60981.40 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 6098.14 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137679.699921 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54883.26 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 5488.33 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143168.025965 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49394.93 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 4939.49 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148107.519405 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44455.44 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 4445.54 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152553.063501 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40009.90 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 4000.99 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156554.053187 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36008.91 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.89 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160154.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32408.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163754.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28808.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167354.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25208.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170954.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21608.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174554.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18008.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178154.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14408.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181754.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10808.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185354.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7208.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188954.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3608.02 sec) 1 day 00:36:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192554.943905 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 8.02 sec) 1 day 00:36:00 [GPUGRID] [rr_sim] 2192562.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964242898.35G/439.78G) 1 day 00:36:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854163.87 1 day 00:36:00 [---] pick_jobs_to_run() start 1 day 00:36:00 [---] pick_jobs_to_run() end 1 day 00:36:00 [---] [rr_sim] end 1 day 00:36:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:36:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:36:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:36:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:36:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:36:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:36:00 [---] [cpu_sched_debug] final job list: 1 day 00:36:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:36:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:36:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:36:00 [---] choose_project(): 88560.000000 1 day 00:36:00 [WUProp@Home] doing RPC to get work 1 day 00:36:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 347039.09 cpu 2096912.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096912.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096912.90 1 day 00:36:00 [---] Got 0 tasks 1 day 00:37:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:37:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:37:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:37:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964215307.99G/439.78G) 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192500.22 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 219250.02 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219250.022348 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973250.20 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 197325.02 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416575.042461 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775925.18 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 177592.52 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594167.560564 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598332.66 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 159833.27 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 754000.826855 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438499.40 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 143849.94 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897850.766518 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294649.46 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 129464.95 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027315.712214 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165184.51 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 116518.45 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143834.163341 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048666.06 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 104866.61 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248700.769355 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943799.45 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 94379.95 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343080.714768 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849419.51 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 84941.95 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1428022.665639 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764477.56 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 76447.76 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504470.421423 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688029.80 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 68802.98 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573273.401629 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619226.82 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 61922.68 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635196.083814 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557304.14 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 55730.41 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690926.497781 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501573.73 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 50157.37 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741083.870351 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451416.35 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 45141.64 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786225.505664 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406274.72 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 40627.47 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826852.977446 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365647.25 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 36564.72 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863417.702050 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329082.52 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 32908.25 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896325.954193 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296174.27 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 29617.43 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925943.381122 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266556.84 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 26655.68 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952599.065358 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239901.16 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 23990.12 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976589.181170 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215911.04 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 21591.10 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998180.285401 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194319.94 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 19431.99 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017612.279209 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174887.94 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 17488.79 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035101.073636 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157399.15 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 15739.91 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050840.988621 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141659.23 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 14165.92 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2065006.912107 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127493.31 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 12749.33 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077756.243244 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114743.98 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 11474.40 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089230.641268 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103269.58 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 10326.96 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099557.599490 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92942.62 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 9294.26 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108851.861889 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83648.36 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 8364.84 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117216.698048 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75283.53 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 7528.35 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124745.050591 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67755.17 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 6775.52 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131520.567880 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60979.66 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 6097.97 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137618.533441 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54881.69 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 5488.17 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143106.702445 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49393.52 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 4939.35 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2148046.054548 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44454.17 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 4445.42 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152491.471442 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40008.75 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 4000.88 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156492.346646 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36007.88 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.79 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160093.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32407.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163693.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28807.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167293.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25207.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170893.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21607.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174493.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18007.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178093.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14407.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181693.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10807.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185293.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7207.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188893.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3607.09 sec) 1 day 00:37:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192493.134329 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7.09 sec) 1 day 00:37:00 [GPUGRID] [rr_sim] 2192500.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964215307.99G/439.78G) 1 day 00:37:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854161.14 1 day 00:37:00 [---] pick_jobs_to_run() start 1 day 00:37:00 [---] pick_jobs_to_run() end 1 day 00:37:00 [---] [rr_sim] end 1 day 00:37:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:37:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:37:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:37:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:37:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:37:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:37:00 [---] [cpu_sched_debug] final job list: 1 day 00:37:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:37:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:37:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:37:00 [---] choose_project(): 88620.000000 1 day 00:37:00 [WUProp@Home] doing RPC to get work 1 day 00:37:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346979.09 cpu 2096852.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096852.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096852.90 1 day 00:37:00 [---] Got 0 tasks 1 day 00:38:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:38:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:38:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:38:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964187717.64G/439.78G) 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192437.49 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 219243.75 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219243.748660 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973193.74 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 197319.37 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416563.122454 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775874.36 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 177587.44 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594150.558868 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598286.93 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 159828.69 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753979.251642 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438458.23 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 143845.82 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897825.075137 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294612.41 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 129461.24 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027286.316284 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165151.17 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 116515.12 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143801.433315 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048636.05 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 104863.61 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248665.038644 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943772.45 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 94377.24 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343042.283439 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849395.20 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 84939.52 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427981.803755 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764455.68 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 76445.57 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504427.372040 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 688010.11 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 68801.01 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573228.383496 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619209.10 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 61920.91 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635149.293806 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557288.19 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 55728.82 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690878.113085 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501559.37 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 50155.94 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1741034.050437 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451403.44 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 45140.34 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786174.394053 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406263.09 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 40626.31 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826800.703308 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365636.78 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 36563.68 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863364.381637 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329073.10 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 32907.31 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896271.692133 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296165.79 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 29616.58 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925888.271580 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266549.22 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 26654.92 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952543.193082 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239894.29 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 23989.43 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976532.622433 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215904.86 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 21590.49 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998123.108850 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194314.38 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 19431.44 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017554.546625 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174882.94 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 17488.29 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2035042.840622 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157394.65 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 15739.46 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050782.305220 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141655.18 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 14165.52 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064947.823358 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127489.66 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 12748.97 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077696.789682 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114740.70 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 11474.07 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089170.859374 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103266.63 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 10326.66 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099497.522096 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92939.96 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 9294.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108791.518547 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83645.97 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 8364.60 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117156.115352 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75281.37 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 7528.14 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124684.252477 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67753.23 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 6775.32 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131459.575889 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60977.91 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 6097.79 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137557.366960 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54880.12 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 5488.01 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2143045.378924 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49392.11 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 4939.21 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147984.589692 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44452.90 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 4445.29 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152429.879382 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40007.61 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 4000.76 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156430.640104 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36006.85 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.68 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2160031.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32406.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163631.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28806.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167231.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25206.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170831.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21606.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174431.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18006.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2178031.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14406.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181631.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10806.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185231.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7206.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188831.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3606.16 sec) 1 day 00:38:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192431.324754 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 6.16 sec) 1 day 00:38:00 [GPUGRID] [rr_sim] 2192437.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964187717.64G/439.78G) 1 day 00:38:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854158.40 1 day 00:38:00 [---] pick_jobs_to_run() start 1 day 00:38:00 [---] pick_jobs_to_run() end 1 day 00:38:00 [---] [rr_sim] end 1 day 00:38:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:38:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:38:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:38:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:38:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:38:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:38:00 [---] [cpu_sched_debug] final job list: 1 day 00:38:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:38:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:38:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:38:00 [---] choose_project(): 88680.000000 1 day 00:38:00 [WUProp@Home] doing RPC to get work 1 day 00:38:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346919.09 cpu 2096792.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096792.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096792.90 1 day 00:38:00 [---] Got 0 tasks 1 day 00:39:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:39:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:39:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:39:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964160127.28G/439.78G) 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192374.75 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 219237.47 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219237.474972 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973137.27 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 197313.73 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416551.202446 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775823.55 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 177582.35 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594133.557173 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598241.19 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 159824.12 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753957.676428 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438417.07 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 143841.71 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897799.383757 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294575.37 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 129457.54 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027256.920353 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165117.83 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 116511.78 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143768.703289 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048606.05 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 104860.60 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248629.307932 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943745.44 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 94374.54 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1343003.852111 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849370.90 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 84937.09 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427940.941871 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764433.81 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 76443.38 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504384.322656 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687990.43 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 68799.04 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573183.365362 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619191.38 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 61919.14 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635102.503797 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557272.25 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 55727.22 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690829.728389 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501545.02 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 50154.50 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740984.230522 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451390.52 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 45139.05 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786123.282442 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406251.47 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 40625.15 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826748.429169 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365626.32 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 36562.63 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863311.061224 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329063.69 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 32906.37 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896217.430073 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296157.32 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 29615.73 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925833.162038 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266541.59 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 26654.16 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952487.320806 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239887.43 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 23988.74 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976476.063697 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215898.69 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 21589.87 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998065.932299 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194308.82 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 19430.88 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017496.814041 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174877.94 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 17487.79 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034984.607608 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157390.14 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 15739.01 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050723.621819 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141651.13 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 14165.11 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064888.734609 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127486.02 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 12748.60 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077637.336120 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114737.41 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 11473.74 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089111.077480 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103263.67 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 10326.37 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099437.444703 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92937.31 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 9293.73 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108731.175205 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83643.57 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 8364.36 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117095.532656 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75279.22 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 7527.92 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124623.454362 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67751.30 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 6775.13 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131398.583898 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60976.17 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 6097.62 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137496.200480 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54878.55 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 5487.85 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142984.055403 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49390.69 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 4939.07 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147923.124835 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44451.62 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 4445.16 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152368.287323 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40006.46 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 4000.65 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156368.933562 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36005.82 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.58 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159969.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32405.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163569.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28805.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167169.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25205.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170769.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21605.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174369.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18005.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177969.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14405.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181569.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10805.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185169.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7205.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188769.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3605.23 sec) 1 day 00:39:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192369.515178 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 5.23 sec) 1 day 00:39:00 [GPUGRID] [rr_sim] 2192374.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964160127.28G/439.78G) 1 day 00:39:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854155.66 1 day 00:39:00 [---] pick_jobs_to_run() start 1 day 00:39:00 [---] pick_jobs_to_run() end 1 day 00:39:00 [---] [rr_sim] end 1 day 00:39:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:39:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:39:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:39:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:39:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:39:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:39:00 [---] [cpu_sched_debug] final job list: 1 day 00:39:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:39:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:39:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:39:00 [---] choose_project(): 88740.000000 1 day 00:39:00 [WUProp@Home] doing RPC to get work 1 day 00:39:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346859.09 cpu 2096732.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096732.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096732.90 1 day 00:39:00 [---] Got 0 tasks 1 day 00:40:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:40:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:40:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:40:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964132536.93G/439.78G) 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192312.01 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 219231.20 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219231.201284 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973080.81 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 197308.08 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416539.282439 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775772.73 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 177577.27 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594116.555478 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598195.46 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 159819.55 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753936.101214 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438375.91 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 143837.59 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897773.692376 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294538.32 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 129453.83 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027227.524422 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165084.49 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 116508.45 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143735.973263 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048576.04 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 104857.60 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248593.577220 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943718.44 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 94371.84 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342965.420782 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849346.59 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 84934.66 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427900.079987 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764411.93 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 76441.19 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504341.273272 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687970.74 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 68797.07 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573138.347228 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619173.67 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 61917.37 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635055.713789 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557256.30 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 55725.63 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690781.343694 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501530.67 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 50153.07 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740934.410608 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451377.60 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 45137.76 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786072.170830 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406239.84 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 40623.98 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826696.155031 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365615.86 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 36561.59 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863257.740811 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329054.27 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 32905.43 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896163.168014 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296148.84 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 29614.88 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925778.052496 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266533.96 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 26653.40 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952431.448530 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239880.56 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 23988.06 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976419.504960 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215892.51 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 21589.25 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1998008.755748 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194303.26 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 19430.33 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017439.081457 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174872.93 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 17487.29 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034926.374594 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157385.64 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 15738.56 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050664.938418 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141647.07 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 14164.71 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064829.645860 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127482.37 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 12748.24 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077577.882558 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114734.13 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 11473.41 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2089051.295585 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103260.72 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 10326.07 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099377.367310 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92934.65 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 9293.46 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108670.831863 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83641.18 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 8364.12 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2117034.949960 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75277.06 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 7527.71 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124562.656248 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67749.36 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 6774.94 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131337.591906 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60974.42 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 6097.44 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137435.033999 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54876.98 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 5487.70 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142922.731883 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49389.28 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 4938.93 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147861.659978 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44450.35 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 4445.04 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152306.695264 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40005.32 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 4000.53 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156307.227021 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36004.79 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.48 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159907.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32404.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163507.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28804.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167107.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25204.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170707.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21604.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174307.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18004.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177907.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14404.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181507.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10804.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185107.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7204.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188707.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3604.31 sec) 1 day 00:40:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192307.705602 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 4.31 sec) 1 day 00:40:00 [GPUGRID] [rr_sim] 2192312.01: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964132536.93G/439.78G) 1 day 00:40:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854152.93 1 day 00:40:00 [---] pick_jobs_to_run() start 1 day 00:40:00 [---] pick_jobs_to_run() end 1 day 00:40:00 [---] [rr_sim] end 1 day 00:40:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:40:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:40:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:40:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:40:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:40:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:40:00 [---] [cpu_sched_debug] final job list: 1 day 00:40:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:40:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:40:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:40:00 [---] choose_project(): 88800.000000 1 day 00:40:00 [WUProp@Home] doing RPC to get work 1 day 00:40:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346799.09 cpu 2096672.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096672.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096672.90 1 day 00:40:00 [---] Got 0 tasks 1 day 00:41:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:41:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:41:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:41:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964104946.57G/439.78G) 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192249.28 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 219224.93 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219224.927595 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1973024.35 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 197302.43 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416527.362431 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775721.91 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 177572.19 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594099.553783 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598149.72 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 159814.97 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753914.526000 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438334.75 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 143833.47 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897748.000995 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294501.27 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 129450.13 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027198.128491 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165051.15 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 116505.11 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143703.243237 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048546.03 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 104854.60 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248557.846509 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943691.43 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 94369.14 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342926.989453 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849322.29 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 84932.23 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427859.218103 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764390.06 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 76439.01 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504298.223888 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687951.05 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 68795.11 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573093.329095 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619155.95 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 61915.59 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1635008.923781 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557240.35 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 55724.04 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690732.958998 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501516.32 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 50151.63 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740884.590693 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451364.69 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 45136.47 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1786021.059219 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406228.22 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 40622.82 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826643.880893 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365605.40 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 36560.54 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863204.420399 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329044.86 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 32904.49 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896108.905954 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296140.37 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 29614.04 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925722.942954 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266526.33 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 26652.63 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952375.576254 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239873.70 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 23987.37 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976362.946224 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215886.33 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 21588.63 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997951.579197 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194297.70 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 19429.77 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017381.348872 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174867.93 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 17486.79 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034868.141580 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157381.13 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 15738.11 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050606.255018 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141643.02 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 14164.30 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064770.557111 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127478.72 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 12747.87 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077518.428995 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114730.85 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 11473.08 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088991.513691 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103257.76 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 10325.78 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099317.289917 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92931.99 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 9293.20 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108610.488521 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83638.79 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 8363.88 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116974.367264 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75274.91 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 7527.49 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124501.858133 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67747.42 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 6774.74 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131276.599915 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60972.68 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 6097.27 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137373.867519 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54875.41 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 5487.54 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142861.408362 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49387.87 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 4938.79 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147800.195121 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44449.08 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 4444.91 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152245.103204 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40004.17 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 4000.42 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156245.520479 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36003.76 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.38 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159845.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32403.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163445.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28803.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2167045.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25203.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170645.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21603.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174245.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18003.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177845.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14403.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181445.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10803.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2185045.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7203.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188645.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3603.38 sec) 1 day 00:41:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192245.896027 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3.38 sec) 1 day 00:41:00 [GPUGRID] [rr_sim] 2192249.28: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964104946.57G/439.78G) 1 day 00:41:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854150.19 1 day 00:41:00 [---] pick_jobs_to_run() start 1 day 00:41:00 [---] pick_jobs_to_run() end 1 day 00:41:00 [---] [rr_sim] end 1 day 00:41:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:41:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:41:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:41:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:41:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:41:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:41:00 [---] [cpu_sched_debug] final job list: 1 day 00:41:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:41:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:41:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:41:00 [---] choose_project(): 88860.000000 1 day 00:41:00 [WUProp@Home] doing RPC to get work 1 day 00:41:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346739.09 cpu 2096612.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096612.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096612.90 1 day 00:41:00 [---] Got 0 tasks 1 day 00:44:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:44:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:44:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:44:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964077356.21G/439.78G) 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192186.54 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 219218.65 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219218.653907 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972967.89 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 197296.79 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416515.442423 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775671.10 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 177567.11 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594082.552088 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598103.99 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 159810.40 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753892.950786 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438293.59 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 143829.36 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897722.309615 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294464.23 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 129446.42 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027168.732560 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1165017.81 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 116501.78 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143670.513211 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048516.03 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 104851.60 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248522.115797 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943664.42 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 94366.44 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342888.558125 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849297.98 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 84929.80 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427818.356219 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764368.18 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 76436.82 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504255.174504 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687931.36 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 68793.14 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573048.310961 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619138.23 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 61913.82 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634962.133772 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557224.41 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 55722.44 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690684.574302 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501501.96 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 50150.20 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740834.770779 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451351.77 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 45135.18 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785969.947608 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406216.59 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 40621.66 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826591.606754 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365594.93 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 36559.49 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863151.099986 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329035.44 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 32903.54 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896054.643894 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296131.90 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 29613.19 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925667.833412 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266518.71 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 26651.87 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952319.703978 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239866.84 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 23986.68 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976306.387487 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215880.15 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 21588.02 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997894.402646 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194292.14 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 19429.21 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017323.616288 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174862.92 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 17486.29 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034809.908566 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157376.63 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 15737.66 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050547.571617 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141638.97 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 14163.90 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064711.468362 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127475.07 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 12747.51 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077458.975433 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114727.56 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 11472.76 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088931.731797 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103254.81 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 10325.48 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099257.212524 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92929.33 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 9292.93 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108550.145179 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83636.39 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 8363.64 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116913.784568 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75272.75 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 7527.28 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124441.060018 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67745.48 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 6774.55 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131215.607924 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60970.93 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 6097.09 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137312.701038 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54873.84 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 5487.38 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142800.084842 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49386.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 4938.65 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147738.730264 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44447.81 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 4444.78 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152183.511145 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40003.03 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 4000.30 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156183.813938 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36002.73 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.27 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159784.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32402.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163384.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28802.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166984.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25202.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170584.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21602.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174184.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18002.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177784.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14402.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181384.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10802.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184984.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7202.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188584.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3602.45 sec) 1 day 00:44:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192184.086451 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2.45 sec) 1 day 00:44:00 [GPUGRID] [rr_sim] 2192186.54: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964077356.21G/439.78G) 1 day 00:44:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854267.45 1 day 00:44:00 [---] pick_jobs_to_run() start 1 day 00:44:00 [---] pick_jobs_to_run() end 1 day 00:44:00 [---] [rr_sim] end 1 day 00:44:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:44:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:44:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:44:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:44:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:44:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:44:00 [---] [cpu_sched_debug] final job list: 1 day 00:44:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:44:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:44:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:44:00 [---] choose_project(): 89040.000000 1 day 00:44:00 [WUProp@Home] doing RPC to get work 1 day 00:44:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346559.09 cpu 2096552.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096552.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096552.90 1 day 00:44:00 [---] Got 0 tasks 1 day 00:45:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:45:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:45:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:45:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964049765.86G/439.78G) 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192123.80 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 219212.38 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219212.380219 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972911.42 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 197291.14 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416503.522416 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775620.28 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 177562.03 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594065.550393 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598058.25 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 159805.83 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753871.375573 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438252.43 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 143825.24 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897696.618234 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294427.18 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 129442.72 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027139.336630 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164984.47 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 116498.45 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143637.783186 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048486.02 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 104848.60 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248486.385086 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943637.42 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 94363.74 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342850.126796 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849273.68 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 84927.37 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427777.494335 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764346.31 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 76434.63 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504212.125121 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687911.68 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 68791.17 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1573003.292827 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619120.51 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 61912.05 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634915.343764 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557208.46 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 55720.85 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690636.189606 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501487.61 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 50148.76 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740784.950864 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451338.85 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 45133.89 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785918.835997 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406204.97 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 40620.50 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826539.332616 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365584.47 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 36558.45 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863097.779573 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329026.02 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 32902.60 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1896000.381835 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296123.42 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 29612.34 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925612.723870 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266511.08 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 26651.11 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952263.831702 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239859.97 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 23986.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976249.828751 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215873.97 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 21587.40 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997837.226094 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194286.58 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 19428.66 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017265.883704 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174857.92 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 17485.79 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034751.675552 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157372.13 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 15737.21 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050488.888216 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141634.91 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 14163.49 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064652.379613 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127471.42 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 12747.14 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077399.521871 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114724.28 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 11472.43 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088871.949903 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103251.85 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 10325.19 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099197.135131 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92926.67 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 9292.67 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108489.801837 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83634.00 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 8363.40 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116853.201872 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75270.60 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 7527.06 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124380.261904 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67743.54 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 6774.35 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131154.615932 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60969.19 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 6096.92 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137251.534558 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54872.27 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 5487.23 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142738.761321 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49385.04 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 4938.50 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147677.265408 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44446.54 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 4444.65 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152121.919086 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40001.88 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 4000.19 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156122.107396 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36001.69 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.17 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159722.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32401.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163322.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28801.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166922.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25201.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170522.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21601.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174122.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18001.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177722.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14401.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181322.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10801.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184922.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7201.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188522.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3601.53 sec) 1 day 00:45:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192122.276875 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1.53 sec) 1 day 00:45:00 [GPUGRID] [rr_sim] 2192123.80: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964049765.86G/439.78G) 1 day 00:45:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854264.71 1 day 00:45:00 [---] pick_jobs_to_run() start 1 day 00:45:00 [---] pick_jobs_to_run() end 1 day 00:45:00 [---] [rr_sim] end 1 day 00:45:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:45:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:45:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:45:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:45:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:45:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:45:00 [---] [cpu_sched_debug] final job list: 1 day 00:45:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:45:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:45:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:45:00 [---] choose_project(): 89100.000000 1 day 00:45:00 [WUProp@Home] doing RPC to get work 1 day 00:45:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346499.09 cpu 2096492.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096492.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096492.90 1 day 00:45:00 [---] Got 0 tasks 1 day 00:46:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:46:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:46:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:46:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (964022175.50G/439.78G) 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2192061.07 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 219206.11 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219206.106531 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972854.96 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 197285.50 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416491.602408 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775569.46 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 177556.95 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594048.548698 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1598012.52 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 159801.25 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753849.800359 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438211.26 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 143821.13 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897670.926854 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294390.14 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 129439.01 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027109.940699 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164951.12 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 116495.11 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143605.053160 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048456.01 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 104845.60 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248450.654374 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943610.41 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 94361.04 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342811.695468 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849249.37 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 84924.94 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427736.632451 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764324.43 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 76432.44 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504169.075737 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687891.99 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 68789.20 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572958.274694 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619102.79 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 61910.28 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634868.553755 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557192.51 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 55719.25 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690587.804910 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501473.26 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 50147.33 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740735.130950 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451325.93 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 45132.59 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785867.724386 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406193.34 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 40619.33 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826487.058478 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365574.01 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 36557.40 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1863044.459160 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329016.61 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 32901.66 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895946.119775 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296114.95 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 29611.49 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925557.614328 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266503.45 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 26650.35 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952207.959426 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239853.11 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 23985.31 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976193.270014 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215867.80 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 21586.78 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997780.049543 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194281.02 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 19428.10 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017208.151120 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174852.91 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 17485.29 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034693.442538 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157367.62 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 15736.76 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050430.204815 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141630.86 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 14163.09 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064593.290864 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127467.77 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 12746.78 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077340.068308 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114721.00 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 11472.10 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088812.168008 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103248.90 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 10324.89 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099137.057738 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92924.01 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 9292.40 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108429.458495 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83631.61 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 8363.16 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116792.619176 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75268.45 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 7526.84 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124319.463789 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67741.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 6774.16 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131093.623941 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60967.44 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 6096.74 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137190.368077 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54870.70 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 5487.07 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142677.437800 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49383.63 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 4938.36 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147615.800551 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44445.26 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 4444.53 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2152060.327026 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 40000.74 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 4000.07 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2156060.400854 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 36000.66 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.07 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159660.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32400.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163260.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28800.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166860.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25200.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170460.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21600.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2174060.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 18000.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177660.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14400.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181260.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10800.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184860.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7200.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188460.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3600.60 sec) 1 day 00:46:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2192060.467300 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 0.60 sec) 1 day 00:46:00 [GPUGRID] [rr_sim] 2192061.07: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (964022175.50G/439.78G) 1 day 00:46:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854261.98 1 day 00:46:00 [---] pick_jobs_to_run() start 1 day 00:46:00 [---] pick_jobs_to_run() end 1 day 00:46:00 [---] [rr_sim] end 1 day 00:46:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:46:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:46:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:46:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:46:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:46:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:46:00 [---] [cpu_sched_debug] final job list: 1 day 00:46:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:46:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:46:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:46:00 [---] choose_project(): 89160.000000 1 day 00:46:00 [WUProp@Home] doing RPC to get work 1 day 00:46:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346439.09 cpu 2096432.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096432.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096432.90 1 day 00:46:00 [---] Got 0 tasks 1 day 00:47:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:47:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:47:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:47:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963994585.14G/439.78G) 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191998.33 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 219199.83 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219199.832842 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972798.50 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 197279.85 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416479.682401 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775518.65 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 177551.86 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594031.547003 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597966.78 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 159796.68 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753828.225145 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438170.10 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 143817.01 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897645.235473 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294353.09 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 129435.31 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027080.544768 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164917.78 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 116491.78 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143572.323134 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048426.01 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 104842.60 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248414.923663 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943583.40 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 94358.34 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342773.264139 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849225.06 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 84922.51 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427695.770567 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764302.56 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 76430.26 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504126.026353 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687872.30 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 68787.23 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572913.256560 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619085.07 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 61908.51 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634821.763747 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557176.56 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 55717.66 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690539.420214 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501458.91 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 50145.89 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740685.311035 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451313.02 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 45131.30 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785816.612774 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406181.72 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 40618.17 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826434.784339 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365563.54 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 36556.35 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862991.138748 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 329007.19 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 32900.72 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895891.857715 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296106.47 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 29610.65 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925502.504786 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266495.82 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 26649.58 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952152.087150 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239846.24 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 23984.62 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976136.711278 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215861.62 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 21586.16 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997722.872992 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194275.46 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 19427.55 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017150.418535 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174847.91 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 17484.79 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034635.209524 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157363.12 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 15736.31 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050371.521414 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141626.81 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 14162.68 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064534.202115 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127464.13 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 12746.41 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077280.614746 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114717.71 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 11471.77 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088752.386114 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103245.94 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 10324.59 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099076.980345 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92921.35 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 9292.13 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108369.115153 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83629.21 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 8362.92 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116732.036480 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75266.29 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 7526.63 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124258.665674 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67739.66 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 6773.97 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2131032.631949 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60965.70 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 6096.57 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137129.201597 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54869.13 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 5486.91 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142616.114280 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49382.21 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 4938.22 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147554.335694 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44443.99 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 4444.40 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151998.734967 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39999.59 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3999.96 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155998.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35999.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159598.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32399.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163198.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28799.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166798.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25199.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170398.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21599.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173998.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17999.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177598.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14399.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181198.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10799.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184798.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7199.63 sec) 1 day 00:47:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188398.694313 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3599.63 sec) 1 day 00:47:00 [GPUGRID] [rr_sim] 2191998.33: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963994585.14G/439.78G) 1 day 00:47:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854259.24 1 day 00:47:00 [---] pick_jobs_to_run() start 1 day 00:47:00 [---] pick_jobs_to_run() end 1 day 00:47:00 [---] [rr_sim] end 1 day 00:47:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:47:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:47:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:47:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:47:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:47:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:47:00 [---] [cpu_sched_debug] final job list: 1 day 00:47:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:47:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:47:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:47:00 [---] choose_project(): 89220.000000 1 day 00:47:00 [WUProp@Home] doing RPC to get work 1 day 00:47:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346379.09 cpu 2096372.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096372.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096372.90 1 day 00:47:00 [---] Got 0 tasks 1 day 00:48:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:48:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:48:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:48:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963966994.79G/439.78G) 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191935.59 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 219193.56 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219193.559154 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972742.03 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 197274.20 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416467.762393 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775467.83 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 177546.78 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 594014.545308 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597921.05 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 159792.10 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753806.649931 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438128.94 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 143812.89 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897619.544092 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294316.05 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 129431.60 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027051.148837 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164884.44 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 116488.44 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143539.593108 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048396.00 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 104839.60 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248379.192951 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943556.40 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 94355.64 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342734.832810 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849200.76 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 84920.08 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427654.908684 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764280.68 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 76428.07 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504082.976969 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687852.61 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 68785.26 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572868.238427 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619067.35 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 61906.74 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634774.973738 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557160.62 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 55716.06 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690491.035519 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501444.56 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 50144.46 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740635.491121 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451300.10 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 45130.01 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785765.501163 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406170.09 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 40617.01 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826382.510201 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365553.08 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 36555.31 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862937.818335 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328997.77 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 32899.78 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895837.595656 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296098.00 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 29609.80 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925447.395244 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266488.20 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 26648.82 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952096.214874 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239839.38 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 23983.94 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976080.152541 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215855.44 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 21585.54 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997665.696441 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194269.90 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 19426.99 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017092.685951 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174842.91 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 17484.29 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034576.976510 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157358.62 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 15735.86 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050312.838013 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141622.75 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 14162.28 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064475.113366 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127460.48 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 12746.05 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077221.161184 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114714.43 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 11471.44 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088692.604220 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103242.99 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 10324.30 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2099016.902952 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92918.69 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 9291.87 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108308.771811 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83626.82 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 8362.68 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116671.453784 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75264.14 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 7526.41 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124197.867560 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67737.72 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 6773.77 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130971.639958 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60963.95 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 6096.40 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137068.035116 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54867.56 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 5486.76 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142554.790759 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49380.80 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 4938.08 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147492.870837 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44442.72 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 4444.27 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151937.142908 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39998.45 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3999.84 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155936.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35998.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159536.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32398.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163136.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28798.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166736.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25198.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170336.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21598.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173936.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17998.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177536.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14398.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181136.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10798.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184736.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7198.60 sec) 1 day 00:48:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188336.987771 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3598.60 sec) 1 day 00:48:00 [GPUGRID] [rr_sim] 2191935.59: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963966994.79G/439.78G) 1 day 00:48:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854256.50 1 day 00:48:00 [---] pick_jobs_to_run() start 1 day 00:48:00 [---] pick_jobs_to_run() end 1 day 00:48:00 [---] [rr_sim] end 1 day 00:48:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:48:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:48:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:48:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:48:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:48:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:48:00 [---] [cpu_sched_debug] final job list: 1 day 00:48:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:48:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:48:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:48:00 [---] choose_project(): 89280.000000 1 day 00:48:00 [WUProp@Home] doing RPC to get work 1 day 00:48:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346319.09 cpu 2096312.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096312.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096312.90 1 day 00:48:00 [---] Got 0 tasks 1 day 00:49:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:49:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:49:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:49:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963939404.43G/439.78G) 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191872.85 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 219187.29 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219187.285466 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972685.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 197268.56 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416455.842385 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775417.01 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 177541.70 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593997.543613 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597875.31 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 159787.53 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753785.074718 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438087.78 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 143808.78 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897593.852712 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294279.00 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 129427.90 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1027021.752907 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164851.10 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 116485.11 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143506.863082 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048365.99 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 104836.60 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248343.462240 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943529.39 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 94352.94 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342696.401482 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849176.45 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 84917.65 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427614.046800 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764258.81 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 76425.88 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1504039.927586 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687832.93 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 68783.29 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572823.220293 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619049.63 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 61904.96 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634728.183730 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557144.67 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 55714.47 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690442.650823 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501430.20 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 50143.02 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740585.671206 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451287.18 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 45128.72 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785714.389552 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406158.47 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 40615.85 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826330.236063 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365542.62 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 36554.26 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862884.497922 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328988.36 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 32898.84 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895783.333596 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296089.52 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 29608.95 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925392.285702 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266480.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 26648.06 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1952040.342598 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239832.51 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 23983.25 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1976023.593804 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215849.26 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 21584.93 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997608.519890 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194264.33 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 19426.43 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2017034.953367 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174837.90 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 17483.79 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034518.743496 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157354.11 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 15735.41 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050254.154613 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141618.70 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 14161.87 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064416.024617 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127456.83 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 12745.68 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077161.707622 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114711.15 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 11471.11 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088632.822325 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103240.03 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 10324.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098956.825559 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92916.03 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 9291.60 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108248.428469 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83624.43 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 8362.44 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116610.871088 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75261.98 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 7526.20 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124137.069445 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67735.79 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 6773.58 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130910.647967 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60962.21 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 6096.22 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2137006.868636 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54865.99 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 5486.60 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142493.467238 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49379.39 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 4937.94 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147431.405981 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44441.45 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 4444.14 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151875.550849 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39997.30 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3999.73 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155875.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35997.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159475.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32397.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163075.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28797.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166675.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25197.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170275.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21597.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173875.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17997.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177475.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14397.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181075.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10797.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184675.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7197.57 sec) 1 day 00:49:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188275.281230 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3597.57 sec) 1 day 00:49:00 [GPUGRID] [rr_sim] 2191872.85: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963939404.43G/439.78G) 1 day 00:49:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854253.77 1 day 00:49:00 [---] pick_jobs_to_run() start 1 day 00:49:00 [---] pick_jobs_to_run() end 1 day 00:49:00 [---] [rr_sim] end 1 day 00:49:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:49:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:49:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:49:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:49:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:49:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:49:00 [---] [cpu_sched_debug] final job list: 1 day 00:49:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:49:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:49:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:49:00 [---] choose_project(): 89340.000000 1 day 00:49:00 [WUProp@Home] doing RPC to get work 1 day 00:49:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346259.09 cpu 2096252.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096252.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096252.90 1 day 00:49:00 [---] Got 0 tasks 1 day 00:50:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:50:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:50:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:50:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963911814.08G/439.78G) 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191810.12 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 219181.01 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219181.011778 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972629.11 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 197262.91 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416443.922378 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775366.20 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 177536.62 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593980.541918 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597829.58 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 159782.96 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753763.499504 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438046.62 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 143804.66 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897568.161331 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294241.96 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 129424.20 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026992.356976 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164817.76 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 116481.78 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143474.133056 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048335.98 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 104833.60 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248307.731528 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943502.39 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 94350.24 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342657.970153 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849152.15 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 84915.21 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427573.184916 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764236.93 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 76423.69 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503996.878202 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687813.24 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 68781.32 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572778.202159 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619031.92 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 61903.19 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634681.393721 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557128.72 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 55712.87 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690394.266127 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501415.85 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 50141.59 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740535.851292 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451274.27 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 45127.43 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785663.277941 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406146.84 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 40614.68 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826277.961924 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365532.16 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 36553.22 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862831.177510 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328978.94 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 32897.89 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895729.071536 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296081.05 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 29608.10 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925337.176161 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266472.94 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 26647.29 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951984.470322 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239825.65 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 23982.56 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975967.035068 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215843.08 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 21584.31 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997551.343339 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194258.77 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 19425.88 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016977.220783 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174832.90 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 17483.29 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034460.510482 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157349.61 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 15734.96 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050195.471212 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141614.65 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 14161.46 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064356.935868 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127453.18 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 12745.32 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077102.254059 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114707.86 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 11470.79 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088573.040431 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103237.08 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 10323.71 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098896.748166 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92913.37 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 9291.34 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108188.085127 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83622.03 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 8362.20 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116550.288392 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75259.83 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 7525.98 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124076.271331 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67733.85 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 6773.38 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130849.655975 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60960.46 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 6096.05 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136945.702156 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54864.42 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 5486.44 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142432.143718 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49377.97 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 4937.80 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147369.941124 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44440.18 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 4444.02 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151813.958789 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39996.16 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3999.62 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155813.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35996.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159413.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32396.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2163013.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28796.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166613.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25196.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170213.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21596.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173813.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17996.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177413.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14396.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2181013.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10796.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184613.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7196.54 sec) 1 day 00:50:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188213.574688 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3596.54 sec) 1 day 00:50:00 [GPUGRID] [rr_sim] 2191810.12: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963911814.08G/439.78G) 1 day 00:50:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854251.03 1 day 00:50:00 [---] pick_jobs_to_run() start 1 day 00:50:00 [---] pick_jobs_to_run() end 1 day 00:50:00 [---] [rr_sim] end 1 day 00:50:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:50:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:50:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:50:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:50:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:50:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:50:00 [---] [cpu_sched_debug] final job list: 1 day 00:50:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:50:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:50:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:50:00 [---] choose_project(): 89400.000000 1 day 00:50:00 [WUProp@Home] doing RPC to get work 1 day 00:50:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346199.09 cpu 2096192.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096192.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096192.90 1 day 00:50:00 [---] Got 0 tasks 1 day 00:51:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:51:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:51:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:51:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963884223.72G/439.78G) 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191747.38 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 219174.74 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219174.738090 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972572.64 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 197257.26 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416432.002370 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775315.38 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 177531.54 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593963.540223 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597783.84 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 159778.38 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753741.924290 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1438005.46 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 143800.55 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897542.469951 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294204.91 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 129420.49 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026962.961045 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164784.42 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 116478.44 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143441.403030 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048305.98 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 104830.60 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248272.000817 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943475.38 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 94347.54 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342619.538825 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849127.84 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 84912.78 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427532.323032 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764215.06 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 76421.51 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503953.828818 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687793.55 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 68779.36 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572733.184026 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 619014.20 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 61901.42 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634634.603713 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557112.78 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 55711.28 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690345.881431 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501401.50 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 50140.15 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740486.031378 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451261.35 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 45126.13 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785612.166329 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406135.21 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 40613.52 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826225.687786 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365521.69 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 36552.17 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862777.857097 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328969.52 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 32896.95 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895674.809477 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296072.57 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 29607.26 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925282.066619 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266465.31 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 26646.53 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951928.598046 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239818.78 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 23981.88 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975910.476331 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215836.90 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 21583.69 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997494.166788 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194253.21 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 19425.32 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016919.488198 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174827.89 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 17482.79 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034402.277468 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157345.10 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 15734.51 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050136.787811 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141610.59 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 14161.06 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064297.847119 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127449.53 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 12744.95 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2077042.800497 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114704.58 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 11470.46 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088513.258537 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103234.12 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 10323.41 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098836.670773 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92910.71 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 9291.07 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108127.741785 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83619.64 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 8361.96 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116489.705696 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75257.68 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 7525.77 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2124015.473216 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67731.91 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 6773.19 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130788.663984 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60958.72 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 6095.87 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136884.535675 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54862.85 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 5486.28 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142370.820197 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49376.56 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 4937.66 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147308.476267 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44438.90 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 4443.89 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151752.366730 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39995.01 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3999.50 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155751.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35995.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159351.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32395.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162951.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28795.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166551.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25195.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170151.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21595.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173751.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17995.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177351.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14395.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180951.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10795.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184551.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7195.51 sec) 1 day 00:51:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188151.868146 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3595.51 sec) 1 day 00:51:00 [GPUGRID] [rr_sim] 2191747.38: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963884223.72G/439.78G) 1 day 00:51:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854248.29 1 day 00:51:00 [---] pick_jobs_to_run() start 1 day 00:51:00 [---] pick_jobs_to_run() end 1 day 00:51:00 [---] [rr_sim] end 1 day 00:51:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:51:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:51:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:51:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:51:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:51:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:51:00 [---] [cpu_sched_debug] final job list: 1 day 00:51:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:51:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:51:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:51:00 [---] choose_project(): 89460.000000 1 day 00:51:00 [WUProp@Home] doing RPC to get work 1 day 00:51:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346139.09 cpu 2096132.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096132.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096132.90 1 day 00:51:00 [---] Got 0 tasks 1 day 00:52:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:52:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:52:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:52:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963856633.36G/439.78G) 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191684.64 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 219168.46 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219168.464401 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972516.18 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 197251.62 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416420.082363 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775264.56 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 177526.46 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593946.538528 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597738.11 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 159773.81 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753720.349076 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437964.29 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 143796.43 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897516.778570 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294167.87 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 129416.79 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026933.565114 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164751.08 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 116475.11 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143408.673004 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048275.97 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 104827.60 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248236.270105 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943448.37 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 94344.84 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342581.107496 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849103.54 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 84910.35 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427491.461148 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764193.18 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 76419.32 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503910.779434 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687773.86 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 68777.39 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572688.165892 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618996.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 61899.65 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634587.813704 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557096.83 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 55709.68 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690297.496735 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501387.15 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 50138.71 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740436.211463 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451248.43 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 45124.84 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785561.054718 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406123.59 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 40612.36 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826173.413648 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365511.23 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 36551.12 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862724.536684 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328960.11 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 32896.01 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895620.547417 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296064.10 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 29606.41 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925226.957077 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266457.69 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 26645.77 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951872.725770 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239811.92 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 23981.19 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975853.917595 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215830.73 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 21583.07 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997436.990237 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194247.65 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 19424.77 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016861.755614 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174822.89 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 17482.29 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034344.044454 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157340.60 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 15734.06 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050078.104410 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141606.54 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 14160.65 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064238.758370 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127445.89 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 12744.59 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076983.346935 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114701.30 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 11470.13 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088453.476643 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103231.17 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 10323.12 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098776.593380 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92908.05 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 9290.81 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108067.398443 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83617.25 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 8361.72 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116429.123000 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75255.52 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 7525.55 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123954.675101 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67729.97 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 6773.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130727.671993 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60956.97 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 6095.70 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136823.369195 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54861.27 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 5486.13 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142309.496677 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49375.15 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 4937.51 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147247.011410 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44437.63 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 4443.76 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151690.774671 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39993.87 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3999.39 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155690.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35994.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159290.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32394.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162890.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28794.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166490.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25194.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170090.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21594.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173690.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17994.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177290.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14394.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180890.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10794.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184490.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7194.48 sec) 1 day 00:52:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188090.161605 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3594.48 sec) 1 day 00:52:00 [GPUGRID] [rr_sim] 2191684.64: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963856633.36G/439.78G) 1 day 00:52:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854245.56 1 day 00:52:00 [---] pick_jobs_to_run() start 1 day 00:52:00 [---] pick_jobs_to_run() end 1 day 00:52:00 [---] [rr_sim] end 1 day 00:52:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:52:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:52:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:52:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:52:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:52:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:52:00 [---] [cpu_sched_debug] final job list: 1 day 00:52:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:52:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:52:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:52:00 [---] choose_project(): 89520.000000 1 day 00:52:00 [WUProp@Home] doing RPC to get work 1 day 00:52:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346079.09 cpu 2096072.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096072.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096072.90 1 day 00:52:00 [---] Got 0 tasks 1 day 00:53:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:53:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:53:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:53:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963829043.01G/439.78G) 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191621.91 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 219162.19 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219162.190713 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972459.72 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 197245.97 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416408.162355 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775213.74 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 177521.37 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593929.536833 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597692.37 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 159769.24 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753698.773862 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437923.13 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 143792.31 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897491.087189 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294130.82 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 129413.08 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026904.169184 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164717.74 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 116471.77 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143375.942978 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048245.96 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 104824.60 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248200.539394 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943421.37 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 94342.14 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342542.676167 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849079.23 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 84907.92 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427450.599264 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764171.31 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 76417.13 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503867.730050 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687754.18 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 68775.42 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572643.147759 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618978.76 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 61897.88 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634541.023696 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557080.88 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 55708.09 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690249.112039 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501372.80 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 50137.28 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740386.391549 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451235.52 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 45123.55 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785509.943107 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406111.96 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 40611.20 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826121.139509 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365500.77 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 36550.08 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862671.216271 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328950.69 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 32895.07 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895566.285357 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296055.62 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 29605.56 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925171.847535 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266450.06 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 26645.01 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951816.853494 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239805.05 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 23980.51 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975797.358858 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215824.55 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 21582.45 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997379.813685 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194242.09 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 19424.21 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016804.023030 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174817.88 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 17481.79 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034285.811440 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157336.10 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 15733.61 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2050019.421009 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141602.49 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 14160.25 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064179.669621 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127442.24 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 12744.22 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076923.893372 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114698.01 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 11469.80 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088393.694748 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103228.21 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 10322.82 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098716.515987 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92905.39 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 9290.54 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2108007.055101 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83614.85 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 8361.49 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116368.540304 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75253.37 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 7525.34 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123893.876987 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67728.03 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 6772.80 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130666.680001 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60955.23 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 6095.52 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136762.202714 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54859.70 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 5485.97 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142248.173156 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49373.73 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 4937.37 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147185.546553 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44436.36 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 4443.64 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151629.182611 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39992.72 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3999.27 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155628.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35993.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159228.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32393.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162828.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28793.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166428.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25193.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2170028.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21593.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173628.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17993.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177228.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14393.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180828.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10793.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184428.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7193.45 sec) 1 day 00:53:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2188028.455063 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3593.45 sec) 1 day 00:53:00 [GPUGRID] [rr_sim] 2191621.91: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963829043.01G/439.78G) 1 day 00:53:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854242.82 1 day 00:53:00 [---] pick_jobs_to_run() start 1 day 00:53:00 [---] pick_jobs_to_run() end 1 day 00:53:00 [---] [rr_sim] end 1 day 00:53:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:53:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:53:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:53:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:53:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:53:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:53:00 [---] [cpu_sched_debug] final job list: 1 day 00:53:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:53:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:53:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:53:00 [---] choose_project(): 89580.000000 1 day 00:53:00 [WUProp@Home] doing RPC to get work 1 day 00:53:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 346019.09 cpu 2096012.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2096012.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2096012.90 1 day 00:53:00 [---] Got 0 tasks 1 day 00:54:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:54:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:54:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:54:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963801452.65G/439.78G) 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191559.17 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 219155.92 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219155.917025 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972403.25 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 197240.33 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416396.242347 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775162.93 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 177516.29 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593912.535138 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597646.64 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 159764.66 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753677.198649 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437881.97 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 143788.20 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897465.395809 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294093.77 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 129409.38 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026874.773253 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164684.40 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 116468.44 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143343.212952 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048215.96 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 104821.60 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248164.808682 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943394.36 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 94339.44 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342504.244839 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849054.93 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 84905.49 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427409.737380 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764149.43 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 76414.94 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503824.680667 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687734.49 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 68773.45 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572598.129625 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618961.04 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 61896.10 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634494.233687 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557064.94 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 55706.49 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690200.727344 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501358.44 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 50135.84 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740336.571634 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451222.60 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 45122.26 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785458.831496 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406100.34 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 40610.03 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826068.865371 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365490.30 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 36549.03 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862617.895859 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328941.27 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 32894.13 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895512.023298 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296047.15 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 29604.71 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925116.737993 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266442.43 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 26644.24 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951760.981219 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239798.19 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 23979.82 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975740.800122 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215818.37 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 21581.84 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997322.637134 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194236.53 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 19423.65 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016746.290446 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174812.88 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 17481.29 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034227.578426 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157331.59 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 15733.16 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049960.737608 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141598.43 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 14159.84 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064120.580873 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127438.59 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 12743.86 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076864.439810 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114694.73 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 11469.47 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088333.912854 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103225.26 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 10322.53 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098656.438594 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92902.73 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 9290.27 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107946.711759 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83612.46 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 8361.25 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116307.957608 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75251.21 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 7525.12 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123833.078872 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67726.09 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 6772.61 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130605.688010 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60953.48 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 6095.35 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136701.036234 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54858.13 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 5485.81 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142186.849635 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49372.32 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 4937.23 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147124.081697 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44435.09 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 4443.51 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151567.590552 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39991.58 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3999.16 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155566.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35992.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159166.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32392.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162766.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28792.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166366.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25192.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169966.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21592.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173566.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17992.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177166.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14392.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180766.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10792.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184366.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7192.42 sec) 1 day 00:54:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187966.748522 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3592.42 sec) 1 day 00:54:00 [GPUGRID] [rr_sim] 2191559.17: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963801452.65G/439.78G) 1 day 00:54:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854240.08 1 day 00:54:00 [---] pick_jobs_to_run() start 1 day 00:54:00 [---] pick_jobs_to_run() end 1 day 00:54:00 [---] [rr_sim] end 1 day 00:54:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:54:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:54:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:54:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:54:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:54:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:54:00 [---] [cpu_sched_debug] final job list: 1 day 00:54:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:54:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:54:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:54:00 [---] choose_project(): 89640.000000 1 day 00:54:00 [WUProp@Home] doing RPC to get work 1 day 00:54:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 345959.09 cpu 2095952.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2095952.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2095952.90 1 day 00:54:00 [---] Got 0 tasks 1 day 00:55:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:55:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:55:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:55:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963773862.29G/439.78G) 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191496.43 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 219149.64 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219149.643337 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972346.79 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 197234.68 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416384.322340 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775112.11 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 177511.21 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593895.533442 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597600.90 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 159760.09 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753655.623435 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437840.81 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 143784.08 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897439.704428 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294056.73 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 129405.67 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026845.377322 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164651.06 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 116465.11 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143310.482926 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048185.95 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 104818.60 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248129.077971 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943367.36 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 94336.74 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342465.813510 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849030.62 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 84903.06 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427368.875496 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764127.56 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 76412.76 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503781.631283 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687714.80 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 68771.48 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572553.111491 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618943.32 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 61894.33 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634447.443679 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557048.99 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 55704.90 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690152.342648 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501344.09 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 50134.41 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740286.751720 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451209.68 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 45120.97 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785407.719884 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406088.71 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 40608.87 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1826016.591233 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365479.84 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 36547.98 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862564.575446 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328931.86 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 32893.19 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895457.761238 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296038.67 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 29603.87 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925061.628451 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266434.80 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 26643.48 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951705.108943 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239791.32 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 23979.13 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975684.241385 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215812.19 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 21581.22 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997265.460583 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194230.97 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 19423.10 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016688.557862 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174807.88 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 17480.79 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034169.345412 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157327.09 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 15732.71 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049902.054208 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141594.38 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 14159.44 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064061.492124 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127434.94 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 12743.49 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076804.986248 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114691.45 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 11469.14 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088274.130960 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103222.30 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 10322.23 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098596.361201 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92900.07 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 9290.01 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107886.368417 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83610.06 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 8361.01 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116247.374912 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75249.06 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 7524.91 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123772.280758 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67724.15 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 6772.42 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130544.696019 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60951.74 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 6095.17 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136639.869753 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54856.56 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 5485.66 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142125.526115 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49370.91 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 4937.09 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147062.616840 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44433.82 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 4443.38 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151505.998493 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39990.43 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3999.04 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155505.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35991.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159105.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32391.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162705.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28791.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166305.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25191.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169905.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21591.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173505.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17991.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177105.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14391.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180705.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10791.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184305.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7191.39 sec) 1 day 00:55:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187905.041980 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3591.39 sec) 1 day 00:55:00 [GPUGRID] [rr_sim] 2191496.43: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963773862.29G/439.78G) 1 day 00:55:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854237.35 1 day 00:55:00 [---] pick_jobs_to_run() start 1 day 00:55:00 [---] pick_jobs_to_run() end 1 day 00:55:00 [---] [rr_sim] end 1 day 00:55:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:55:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:55:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:55:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:55:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:55:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:55:00 [---] [cpu_sched_debug] final job list: 1 day 00:55:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:55:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:55:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:55:00 [---] choose_project(): 89700.000000 1 day 00:55:00 [WUProp@Home] doing RPC to get work 1 day 00:55:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 345899.09 cpu 2095892.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2095892.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2095892.90 1 day 00:55:00 [---] Got 0 tasks 1 day 00:56:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:56:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:56:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:56:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963746271.94G/439.78G) 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191433.70 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 219143.37 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219143.369648 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972290.33 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 197229.03 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416372.402332 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775061.29 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 177506.13 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593878.531747 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597555.16 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 159755.52 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753634.048221 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437799.65 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 143779.96 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897414.013047 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1294019.68 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 129401.97 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026815.981391 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164617.72 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 116461.77 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143277.752901 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048155.94 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 104815.59 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248093.347259 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943340.35 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 94334.03 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342427.382182 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 849006.31 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 84900.63 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427328.013612 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764105.68 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 76410.57 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503738.581899 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687695.11 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 68769.51 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572508.093358 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618925.60 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 61892.56 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634400.653670 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557033.04 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 55703.30 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690103.957952 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501329.74 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 50132.97 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740236.931805 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451196.76 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 45119.68 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785356.608273 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406077.09 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 40607.71 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825964.317094 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365469.38 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 36546.94 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862511.255033 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328922.44 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 32892.24 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895403.499178 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296030.20 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 29603.02 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1925006.518909 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266427.18 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 26642.72 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951649.236667 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239784.46 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 23978.45 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975627.682648 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215806.01 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 21580.60 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997208.284032 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194225.41 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 19422.54 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016630.825277 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174802.87 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 17480.29 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034111.112398 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157322.58 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 15732.26 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049843.370807 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141590.33 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 14159.03 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2064002.403375 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127431.29 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 12743.13 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076745.532686 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114688.16 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 11468.82 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088214.349066 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103219.35 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 10321.93 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098536.283807 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92897.41 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 9289.74 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107826.025075 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83607.67 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 8360.77 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116186.792216 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75246.90 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 7524.69 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123711.482643 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67722.21 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 6772.22 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130483.704027 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60949.99 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 6095.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136578.703273 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54854.99 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 5485.50 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142064.202594 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49369.49 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 4936.95 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2147001.151983 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44432.54 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 4443.25 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151444.406433 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39989.29 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3998.93 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155443.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35990.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2159043.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32390.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162643.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28790.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166243.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25190.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169843.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21590.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173443.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17990.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2177043.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14390.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180643.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10790.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184243.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7190.36 sec) 1 day 00:56:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187843.335438 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3590.36 sec) 1 day 00:56:00 [GPUGRID] [rr_sim] 2191433.70: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963746271.94G/439.78G) 1 day 00:56:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854234.61 1 day 00:56:00 [---] pick_jobs_to_run() start 1 day 00:56:00 [---] pick_jobs_to_run() end 1 day 00:56:00 [---] [rr_sim] end 1 day 00:56:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:56:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:56:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:56:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:56:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:56:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:56:00 [---] [cpu_sched_debug] final job list: 1 day 00:56:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:56:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:56:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:56:00 [---] choose_project(): 89760.000000 1 day 00:56:00 [WUProp@Home] doing RPC to get work 1 day 00:56:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 345839.09 cpu 2095832.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2095832.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2095832.90 1 day 00:56:00 [---] Got 0 tasks 1 day 00:57:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:57:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:57:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:57:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963718681.58G/439.78G) 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191370.96 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 219137.10 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219137.095960 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972233.86 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 197223.39 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416360.482324 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1775010.48 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 177501.05 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593861.530052 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597509.43 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 159750.94 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753612.473007 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437758.49 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 143775.85 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897388.321667 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1293982.64 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 129398.26 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026786.585460 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164584.37 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 116458.44 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143245.022875 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048125.94 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 104812.59 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248057.616547 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943313.34 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 94331.33 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342388.950853 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 848982.01 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 84898.20 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427287.151728 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764083.81 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 76408.38 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503695.532515 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687675.43 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 68767.54 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572463.075224 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618907.88 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 61890.79 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634353.863662 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557017.10 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 55701.71 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690055.573256 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501315.39 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 50131.54 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740187.111891 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451183.85 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 45118.38 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785305.496662 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406065.46 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 40606.55 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825912.042956 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365458.92 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 36545.89 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862457.934621 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328913.02 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 32891.30 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895349.237119 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296021.72 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 29602.17 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924951.409367 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266419.55 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 26641.96 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951593.364391 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239777.60 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 23977.76 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975571.123912 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215799.84 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 21579.98 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997151.107481 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194219.85 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 19421.99 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016573.092693 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174797.87 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 17479.79 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2034052.879384 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157318.08 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 15731.81 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049784.687406 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141586.27 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 14158.63 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063943.314626 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127427.64 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 12742.76 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076686.079123 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114684.88 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 11468.49 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088154.567171 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103216.39 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 10321.64 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098476.206414 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92894.75 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 9289.48 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107765.681733 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83605.28 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 8360.53 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116126.209520 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75244.75 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 7524.48 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123650.684528 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67720.28 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 6772.03 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130422.712036 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60948.25 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 6094.82 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136517.536792 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54853.42 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 5485.34 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2142002.879073 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49368.08 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 4936.81 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146939.687126 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44431.27 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 4443.13 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151382.814374 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39988.15 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3998.81 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155381.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35989.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158981.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32389.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162581.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28789.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166181.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25189.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169781.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21589.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173381.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17989.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176981.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14389.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180581.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10789.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184181.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7189.33 sec) 1 day 00:57:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187781.628897 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3589.33 sec) 1 day 00:57:00 [GPUGRID] [rr_sim] 2191370.96: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963718681.58G/439.78G) 1 day 00:57:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854231.87 1 day 00:57:00 [---] pick_jobs_to_run() start 1 day 00:57:00 [---] pick_jobs_to_run() end 1 day 00:57:00 [---] [rr_sim] end 1 day 00:57:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:57:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:57:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:57:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:57:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:57:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:57:00 [---] [cpu_sched_debug] final job list: 1 day 00:57:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:57:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:57:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:57:00 [---] choose_project(): 89820.000000 1 day 00:57:00 [WUProp@Home] doing RPC to get work 1 day 00:57:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 345779.09 cpu 2095772.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2095772.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2095772.90 1 day 00:57:00 [---] Got 0 tasks 1 day 00:58:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:58:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:58:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:58:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963691091.23G/439.78G) 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191308.22 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 219130.82 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219130.822272 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972177.40 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 197217.74 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416348.562317 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1774959.66 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 177495.97 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593844.528357 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597463.69 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 159746.37 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753590.897794 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437717.32 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 143771.73 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897362.630286 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1293945.59 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 129394.56 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026757.189530 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164551.03 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 116455.10 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143212.292849 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048095.93 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 104809.59 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1248021.885836 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943286.34 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 94328.63 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342350.519524 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 848957.70 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 84895.77 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427246.289844 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764061.93 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 76406.19 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503652.483132 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687655.74 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 68765.57 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572418.057091 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618890.17 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 61889.02 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634307.073653 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 557001.15 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 55700.11 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1690007.188560 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501301.03 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 50130.10 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740137.291976 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451170.93 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 45117.09 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785254.385051 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406053.84 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 40605.38 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825859.768818 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365448.45 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 36544.85 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862404.614208 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328903.61 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 32890.36 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895294.975059 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296013.25 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 29601.32 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924896.299825 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266411.92 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 26641.19 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951537.492115 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239770.73 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 23977.07 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975514.565175 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215793.66 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 21579.37 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997093.930930 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194214.29 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 19421.43 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016515.360109 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174792.86 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 17479.29 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033994.646370 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157313.58 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 15731.36 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049726.004005 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141582.22 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 14158.22 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063884.225877 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127424.00 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 12742.40 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076626.625561 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114681.60 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 11468.16 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088094.785277 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103213.44 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 10321.34 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098416.129021 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92892.09 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 9289.21 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107705.338391 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83602.88 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 8360.29 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116065.626824 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75242.60 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 7524.26 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123589.886414 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67718.34 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 6771.83 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130361.720044 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60946.50 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 6094.65 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136456.370312 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54851.85 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 5485.19 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141941.555553 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49366.67 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 4936.67 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146878.222270 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44430.00 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 4443.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151321.222315 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39987.00 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3998.70 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155319.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35988.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158919.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32388.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162519.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28788.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166119.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25188.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169719.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21588.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173319.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17988.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176919.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14388.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180519.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10788.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184119.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7188.30 sec) 1 day 00:58:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187719.922355 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3588.30 sec) 1 day 00:58:00 [GPUGRID] [rr_sim] 2191308.22: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963691091.23G/439.78G) 1 day 00:58:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854229.14 1 day 00:58:00 [---] pick_jobs_to_run() start 1 day 00:58:00 [---] pick_jobs_to_run() end 1 day 00:58:00 [---] [rr_sim] end 1 day 00:58:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:58:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:58:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:58:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:58:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:58:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:58:00 [---] [cpu_sched_debug] final job list: 1 day 00:58:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:58:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:58:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:58:00 [---] choose_project(): 89880.000000 1 day 00:58:00 [WUProp@Home] doing RPC to get work 1 day 00:58:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 345719.09 cpu 2095712.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2095712.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2095712.90 1 day 00:58:00 [---] Got 0 tasks 1 day 00:59:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 00:59:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 00:59:00 [---] [rr_sim] doing sim: CPU sched 1 day 00:59:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963663500.87G/439.78G) 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191245.49 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 219124.55 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219124.548584 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972120.94 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 197212.09 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416336.642309 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1774908.84 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 177490.88 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593827.526662 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597417.96 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 159741.80 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753569.322580 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437676.16 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 143767.62 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897336.938906 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1293908.55 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 129390.85 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026727.793599 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164517.69 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 116451.77 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143179.562823 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048065.92 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 104806.59 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1247986.155124 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943259.33 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 94325.93 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342312.088196 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 848933.40 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 84893.34 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427205.427960 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764040.06 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 76404.01 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503609.433748 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687636.05 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 68763.61 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572373.038957 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618872.45 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 61887.24 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634260.283645 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 556985.20 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 55698.52 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689958.803864 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501286.68 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 50128.67 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740087.472062 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451158.01 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 45115.80 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785203.273439 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406042.21 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 40604.22 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825807.494679 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365437.99 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 36543.80 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862351.293795 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328894.19 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 32889.42 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895240.712999 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 296004.77 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 29600.48 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924841.190283 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266404.30 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 26640.43 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951481.619839 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239763.87 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 23976.39 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975458.006439 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215787.48 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 21578.75 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1997036.754379 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194208.73 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 19420.87 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016457.627525 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174787.86 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 17478.79 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033936.413356 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157309.07 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 15730.91 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049667.320604 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141578.17 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 14157.82 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063825.137128 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127420.35 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 12742.03 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076567.171999 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114678.31 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 11467.83 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2088035.003383 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103210.48 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 10321.05 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098356.051628 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92889.43 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 9288.94 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107644.995049 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83600.49 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 8360.05 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2116005.044128 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75240.44 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 7524.04 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123529.088299 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67716.40 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 6771.64 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130300.728053 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60944.76 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 6094.48 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136395.203832 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54850.28 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 5485.03 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141880.232032 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49365.25 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 4936.53 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146816.757413 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44428.73 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 4442.87 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151259.630255 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39985.86 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3998.59 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155258.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35987.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158858.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32387.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162458.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28787.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2166058.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25187.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169658.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21587.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173258.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17987.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176858.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14387.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180458.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10787.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2184058.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7187.27 sec) 1 day 00:59:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187658.215814 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3587.27 sec) 1 day 00:59:00 [GPUGRID] [rr_sim] 2191245.49: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963663500.87G/439.78G) 1 day 00:59:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854226.40 1 day 00:59:00 [---] pick_jobs_to_run() start 1 day 00:59:00 [---] pick_jobs_to_run() end 1 day 00:59:00 [---] [rr_sim] end 1 day 00:59:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 00:59:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 00:59:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 00:59:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 00:59:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 00:59:00 [---] [cpu_sched_debug] preliminary job list: 1 day 00:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:59:00 [---] [cpu_sched_debug] final job list: 1 day 00:59:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 00:59:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 00:59:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 00:59:00 [---] choose_project(): 89940.000000 1 day 00:59:00 [WUProp@Home] doing RPC to get work 1 day 00:59:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 345659.09 cpu 2095652.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2095652.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2095652.90 1 day 00:59:00 [---] Got 0 tasks 1 day 01:00:00 [---] [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling 1 day 01:00:00 [---] [cpu_sched_debug] schedule_cpus(): start 1 day 01:00:00 [---] [rr_sim] doing sim: CPU sched 1 day 01:00:00 [---] [rr_sim] start: work_buf min 8640 additional 8640 total 17280 on_frac 0.970 active_frac 0.994 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim_detail] 0.00: starting e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (8.00 CPU + 1.00 NVIDIA GPU) (963635910.51G/439.78G) 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 0.000000 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 2191182.75 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 219118.27 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 219118.274896 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1972064.47 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 197206.45 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 416324.722302 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1774858.03 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 177485.80 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 593810.524967 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1597372.22 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 159737.22 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 753547.747366 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1437635.00 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 143763.50 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 897311.247525 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1293871.50 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 129387.15 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1026698.397668 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1164484.35 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 116448.44 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1143146.832797 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 1048035.92 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 104803.59 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1247950.424413 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 943232.32 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 94323.23 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1342273.656867 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 848909.09 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 84890.91 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1427164.566076 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 764018.18 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 76401.82 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1503566.384364 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 687616.36 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 68761.64 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1572328.020823 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 618854.73 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 61885.47 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1634213.493637 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 556969.26 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 55696.93 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1689910.419169 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 501272.33 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 50127.23 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1740037.652147 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 451145.10 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 45114.51 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1785152.161828 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 406030.59 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 40603.06 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1825755.220541 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 365427.53 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 36542.75 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1862297.973382 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 328884.78 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 32888.48 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1895186.450940 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 295996.30 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 29599.63 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1924786.080741 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 266396.67 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 26639.67 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1951425.747563 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 239757.00 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 23975.70 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1975401.447702 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 215781.30 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 21578.13 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 1996979.577828 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 194203.17 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 19420.32 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2016399.894940 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 174782.85 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 17478.29 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2033878.180342 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 157304.57 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 15730.46 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2049608.637203 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 141574.11 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 14157.41 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2063766.048379 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 127416.70 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 12741.67 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2076507.718436 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 114675.03 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 11467.50 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2087975.221488 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 103207.53 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 10320.75 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2098295.974235 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 92886.77 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 9288.68 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2107584.651707 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 83598.10 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 8359.81 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2115944.461432 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 75238.29 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 7523.83 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2123468.290185 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 67714.46 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 6771.45 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2130239.736062 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 60943.01 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 6094.30 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2136334.037351 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 54848.71 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 5484.87 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2141818.908512 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 49363.84 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 4936.38 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2146755.292556 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 44427.46 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 4442.75 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2151198.038196 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 39984.71 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3998.47 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2155196.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 35986.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2158796.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 32386.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2162396.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 28786.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2165996.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 25186.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2169596.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 21586.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2173196.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 17986.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2176796.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 14386.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2180396.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 10786.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2183996.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 7186.24 sec) 1 day 01:00:00 [---] [rrsim_detail] taking time-slice step of 3600.00 sec 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [GPUGRID] [rr_sim] at app max concurrent for PythonGPU; t 2187596.509272 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rrsim_detail] next job to finish: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (will finish in 3586.24 sec) 1 day 01:00:00 [GPUGRID] [rr_sim] 2191182.75: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 finishes (8.00 CPU + 1.00 NVIDIA GPU) (963635910.51G/439.78G) 1 day 01:00:00 [GPUGRID] [rr_sim] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses deadline by 1854223.66 1 day 01:00:00 [---] pick_jobs_to_run() start 1 day 01:00:00 [---] pick_jobs_to_run() end 1 day 01:00:00 [---] [rr_sim] end 1 day 01:00:00 [GPUGRID] [rr_sim] Result e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 projected to miss deadline. 1 day 01:00:00 [GPUGRID] [rr_sim] Project has 1 projected NVIDIA GPU deadline misses 1 day 01:00:00 [GPUGRID] [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA 1 day 01:00:00 [GPUGRID] [cpu_sched_debug] add to run list: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (NVIDIA GPU, FIFO) (prio -1.000000) 1 day 01:00:00 [---] [cpu_sched_debug] enforce_run_list(): start 1 day 01:00:00 [---] [cpu_sched_debug] preliminary job list: 1 day 01:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 01:00:00 [---] [cpu_sched_debug] final job list: 1 day 01:00:00 [GPUGRID] [cpu_sched_debug] 0: e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 (MD: no; UTS: no) 1 day 01:00:00 [GPUGRID] [cpu_sched_debug] scheduling e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 1 day 01:00:00 [---] [cpu_sched_debug] enforce_run_list: end 1 day 01:00:00 [---] choose_project(): 90000.000000 1 day 01:00:00 [WUProp@Home] doing RPC to get work 1 day 01:00:00 [WUProp@Home] RPC: [work_fetch] request: CPU (1.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst) [edf_detail] init_ip_results; work_buf_min 0.00 ncpus 8: [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0: deadline 345599.09 cpu 2095592.90 [edf_detail] mark_edf_misses [edf_detail] running e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 on cpu 0; finishes at 2095592.90 [edf_detail] e00007a00022-ABOU_rnd_ppod_expand_demos25_3-0-1-RND5255_0 misses_deadline; est completion 2095592.90 1 day 01:00:00 [---] Got 0 tasks